2024-09-17T21:18:29.094 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T21:18:29.110 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T21:18:29.470 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909352 branch: main description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} email: ceph-qa@ceph.com first_in_suite: false job_id: '7909352' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client: rbd cache: true rbd cache policy: writearound global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 08cec6974d86f81dc304360b31113f30b4085bf8 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 08cec6974d86f81dc304360b31113f30b4085bf8 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 5ecc7400dbbb63ce990b663d2a92e87dbc17f30d owner: scheduled_teuthology@teuthology priority: 950 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 4954 sha1: 08cec6974d86f81dc304360b31113f30b4085bf8 sleep_before_teardown: 0 subset: 16/128 suite: rbd suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_5ecc7400dbbb63ce990b663d2a92e87dbc17f30d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 5ecc7400dbbb63ce990b663d2a92e87dbc17f30d targets: smithi129.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBlWv4OLCblN2HmVxOdsCPfdJzju+eEOhxtK05GoDQrZU3Evcgpcndl85WagAwpxVwaCM1lfrIAuDXQh3sPgfC4= smithi171.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLNBgFzCjxfas+H7+r/VLB4Bz3W0KiEyLYSc3gPE5/Vc7TQhda/3+qG5jUiIgzsLXfCs6Ba3jPJMSb/J2J9aBTA= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_20:16:19 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T21:18:29.471 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_5ecc7400dbbb63ce990b663d2a92e87dbc17f30d/qa; will attempt to use it 2024-09-17T21:18:29.471 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_5ecc7400dbbb63ce990b663d2a92e87dbc17f30d/qa/tasks 2024-09-17T21:18:29.472 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T21:18:29.473 INFO:teuthology.task.internal:Checking packages... 2024-09-17T21:18:29.494 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '08cec6974d86f81dc304360b31113f30b4085bf8' 2024-09-17T21:18:29.494 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T21:18:29.494 INFO:teuthology.packaging:ref: None 2024-09-17T21:18:29.494 INFO:teuthology.packaging:tag: None 2024-09-17T21:18:29.494 INFO:teuthology.packaging:branch: main 2024-09-17T21:18:29.494 INFO:teuthology.packaging:sha1: 08cec6974d86f81dc304360b31113f30b4085bf8 2024-09-17T21:18:29.494 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-09-17T21:18:29.725 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4959.g2b698643 2024-09-17T21:18:29.727 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T21:18:29.734 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T21:18:29.734 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T21:18:29.757 INFO:teuthology.task.internal:Saving configuration 2024-09-17T21:18:29.769 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T21:18:29.776 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T21:18:29.800 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi129.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909352', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 21:13:14.054474', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBlWv4OLCblN2HmVxOdsCPfdJzju+eEOhxtK05GoDQrZU3Evcgpcndl85WagAwpxVwaCM1lfrIAuDXQh3sPgfC4='} 2024-09-17T21:18:29.820 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909352', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 21:13:14.055643', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLNBgFzCjxfas+H7+r/VLB4Bz3W0KiEyLYSc3gPE5/Vc7TQhda/3+qG5jUiIgzsLXfCs6Ba3jPJMSb/J2J9aBTA='} 2024-09-17T21:18:29.820 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T21:18:29.827 INFO:teuthology.task.internal:roles: ubuntu@smithi129.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-17T21:18:29.827 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-17T21:18:29.827 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T21:18:29.886 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff542a393f0>, signals=[15]) 2024-09-17T21:18:29.886 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T21:18:29.893 INFO:teuthology.task.internal:Opening connections... 2024-09-17T21:18:29.893 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T21:18:29.894 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:18:29.972 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T21:18:29.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:18:30.045 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T21:18:30.052 DEBUG:teuthology.orchestra.run.smithi129:> uname -m 2024-09-17T21:18:30.071 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2024-09-17T21:18:30.071 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2024-09-17T21:18:30.129 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="CentOS Stream" 2024-09-17T21:18:30.129 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="9" 2024-09-17T21:18:30.129 INFO:teuthology.orchestra.run.smithi129.stdout:ID="centos" 2024-09-17T21:18:30.129 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="rhel fedora" 2024-09-17T21:18:30.129 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="9" 2024-09-17T21:18:30.129 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el9" 2024-09-17T21:18:30.129 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T21:18:30.129 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2024-09-17T21:18:30.130 INFO:teuthology.orchestra.run.smithi129.stdout:LOGO="fedora-logo-icon" 2024-09-17T21:18:30.130 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T21:18:30.130 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://centos.org/" 2024-09-17T21:18:30.130 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T21:18:30.130 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T21:18:30.130 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T21:18:30.131 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2024-09-17T21:18:30.154 DEBUG:teuthology.orchestra.run.smithi171:> uname -m 2024-09-17T21:18:30.172 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2024-09-17T21:18:30.173 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release 2024-09-17T21:18:30.231 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream" 2024-09-17T21:18:30.232 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="9" 2024-09-17T21:18:30.232 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos" 2024-09-17T21:18:30.232 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora" 2024-09-17T21:18:30.232 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="9" 2024-09-17T21:18:30.232 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el9" 2024-09-17T21:18:30.232 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T21:18:30.232 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31" 2024-09-17T21:18:30.232 INFO:teuthology.orchestra.run.smithi171.stdout:LOGO="fedora-logo-icon" 2024-09-17T21:18:30.232 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T21:18:30.232 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/" 2024-09-17T21:18:30.233 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T21:18:30.233 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T21:18:30.233 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T21:18:30.233 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2024-09-17T21:18:30.263 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T21:18:30.272 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T21:18:30.276 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T21:18:30.276 DEBUG:teuthology.orchestra.run.smithi129:> test '!' -e /home/ubuntu/cephtest 2024-09-17T21:18:30.279 DEBUG:teuthology.orchestra.run.smithi171:> test '!' -e /home/ubuntu/cephtest 2024-09-17T21:18:30.296 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T21:18:30.302 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T21:18:30.302 DEBUG:teuthology.orchestra.run.smithi129:> test -z $(ls -A /var/lib/ceph) 2024-09-17T21:18:30.337 DEBUG:teuthology.orchestra.run.smithi171:> test -z $(ls -A /var/lib/ceph) 2024-09-17T21:18:30.474 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T21:18:30.526 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T21:18:30.540 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T21:18:30.540 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T21:18:30.541 DEBUG:teuthology.orchestra.run.smithi129:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T21:18:30.541 DEBUG:teuthology.orchestra.run.smithi171:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T21:18:30.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:18:30.558 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-09-17T21:18:30.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:18:30.559 DEBUG:teuthology.orchestra.run.smithi171:> uname -r 2024-09-17T21:18:30.616 INFO:teuthology.orchestra.run.smithi129.stdout:5.14.0-503.el9.x86_64 2024-09-17T21:18:30.616 INFO:teuthology.task.kernel:Running kernel on smithi129: 5.14.0-503.el9.x86_64 2024-09-17T21:18:30.616 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum install -y kernel 2024-09-17T21:18:30.617 INFO:teuthology.orchestra.run.smithi171.stdout:5.14.0-503.el9.x86_64 2024-09-17T21:18:30.617 INFO:teuthology.task.kernel:Running kernel on smithi171: 5.14.0-503.el9.x86_64 2024-09-17T21:18:30.618 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum install -y kernel 2024-09-17T21:18:33.583 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 6.8 MB/s | 8.3 MB 00:01 2024-09-17T21:18:34.240 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.3 MB 00:01 2024-09-17T21:18:36.109 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 14 MB/s | 20 MB 00:01 2024-09-17T21:18:40.140 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 4.2 MB/s | 20 MB 00:04 2024-09-17T21:18:40.277 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-17T21:18:41.836 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 78 kB/s | 19 kB 00:00 2024-09-17T21:18:43.073 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-17T21:18:44.391 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-17T21:18:45.817 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 85 kB/s | 19 kB 00:00 2024-09-17T21:18:47.171 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-17T21:18:48.479 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-17T21:18:50.377 INFO:teuthology.orchestra.run.smithi129.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T21:18:50.377 INFO:teuthology.orchestra.run.smithi129.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T21:18:50.477 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:18:50.486 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:18:50.486 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-09-17T21:18:50.486 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:18:50.486 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T21:18:50.486 INFO:teuthology.orchestra.run.smithi129.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T21:18:50.486 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T21:18:50.487 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T21:18:50.487 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T21:18:50.487 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T21:18:50.487 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:18:50.487 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T21:18:50.487 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:18:50.487 INFO:teuthology.orchestra.run.smithi129.stdout:Install 4 Packages 2024-09-17T21:18:50.487 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:18:50.488 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 79 M 2024-09-17T21:18:50.488 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 126 M 2024-09-17T21:18:50.488 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T21:18:50.622 INFO:teuthology.orchestra.run.smithi129.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 322 kB/s | 36 kB 00:00 2024-09-17T21:18:52.593 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-17T21:18:53.455 INFO:teuthology.orchestra.run.smithi129.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 5.3 MB/s | 16 MB 00:02 2024-09-17T21:18:54.483 INFO:teuthology.orchestra.run.smithi171.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T21:18:54.483 INFO:teuthology.orchestra.run.smithi171.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T21:18:54.568 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:18:54.575 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:18:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: Package Architecture Version Repository Size 2024-09-17T21:18:54.575 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:18:54.575 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T21:18:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T21:18:54.575 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-17T21:18:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T21:18:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T21:18:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T21:18:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:18:54.575 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T21:18:54.575 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:18:54.576 INFO:teuthology.orchestra.run.smithi171.stdout:Install 4 Packages 2024-09-17T21:18:54.576 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:18:54.576 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 79 M 2024-09-17T21:18:54.576 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 126 M 2024-09-17T21:18:54.576 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T21:18:54.811 INFO:teuthology.orchestra.run.smithi171.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 375 kB/s | 36 kB 00:00 2024-09-17T21:18:55.299 INFO:teuthology.orchestra.run.smithi171.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-17T21:18:55.845 INFO:teuthology.orchestra.run.smithi171.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 31 MB/s | 35 MB 00:01 2024-09-17T21:18:56.064 INFO:teuthology.orchestra.run.smithi129.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 5.3 MB/s | 29 MB 00:05 2024-09-17T21:18:56.270 INFO:teuthology.orchestra.run.smithi171.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 20 MB/s | 29 MB 00:01 2024-09-17T21:18:56.271 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:18:56.271 INFO:teuthology.orchestra.run.smithi171.stdout:Total 46 MB/s | 79 MB 00:01 2024-09-17T21:18:56.939 INFO:teuthology.orchestra.run.smithi129.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 5.4 MB/s | 35 MB 00:06 2024-09-17T21:18:56.939 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:18:56.939 INFO:teuthology.orchestra.run.smithi129.stdout:Total 12 MB/s | 79 MB 00:06 2024-09-17T21:18:56.951 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T21:18:57.291 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T21:18:57.291 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T21:18:57.616 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T21:18:57.955 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T21:18:57.955 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T21:18:58.243 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T21:18:58.243 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T21:18:58.903 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T21:18:58.904 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T21:19:00.692 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T21:19:01.330 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T21:19:01.342 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T21:19:01.439 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T21:19:01.988 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T21:19:02.072 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T21:19:04.200 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T21:19:04.323 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T21:19:04.813 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T21:19:04.931 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T21:19:09.201 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T21:19:09.240 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:19:09.751 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T21:19:09.875 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:19:14.133 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:19:14.697 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:19:52.460 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:19:52.500 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:19:52.837 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:19:52.837 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T21:19:52.837 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T21:19:52.837 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T21:19:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:19:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:19:53.251 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:19:53.251 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T21:19:53.251 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T21:19:53.251 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T21:19:53.685 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:19:53.685 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:19:53.686 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T21:19:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T21:19:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T21:19:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T21:19:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T21:19:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:19:53.686 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:19:54.051 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T21:19:54.120 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:19:54.121 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:19:54.121 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T21:19:54.121 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T21:19:54.121 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T21:19:54.121 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T21:19:54.121 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T21:19:54.121 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:19:54.121 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:19:54.132 INFO:teuthology.orchestra.run.smithi171.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T21:19:54.133 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T21:19:54.133 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T21:19:54.133 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T21:19:54.133 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T21:19:54.134 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-17T21:19:54.134 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi171.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T21:19:54.134 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T21:19:54.193 INFO:teuthology.orchestra.run.smithi171.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T21:19:54.194 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T21:19:54.194 DEBUG:teuthology.orchestra.run.smithi171:> sudo rpm -qi grub2-tools 2024-09-17T21:19:54.237 INFO:teuthology.orchestra.run.smithi171.stdout:Name : grub2-tools 2024-09-17T21:19:54.237 INFO:teuthology.orchestra.run.smithi171.stdout:Epoch : 1 2024-09-17T21:19:54.237 INFO:teuthology.orchestra.run.smithi171.stdout:Version : 2.06 2024-09-17T21:19:54.237 INFO:teuthology.orchestra.run.smithi171.stdout:Release : 82.el9 2024-09-17T21:19:54.237 INFO:teuthology.orchestra.run.smithi171.stdout:Architecture: x86_64 2024-09-17T21:19:54.237 INFO:teuthology.orchestra.run.smithi171.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T21:19:54.237 INFO:teuthology.orchestra.run.smithi171.stdout:Group : Unspecified 2024-09-17T21:19:54.237 INFO:teuthology.orchestra.run.smithi171.stdout:Size : 8274670 2024-09-17T21:19:54.237 INFO:teuthology.orchestra.run.smithi171.stdout:License : GPLv3+ 2024-09-17T21:19:54.237 INFO:teuthology.orchestra.run.smithi171.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T21:19:54.238 INFO:teuthology.orchestra.run.smithi171.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T21:19:54.238 INFO:teuthology.orchestra.run.smithi171.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T21:19:54.238 INFO:teuthology.orchestra.run.smithi171.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T21:19:54.238 INFO:teuthology.orchestra.run.smithi171.stdout:Packager : builder@centos.org 2024-09-17T21:19:54.238 INFO:teuthology.orchestra.run.smithi171.stdout:Vendor : CentOS 2024-09-17T21:19:54.238 INFO:teuthology.orchestra.run.smithi171.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T21:19:54.238 INFO:teuthology.orchestra.run.smithi171.stdout:Summary : Support tools for GRUB. 2024-09-17T21:19:54.238 INFO:teuthology.orchestra.run.smithi171.stdout:Description : 2024-09-17T21:19:54.238 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:19:54.238 INFO:teuthology.orchestra.run.smithi171.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T21:19:54.238 INFO:teuthology.orchestra.run.smithi171.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T21:19:54.238 INFO:teuthology.orchestra.run.smithi171.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T21:19:54.238 INFO:teuthology.orchestra.run.smithi171.stdout:hardware devices. 2024-09-17T21:19:54.238 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:19:54.239 INFO:teuthology.orchestra.run.smithi171.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T21:19:54.239 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T21:19:54.240 INFO:teuthology.task.kernel:Updating grub on smithi171 to boot 5.14.0-508.el9.x86_64 2024-09-17T21:19:54.240 DEBUG:teuthology.orchestra.run.smithi171:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T21:19:54.486 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T21:19:54.570 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T21:19:54.571 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T21:19:54.571 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T21:19:54.571 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T21:19:54.571 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T21:19:54.571 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T21:19:54.571 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi129.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T21:19:54.571 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T21:19:54.628 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T21:19:54.629 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T21:19:54.629 DEBUG:teuthology.orchestra.run.smithi129:> sudo rpm -qi grub2-tools 2024-09-17T21:19:54.661 INFO:teuthology.orchestra.run.smithi129.stdout:Name : grub2-tools 2024-09-17T21:19:54.661 INFO:teuthology.orchestra.run.smithi129.stdout:Epoch : 1 2024-09-17T21:19:54.661 INFO:teuthology.orchestra.run.smithi129.stdout:Version : 2.06 2024-09-17T21:19:54.661 INFO:teuthology.orchestra.run.smithi129.stdout:Release : 82.el9 2024-09-17T21:19:54.661 INFO:teuthology.orchestra.run.smithi129.stdout:Architecture: x86_64 2024-09-17T21:19:54.661 INFO:teuthology.orchestra.run.smithi129.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T21:19:54.661 INFO:teuthology.orchestra.run.smithi129.stdout:Group : Unspecified 2024-09-17T21:19:54.662 INFO:teuthology.orchestra.run.smithi129.stdout:Size : 8274670 2024-09-17T21:19:54.662 INFO:teuthology.orchestra.run.smithi129.stdout:License : GPLv3+ 2024-09-17T21:19:54.662 INFO:teuthology.orchestra.run.smithi129.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T21:19:54.662 INFO:teuthology.orchestra.run.smithi129.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T21:19:54.662 INFO:teuthology.orchestra.run.smithi129.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T21:19:54.662 INFO:teuthology.orchestra.run.smithi129.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T21:19:54.662 INFO:teuthology.orchestra.run.smithi129.stdout:Packager : builder@centos.org 2024-09-17T21:19:54.662 INFO:teuthology.orchestra.run.smithi129.stdout:Vendor : CentOS 2024-09-17T21:19:54.662 INFO:teuthology.orchestra.run.smithi129.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T21:19:54.662 INFO:teuthology.orchestra.run.smithi129.stdout:Summary : Support tools for GRUB. 2024-09-17T21:19:54.662 INFO:teuthology.orchestra.run.smithi129.stdout:Description : 2024-09-17T21:19:54.662 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:19:54.662 INFO:teuthology.orchestra.run.smithi129.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T21:19:54.663 INFO:teuthology.orchestra.run.smithi129.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T21:19:54.663 INFO:teuthology.orchestra.run.smithi129.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T21:19:54.663 INFO:teuthology.orchestra.run.smithi129.stdout:hardware devices. 2024-09-17T21:19:54.663 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:19:54.663 INFO:teuthology.orchestra.run.smithi129.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T21:19:54.664 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T21:19:54.664 INFO:teuthology.task.kernel:Updating grub on smithi129 to boot 5.14.0-508.el9.x86_64 2024-09-17T21:19:54.664 DEBUG:teuthology.orchestra.run.smithi129:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T21:19:54.868 INFO:teuthology.orchestra.run.smithi171.stderr:Generating grub configuration file ... 2024-09-17T21:19:55.333 INFO:teuthology.orchestra.run.smithi129.stderr:Generating grub configuration file ... 2024-09-17T21:19:56.235 INFO:teuthology.orchestra.run.smithi171.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T21:19:56.271 INFO:teuthology.orchestra.run.smithi171.stderr:done 2024-09-17T21:19:56.274 DEBUG:teuthology.orchestra.run.smithi171:> mktemp 2024-09-17T21:19:56.290 INFO:teuthology.orchestra.run.smithi171.stdout:/tmp/tmp.6DmKMkkg7X 2024-09-17T21:19:56.290 DEBUG:teuthology.orchestra.run.smithi171:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6DmKMkkg7X 2024-09-17T21:19:56.357 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0666 /tmp/tmp.6DmKMkkg7X 2024-09-17T21:19:56.491 DEBUG:teuthology.orchestra.remote:smithi171:/tmp/tmp.6DmKMkkg7X is 6KB 2024-09-17T21:19:56.540 DEBUG:teuthology.orchestra.run.smithi171:> rm -fr /tmp/tmp.6DmKMkkg7X 2024-09-17T21:19:56.555 DEBUG:teuthology.orchestra.run.smithi171:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T21:19:56.589 INFO:teuthology.orchestra.run.smithi129.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T21:19:56.617 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T21:19:56.618 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T21:19:56.618 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T21:19:56.618 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T21:19:56.619 DEBUG:teuthology.orchestra.run.smithi171:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T21:19:56.624 INFO:teuthology.orchestra.run.smithi129.stderr:done 2024-09-17T21:19:56.635 DEBUG:teuthology.orchestra.run.smithi129:> mktemp 2024-09-17T21:19:56.651 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.ZFTMmhFz0j 2024-09-17T21:19:56.652 DEBUG:teuthology.orchestra.run.smithi129:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ZFTMmhFz0j 2024-09-17T21:19:56.725 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.ZFTMmhFz0j 2024-09-17T21:19:56.772 DEBUG:teuthology.orchestra.run.smithi171:> sudo shutdown -r now 2024-09-17T21:19:56.867 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.ZFTMmhFz0j is 6KB 2024-09-17T21:19:56.916 DEBUG:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.ZFTMmhFz0j 2024-09-17T21:19:56.930 DEBUG:teuthology.orchestra.run.smithi129:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T21:19:56.993 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T21:19:56.993 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T21:19:56.993 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T21:19:56.993 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T21:19:56.995 DEBUG:teuthology.orchestra.run.smithi129:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T21:19:57.151 DEBUG:teuthology.orchestra.run.smithi129:> sudo shutdown -r now 2024-09-17T21:20:26.803 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-17T21:20:26.803 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-17T21:20:26.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:20:27.154 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T21:20:27.155 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T21:20:27.155 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:20:45.373 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.171 2024-09-17T21:20:54.383 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-17T21:20:54.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:20:57.437 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.171 2024-09-17T21:21:01.757 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-09-17T21:21:09.445 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-17T21:21:09.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:21:10.759 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T21:21:10.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:21:29.149 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-09-17T21:21:41.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T21:21:41.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:21:41.815 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-09-17T21:21:42.276 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T21:21:42.276 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-17T21:21:42.276 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-09-17T21:21:42.291 INFO:teuthology.orchestra.run.smithi129.stdout:5.14.0-508.el9.x86_64 2024-09-17T21:21:42.291 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T21:21:42.292 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T21:21:42.292 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T21:21:43.292 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T21:21:43.292 DEBUG:teuthology.orchestra.run.smithi129:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T21:21:43.384 INFO:teuthology.orchestra.run.smithi129.stdout:ttyS1 2024-09-17T21:21:43.403 DEBUG:teuthology.parallel:result is None 2024-09-17T21:22:09.448 DEBUG:teuthology.orchestra.remote:timed out 2024-09-17T21:22:24.449 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-17T21:22:24.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:22:24.927 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-09-17T21:22:25.274 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-17T21:22:25.274 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-17T21:22:25.274 DEBUG:teuthology.orchestra.run.smithi171:> uname -r 2024-09-17T21:22:25.289 INFO:teuthology.orchestra.run.smithi171.stdout:5.14.0-508.el9.x86_64 2024-09-17T21:22:25.289 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T21:22:25.290 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T21:22:25.290 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T21:22:26.291 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-17T21:22:26.291 DEBUG:teuthology.orchestra.run.smithi171:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T21:22:26.359 INFO:teuthology.orchestra.run.smithi171.stdout:ttyS1 2024-09-17T21:22:26.378 DEBUG:teuthology.parallel:result is None 2024-09-17T21:22:26.378 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T21:22:26.385 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T21:22:26.386 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T21:22:26.389 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T21:22:26.406 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T21:22:26.414 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T21:22:26.421 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T21:22:26.422 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T21:22:26.449 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T21:22:26.480 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T21:22:26.486 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T21:22:26.487 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T21:22:26.517 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T21:22:26.549 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T21:22:26.551 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T21:22:26.561 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T21:22:26.563 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T21:22:26.565 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T21:22:26.572 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T21:22:26.572 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T21:22:26.608 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T21:22:26.636 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T21:22:26.645 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T21:22:26.645 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T21:22:26.677 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T21:22:26.694 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T21:22:26.771 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T21:22:26.824 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T21:22:26.904 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T21:22:26.949 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:22:26.949 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T21:22:27.015 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T21:22:27.054 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T21:22:27.112 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T21:22:27.188 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T21:22:27.240 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:22:27.240 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T21:22:27.307 DEBUG:teuthology.orchestra.run.smithi129:> sudo service rsyslog restart 2024-09-17T21:22:27.309 DEBUG:teuthology.orchestra.run.smithi171:> sudo service rsyslog restart 2024-09-17T21:22:27.349 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T21:22:27.402 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T21:22:27.780 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T21:22:27.787 INFO:teuthology.task.internal:Starting timer... 2024-09-17T21:22:27.787 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T21:22:27.805 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T21:22:27.815 DEBUG:teuthology.orchestra.run.smithi129:> sudo service auditd rotate 2024-09-17T21:22:27.883 INFO:teuthology.orchestra.run.smithi129.stdout:Rotating logs: 2024-09-17T21:22:27.885 DEBUG:teuthology.orchestra.run.smithi171:> sudo service auditd rotate 2024-09-17T21:22:27.932 INFO:teuthology.orchestra.run.smithi171.stdout:Rotating logs: 2024-09-17T21:22:27.934 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T21:22:27.935 DEBUG:teuthology.orchestra.run.smithi129:> /usr/sbin/getenforce 2024-09-17T21:22:27.959 INFO:teuthology.orchestra.run.smithi129.stdout:Permissive 2024-09-17T21:22:27.959 DEBUG:teuthology.orchestra.run.smithi171:> /usr/sbin/getenforce 2024-09-17T21:22:27.994 INFO:teuthology.orchestra.run.smithi171.stdout:Permissive 2024-09-17T21:22:27.994 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi129.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi171.front.sepia.ceph.com': 'permissive'} 2024-09-17T21:22:27.994 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T21:22:28.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:22:28.026 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T21:22:28.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:22:28.061 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T21:22:28.061 DEBUG:teuthology.orchestra.run.smithi129:> sudo /usr/sbin/setenforce permissive 2024-09-17T21:22:28.095 DEBUG:teuthology.orchestra.run.smithi171:> sudo /usr/sbin/setenforce permissive 2024-09-17T21:22:28.134 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T21:22:28.143 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-17T21:22:28.146 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-17T21:22:28.258 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T21:22:28.268 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T21:22:28.269 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi129.front.sepia.ceph.com,smithi171.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T21:26:13.827 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi129.front.sepia.ceph.com'), Remote(name='ubuntu@smithi171.front.sepia.ceph.com')] 2024-09-17T21:26:13.828 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T21:26:13.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:26:13.911 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-09-17T21:26:13.988 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T21:26:13.988 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-17T21:26:13.989 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:26:14.105 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-09-17T21:26:14.185 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-17T21:26:14.185 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T21:26:14.197 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T21:26:14.197 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T21:26:14.197 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T21:26:14.200 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T21:26:14.200 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T21:26:14.227 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T21:26:14.241 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T21:26:14.257 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T21:26:14.265 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: ntpd: command not found 2024-09-17T21:26:14.272 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T21:26:14.276 INFO:teuthology.orchestra.run.smithi129.stdout:506 Cannot talk to daemon 2024-09-17T21:26:14.290 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T21:26:14.298 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: ntpd: command not found 2024-09-17T21:26:14.303 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T21:26:14.309 INFO:teuthology.orchestra.run.smithi171.stdout:506 Cannot talk to daemon 2024-09-17T21:26:14.324 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T21:26:14.338 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T21:26:14.366 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-09-17T21:26:14.369 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T21:26:14.369 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-09-17T21:26:14.369 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:26:14.370 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:26:14.370 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:26:14.370 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:26:14.399 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-09-17T21:26:14.402 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T21:26:14.402 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-09-17T21:26:14.402 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:26:14.402 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:26:14.403 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:26:14.403 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:26:14.404 INFO:teuthology.run_tasks:Running task install... 2024-09-17T21:26:14.412 DEBUG:teuthology.task.install:project ceph 2024-09-17T21:26:14.412 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '08cec6974d86f81dc304360b31113f30b4085bf8'}} 2024-09-17T21:26:14.412 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '08cec6974d86f81dc304360b31113f30b4085bf8'} 2024-09-17T21:26:14.412 INFO:teuthology.task.install:Using flavor: default 2024-09-17T21:26:14.418 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T21:26:14.418 INFO:teuthology.task.install:extra packages: [] 2024-09-17T21:26:14.418 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '08cec6974d86f81dc304360b31113f30b4085bf8', 'tag': None, 'wait_for_package': False} 2024-09-17T21:26:14.419 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=08cec6974d86f81dc304360b31113f30b4085bf8 2024-09-17T21:26:14.421 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '08cec6974d86f81dc304360b31113f30b4085bf8', 'tag': None, 'wait_for_package': False} 2024-09-17T21:26:14.421 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=08cec6974d86f81dc304360b31113f30b4085bf8 2024-09-17T21:26:14.565 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/centos/9/flavors/default/ 2024-09-17T21:26:14.565 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4975.g08cec697 2024-09-17T21:26:14.571 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/centos/9/flavors/default/ 2024-09-17T21:26:14.571 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4975.g08cec697 2024-09-17T21:26:14.681 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T21:26:14.681 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:26:14.681 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T21:26:14.690 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-mchangir-data-uninlining-verfication-after-merge/08cec6974d86f81dc304360b31113f30b4085bf8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T21:26:14.690 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:26:14.690 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T21:26:14.708 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T21:26:14.708 DEBUG:teuthology.orchestra.run.smithi129:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/08cec6974d86f81dc304360b31113f30b4085bf8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T21:26:14.716 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T21:26:14.716 DEBUG:teuthology.orchestra.run.smithi171:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/08cec6974d86f81dc304360b31113f30b4085bf8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T21:26:14.777 DEBUG:teuthology.orchestra.run.smithi129:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T21:26:14.782 DEBUG:teuthology.orchestra.run.smithi171:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T21:26:14.853 DEBUG:teuthology.orchestra.run.smithi129:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T21:26:14.857 DEBUG:teuthology.orchestra.run.smithi171:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T21:26:14.916 INFO:teuthology.orchestra.run.smithi129.stdout:check_obsoletes = 1 2024-09-17T21:26:14.918 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum clean all 2024-09-17T21:26:14.922 INFO:teuthology.orchestra.run.smithi171.stdout:check_obsoletes = 1 2024-09-17T21:26:14.924 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum clean all 2024-09-17T21:26:15.145 INFO:teuthology.orchestra.run.smithi129.stdout:45 files removed 2024-09-17T21:26:15.153 INFO:teuthology.orchestra.run.smithi171.stdout:45 files removed 2024-09-17T21:26:15.172 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-radosgw 2024-09-17T21:26:15.180 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-radosgw 2024-09-17T21:26:15.731 INFO:teuthology.orchestra.run.smithi171.stdout:ceph packages for x86_64 319 kB/s | 84 kB 00:00 2024-09-17T21:26:15.736 INFO:teuthology.orchestra.run.smithi129.stdout:ceph packages for x86_64 305 kB/s | 84 kB 00:00 2024-09-17T21:26:16.024 INFO:teuthology.orchestra.run.smithi171.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-17T21:26:16.028 INFO:teuthology.orchestra.run.smithi129.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-17T21:26:16.300 INFO:teuthology.orchestra.run.smithi171.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-17T21:26:16.304 INFO:teuthology.orchestra.run.smithi129.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-17T21:26:16.904 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-17T21:26:18.434 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.3 MB 00:02 2024-09-17T21:26:19.163 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 17 MB/s | 20 MB 00:01 2024-09-17T21:26:20.510 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-17T21:26:25.087 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 8.7 MB/s | 6.5 MB 00:00 2024-09-17T21:26:25.089 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.5 MB 00:02 2024-09-17T21:26:26.546 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 79 kB/s | 19 kB 00:00 2024-09-17T21:26:26.565 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 78 kB/s | 19 kB 00:00 2024-09-17T21:26:28.155 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 15 MB/s | 23 MB 00:01 2024-09-17T21:26:28.167 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 15 MB/s | 23 MB 00:01 2024-09-17T21:26:33.532 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-17T21:26:33.594 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-09-17T21:26:35.766 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:26:35.782 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:26:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T21:26:35.782 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:26:35.782 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T21:26:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw x86_64 2:19.3.0-4975.g08cec697.el9 ceph 11 M 2024-09-17T21:26:35.782 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrading: 2024-09-17T21:26:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: librados2 x86_64 2:19.3.0-4975.g08cec697.el9 ceph 3.5 M 2024-09-17T21:26:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1 x86_64 2:19.3.0-4975.g08cec697.el9 ceph 3.2 M 2024-09-17T21:26:35.782 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-17T21:26:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T21:26:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base x86_64 2:19.3.0-4975.g08cec697.el9 ceph 5.5 M 2024-09-17T21:26:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common x86_64 2:19.3.0-4975.g08cec697.el9 ceph 22 M 2024-09-17T21:26:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux x86_64 2:19.3.0-4975.g08cec697.el9 ceph 25 k 2024-09-17T21:26:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T21:26:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T21:26:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T21:26:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2 x86_64 2:19.3.0-4975.g08cec697.el9 ceph 756 k 2024-09-17T21:26:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-17T21:26:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T21:26:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T21:26:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T21:26:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1 x86_64 2:19.3.0-4975.g08cec697.el9 ceph 508 k 2024-09-17T21:26:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T21:26:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2 x86_64 2:19.3.0-4975.g08cec697.el9 ceph 5.5 M 2024-09-17T21:26:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T21:26:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T21:26:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T21:26:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse x86_64 2:19.3.0-4975.g08cec697.el9 ceph 45 k 2024-09-17T21:26:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common x86_64 2:19.3.0-4975.g08cec697.el9 ceph 143 k 2024-09-17T21:26:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs x86_64 2:19.3.0-4975.g08cec697.el9 ceph 161 k 2024-09-17T21:26:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T21:26:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados x86_64 2:19.3.0-4975.g08cec697.el9 ceph 321 k 2024-09-17T21:26:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd x86_64 2:19.3.0-4975.g08cec697.el9 ceph 306 k 2024-09-17T21:26:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw x86_64 2:19.3.0-4975.g08cec697.el9 ceph 99 k 2024-09-17T21:26:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T21:26:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T21:26:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T21:26:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T21:26:35.785 INFO:teuthology.orchestra.run.smithi171.stdout:Installing weak dependencies: 2024-09-17T21:26:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T21:26:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:26:35.785 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T21:26:35.785 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:26:35.785 INFO:teuthology.orchestra.run.smithi171.stdout:Install 31 Packages 2024-09-17T21:26:35.785 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrade 2 Packages 2024-09-17T21:26:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:26:35.786 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 62 M 2024-09-17T21:26:35.786 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T21:26:35.888 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:26:35.915 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:26:35.915 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T21:26:35.915 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:26:35.915 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T21:26:35.915 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw x86_64 2:19.3.0-4975.g08cec697.el9 ceph 11 M 2024-09-17T21:26:35.915 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrading: 2024-09-17T21:26:35.915 INFO:teuthology.orchestra.run.smithi129.stdout: librados2 x86_64 2:19.3.0-4975.g08cec697.el9 ceph 3.5 M 2024-09-17T21:26:35.915 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1 x86_64 2:19.3.0-4975.g08cec697.el9 ceph 3.2 M 2024-09-17T21:26:35.915 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T21:26:35.915 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T21:26:35.916 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base x86_64 2:19.3.0-4975.g08cec697.el9 ceph 5.5 M 2024-09-17T21:26:35.916 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common x86_64 2:19.3.0-4975.g08cec697.el9 ceph 22 M 2024-09-17T21:26:35.916 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux x86_64 2:19.3.0-4975.g08cec697.el9 ceph 25 k 2024-09-17T21:26:35.916 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T21:26:35.916 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T21:26:35.916 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T21:26:35.916 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2 x86_64 2:19.3.0-4975.g08cec697.el9 ceph 756 k 2024-09-17T21:26:35.916 INFO:teuthology.orchestra.run.smithi129.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-17T21:26:35.916 INFO:teuthology.orchestra.run.smithi129.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T21:26:35.916 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T21:26:35.916 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T21:26:35.916 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1 x86_64 2:19.3.0-4975.g08cec697.el9 ceph 508 k 2024-09-17T21:26:35.917 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T21:26:35.917 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2 x86_64 2:19.3.0-4975.g08cec697.el9 ceph 5.5 M 2024-09-17T21:26:35.917 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T21:26:35.917 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T21:26:35.917 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T21:26:35.917 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse x86_64 2:19.3.0-4975.g08cec697.el9 ceph 45 k 2024-09-17T21:26:35.917 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common x86_64 2:19.3.0-4975.g08cec697.el9 ceph 143 k 2024-09-17T21:26:35.917 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs x86_64 2:19.3.0-4975.g08cec697.el9 ceph 161 k 2024-09-17T21:26:35.917 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T21:26:35.918 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados x86_64 2:19.3.0-4975.g08cec697.el9 ceph 321 k 2024-09-17T21:26:35.918 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd x86_64 2:19.3.0-4975.g08cec697.el9 ceph 306 k 2024-09-17T21:26:35.918 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw x86_64 2:19.3.0-4975.g08cec697.el9 ceph 99 k 2024-09-17T21:26:35.918 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T21:26:35.918 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T21:26:35.918 INFO:teuthology.orchestra.run.smithi129.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T21:26:35.918 INFO:teuthology.orchestra.run.smithi129.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T21:26:35.918 INFO:teuthology.orchestra.run.smithi129.stdout:Installing weak dependencies: 2024-09-17T21:26:35.918 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T21:26:35.918 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:26:35.918 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T21:26:35.918 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:26:35.919 INFO:teuthology.orchestra.run.smithi129.stdout:Install 31 Packages 2024-09-17T21:26:35.919 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrade 2 Packages 2024-09-17T21:26:35.919 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:26:35.919 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 62 M 2024-09-17T21:26:35.919 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T21:26:36.617 INFO:teuthology.orchestra.run.smithi171.stdout:(1/33): ceph-base-19.3.0-4975.g08cec697.el9.x86 9.4 MB/s | 5.5 MB 00:00 2024-09-17T21:26:36.693 INFO:teuthology.orchestra.run.smithi171.stdout:(2/33): ceph-selinux-19.3.0-4975.g08cec697.el9. 336 kB/s | 25 kB 00:00 2024-09-17T21:26:36.793 INFO:teuthology.orchestra.run.smithi171.stdout:(3/33): libcephfs2-19.3.0-4975.g08cec697.el9.x8 7.4 MB/s | 756 kB 00:00 2024-09-17T21:26:36.877 INFO:teuthology.orchestra.run.smithi171.stdout:(4/33): libradosstriper1-19.3.0-4975.g08cec697. 5.9 MB/s | 508 kB 00:00 2024-09-17T21:26:37.011 INFO:teuthology.orchestra.run.smithi171.stdout:(5/33): ceph-radosgw-19.3.0-4975.g08cec697.el9. 11 MB/s | 11 MB 00:00 2024-09-17T21:26:37.094 INFO:teuthology.orchestra.run.smithi171.stdout:(6/33): python3-ceph-argparse-19.3.0-4975.g08ce 543 kB/s | 45 kB 00:00 2024-09-17T21:26:37.178 INFO:teuthology.orchestra.run.smithi171.stdout:(7/33): python3-ceph-common-19.3.0-4975.g08cec6 1.7 MB/s | 143 kB 00:00 2024-09-17T21:26:37.261 INFO:teuthology.orchestra.run.smithi171.stdout:(8/33): librgw2-19.3.0-4975.g08cec697.el9.x86_6 14 MB/s | 5.5 MB 00:00 2024-09-17T21:26:37.287 INFO:teuthology.orchestra.run.smithi171.stdout:(9/33): python3-cephfs-19.3.0-4975.g08cec697.el 1.5 MB/s | 161 kB 00:00 2024-09-17T21:26:37.329 INFO:teuthology.orchestra.run.smithi171.stdout:(10/33): python3-rados-19.3.0-4975.g08cec697.el 4.7 MB/s | 321 kB 00:00 2024-09-17T21:26:37.354 INFO:teuthology.orchestra.run.smithi171.stdout:(11/33): python3-rbd-19.3.0-4975.g08cec697.el9. 4.5 MB/s | 306 kB 00:00 2024-09-17T21:26:37.562 INFO:teuthology.orchestra.run.smithi129.stdout:(1/33): ceph-base-19.3.0-4975.g08cec697.el9.x86 3.7 MB/s | 5.5 MB 00:01 2024-09-17T21:26:37.588 INFO:teuthology.orchestra.run.smithi171.stdout:(12/33): ceph-common-19.3.0-4975.g08cec697.el9. 14 MB/s | 22 MB 00:01 2024-09-17T21:26:37.604 INFO:teuthology.orchestra.run.smithi129.stdout:(2/33): ceph-selinux-19.3.0-4975.g08cec697.el9. 606 kB/s | 25 kB 00:00 2024-09-17T21:26:37.613 INFO:teuthology.orchestra.run.smithi171.stdout:(13/33): python3-rgw-19.3.0-4975.g08cec697.el9. 350 kB/s | 99 kB 00:00 2024-09-17T21:26:37.714 INFO:teuthology.orchestra.run.smithi171.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 298 kB/s | 107 kB 00:00 2024-09-17T21:26:37.721 INFO:teuthology.orchestra.run.smithi129.stdout:(3/33): ceph-radosgw-19.3.0-4975.g08cec697.el9. 6.7 MB/s | 11 MB 00:01 2024-09-17T21:26:37.739 INFO:teuthology.orchestra.run.smithi171.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 1.1 MB/s | 171 kB 00:00 2024-09-17T21:26:37.765 INFO:teuthology.orchestra.run.smithi171.stdout:(16/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-17T21:26:37.779 INFO:teuthology.orchestra.run.smithi129.stdout:(4/33): libradosstriper1-19.3.0-4975.g08cec697. 8.5 MB/s | 508 kB 00:00 2024-09-17T21:26:37.789 INFO:teuthology.orchestra.run.smithi171.stdout:(17/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 598 kB/s | 45 kB 00:00 2024-09-17T21:26:37.830 INFO:teuthology.orchestra.run.smithi129.stdout:(5/33): libcephfs2-19.3.0-4975.g08cec697.el9.x8 3.3 MB/s | 756 kB 00:00 2024-09-17T21:26:37.831 INFO:teuthology.orchestra.run.smithi171.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-17T21:26:37.866 INFO:teuthology.orchestra.run.smithi171.stdout:(19/33): librdkafka-1.6.1-102.el9.x86_64.rpm 5.1 MB/s | 662 kB 00:00 2024-09-17T21:26:37.880 INFO:teuthology.orchestra.run.smithi129.stdout:(6/33): python3-ceph-argparse-19.3.0-4975.g08ce 907 kB/s | 45 kB 00:00 2024-09-17T21:26:37.899 INFO:teuthology.orchestra.run.smithi171.stdout:(20/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-17T21:26:37.924 INFO:teuthology.orchestra.run.smithi171.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 611 kB/s | 36 kB 00:00 2024-09-17T21:26:37.947 INFO:teuthology.orchestra.run.smithi129.stdout:(7/33): python3-ceph-common-19.3.0-4975.g08cec6 2.1 MB/s | 143 kB 00:00 2024-09-17T21:26:37.950 INFO:teuthology.orchestra.run.smithi171.stdout:(22/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 939 kB/s | 47 kB 00:00 2024-09-17T21:26:37.975 INFO:teuthology.orchestra.run.smithi171.stdout:(23/33): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-09-17T21:26:38.030 INFO:teuthology.orchestra.run.smithi129.stdout:(8/33): python3-cephfs-19.3.0-4975.g08cec697.el 1.9 MB/s | 161 kB 00:00 2024-09-17T21:26:38.034 INFO:teuthology.orchestra.run.smithi171.stdout:(24/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 598 kB/s | 26 kB 00:00 2024-09-17T21:26:38.060 INFO:teuthology.orchestra.run.smithi171.stdout:(25/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.2 MB/s | 308 kB 00:00 2024-09-17T21:26:38.085 INFO:teuthology.orchestra.run.smithi171.stdout:(26/33): liboath-2.6.7-2.el9.x86_64.rpm 964 kB/s | 49 kB 00:00 2024-09-17T21:26:38.110 INFO:teuthology.orchestra.run.smithi171.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-17T21:26:38.144 INFO:teuthology.orchestra.run.smithi171.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-17T21:26:38.156 INFO:teuthology.orchestra.run.smithi129.stdout:(9/33): librgw2-19.3.0-4975.g08cec697.el9.x86_6 15 MB/s | 5.5 MB 00:00 2024-09-17T21:26:38.181 INFO:teuthology.orchestra.run.smithi129.stdout:(10/33): python3-rados-19.3.0-4975.g08cec697.el 2.1 MB/s | 321 kB 00:00 2024-09-17T21:26:38.203 INFO:teuthology.orchestra.run.smithi171.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-17T21:26:38.337 INFO:teuthology.orchestra.run.smithi171.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 8.2 MB/s | 1.6 MB 00:00 2024-09-17T21:26:38.390 INFO:teuthology.orchestra.run.smithi129.stdout:(11/33): ceph-common-19.3.0-4975.g08cec697.el9. 9.4 MB/s | 22 MB 00:02 2024-09-17T21:26:38.415 INFO:teuthology.orchestra.run.smithi129.stdout:(12/33): python3-rgw-19.3.0-4975.g08cec697.el9. 425 kB/s | 99 kB 00:00 2024-09-17T21:26:38.440 INFO:teuthology.orchestra.run.smithi129.stdout:(13/33): python3-rbd-19.3.0-4975.g08cec697.el9. 1.1 MB/s | 306 kB 00:00 2024-09-17T21:26:38.499 INFO:teuthology.orchestra.run.smithi129.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 980 kB/s | 107 kB 00:00 2024-09-17T21:26:38.525 INFO:teuthology.orchestra.run.smithi129.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 1.5 MB/s | 171 kB 00:00 2024-09-17T21:26:38.550 INFO:teuthology.orchestra.run.smithi129.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-09-17T21:26:38.554 INFO:teuthology.orchestra.run.smithi171.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 7.3 MB/s | 4.4 MB 00:00 2024-09-17T21:26:38.575 INFO:teuthology.orchestra.run.smithi129.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-17T21:26:38.609 INFO:teuthology.orchestra.run.smithi129.stdout:(18/33): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-09-17T21:26:38.620 INFO:teuthology.orchestra.run.smithi171.stdout:(32/33): librados2-19.3.0-4975.g08cec697.el9.x8 8.3 MB/s | 3.5 MB 00:00 2024-09-17T21:26:38.634 INFO:teuthology.orchestra.run.smithi129.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 721 kB/s | 42 kB 00:00 2024-09-17T21:26:38.659 INFO:teuthology.orchestra.run.smithi129.stdout:(20/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-17T21:26:38.685 INFO:teuthology.orchestra.run.smithi129.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 712 kB/s | 36 kB 00:00 2024-09-17T21:26:38.710 INFO:teuthology.orchestra.run.smithi129.stdout:(22/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-17T21:26:38.735 INFO:teuthology.orchestra.run.smithi129.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-09-17T21:26:38.802 INFO:teuthology.orchestra.run.smithi129.stdout:(24/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 391 kB/s | 26 kB 00:00 2024-09-17T21:26:38.828 INFO:teuthology.orchestra.run.smithi129.stdout:(25/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.1 MB/s | 308 kB 00:00 2024-09-17T21:26:38.837 INFO:teuthology.orchestra.run.smithi171.stdout:(33/33): librbd1-19.3.0-4975.g08cec697.el9.x86_ 6.5 MB/s | 3.2 MB 00:00 2024-09-17T21:26:38.843 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:26:38.843 INFO:teuthology.orchestra.run.smithi171.stdout:Total 20 MB/s | 62 MB 00:03 2024-09-17T21:26:38.844 INFO:teuthology.orchestra.run.smithi129.stdout:(26/33): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-09-17T21:26:38.870 INFO:teuthology.orchestra.run.smithi129.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-17T21:26:38.912 INFO:teuthology.orchestra.run.smithi129.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-17T21:26:38.964 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T21:26:39.002 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T21:26:39.002 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T21:26:39.004 INFO:teuthology.orchestra.run.smithi129.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.1 MB/s | 839 kB 00:00 2024-09-17T21:26:39.129 INFO:teuthology.orchestra.run.smithi129.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-09-17T21:26:39.304 INFO:teuthology.orchestra.run.smithi129.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 7.8 MB/s | 4.4 MB 00:00 2024-09-17T21:26:39.716 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T21:26:39.717 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T21:26:39.763 INFO:teuthology.orchestra.run.smithi129.stdout:(32/33): librbd1-19.3.0-4975.g08cec697.el9.x86_ 5.1 MB/s | 3.2 MB 00:00 2024-09-17T21:26:40.163 INFO:teuthology.orchestra.run.smithi129.stdout:(33/33): librados2-19.3.0-4975.g08cec697.el9.x8 3.0 MB/s | 3.5 MB 00:01 2024-09-17T21:26:40.169 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:26:40.169 INFO:teuthology.orchestra.run.smithi129.stdout:Total 15 MB/s | 62 MB 00:04 2024-09-17T21:26:40.288 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T21:26:40.325 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T21:26:40.325 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T21:26:41.007 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T21:26:41.008 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T21:26:41.296 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T21:26:41.372 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-17T21:26:41.409 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-17T21:26:41.435 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-17T21:26:41.458 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T21:26:41.479 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T21:26:41.619 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T21:26:41.672 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T21:26:42.000 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-17T21:26:42.113 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librados2-2:19.3.0-4975.g08cec697.el9.x86_64 7/35 2024-09-17T21:26:42.206 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:19.3.0-4975.g08cec697.el9.x86_64 7/35 2024-09-17T21:26:42.297 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rados-2:19.3.0-4975.g08cec697.el9.x86_64 8/35 2024-09-17T21:26:42.323 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs2-2:19.3.0-4975.g08cec697.el9.x86_64 9/35 2024-09-17T21:26:42.387 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs2-2:19.3.0-4975.g08cec697.el9.x86_64 9/35 2024-09-17T21:26:42.426 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-17T21:26:42.469 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-argparse-2:19.3.0-4975.g08cec697.el9. 11/35 2024-09-17T21:26:42.531 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cephfs-2:19.3.0-4975.g08cec697.el9.x86_64 12/35 2024-09-17T21:26:42.544 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T21:26:42.560 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libradosstriper1-2:19.3.0-4975.g08cec697.el9.x86_6 13/35 2024-09-17T21:26:42.625 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4975.g08cec697.el9.x86_6 13/35 2024-09-17T21:26:42.631 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-17T21:26:42.659 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-17T21:26:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-17T21:26:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-17T21:26:42.708 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T21:26:42.734 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T21:26:42.771 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-17T21:26:42.802 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-17T21:26:42.848 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-17T21:26:42.876 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-17T21:26:42.905 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-17T21:26:42.925 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T21:26:42.938 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T21:26:42.980 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T21:26:42.981 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-17T21:26:43.313 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-17T21:26:43.348 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-17T21:26:43.361 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librbd1-2:19.3.0-4975.g08cec697.el9.x86_64 23/35 2024-09-17T21:26:43.438 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librados2-2:19.3.0-4975.g08cec697.el9.x86_64 7/35 2024-09-17T21:26:43.458 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:19.3.0-4975.g08cec697.el9.x86_64 23/35 2024-09-17T21:26:43.483 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rbd-2:19.3.0-4975.g08cec697.el9.x86_64 24/35 2024-09-17T21:26:43.519 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-17T21:26:43.531 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:19.3.0-4975.g08cec697.el9.x86_64 7/35 2024-09-17T21:26:43.617 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rados-2:19.3.0-4975.g08cec697.el9.x86_64 8/35 2024-09-17T21:26:43.648 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs2-2:19.3.0-4975.g08cec697.el9.x86_64 9/35 2024-09-17T21:26:43.711 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephfs2-2:19.3.0-4975.g08cec697.el9.x86_64 9/35 2024-09-17T21:26:43.748 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-17T21:26:43.792 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-argparse-2:19.3.0-4975.g08cec697.el9. 11/35 2024-09-17T21:26:43.860 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cephfs-2:19.3.0-4975.g08cec697.el9.x86_64 12/35 2024-09-17T21:26:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libradosstriper1-2:19.3.0-4975.g08cec697.el9.x86_6 13/35 2024-09-17T21:26:43.948 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4975.g08cec697.el9.x86_6 13/35 2024-09-17T21:26:43.982 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-17T21:26:44.011 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-17T21:26:44.039 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librgw2-2:19.3.0-4975.g08cec697.el9.x86_64 27/35 2024-09-17T21:26:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-17T21:26:44.111 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librgw2-2:19.3.0-4975.g08cec697.el9.x86_64 27/35 2024-09-17T21:26:44.125 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-17T21:26:44.172 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-17T21:26:44.190 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rgw-2:19.3.0-4975.g08cec697.el9.x86_64 28/35 2024-09-17T21:26:44.207 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-17T21:26:44.226 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-common-2:19.3.0-4975.g08cec697.el9.x8 29/35 2024-09-17T21:26:44.236 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-17T21:26:44.278 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T21:26:44.339 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-17T21:26:44.663 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-17T21:26:44.708 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librbd1-2:19.3.0-4975.g08cec697.el9.x86_64 23/35 2024-09-17T21:26:44.806 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:19.3.0-4975.g08cec697.el9.x86_64 23/35 2024-09-17T21:26:44.838 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rbd-2:19.3.0-4975.g08cec697.el9.x86_64 24/35 2024-09-17T21:26:44.874 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-17T21:26:45.362 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-17T21:26:45.394 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librgw2-2:19.3.0-4975.g08cec697.el9.x86_64 27/35 2024-09-17T21:26:45.465 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librgw2-2:19.3.0-4975.g08cec697.el9.x86_64 27/35 2024-09-17T21:26:45.543 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rgw-2:19.3.0-4975.g08cec697.el9.x86_64 28/35 2024-09-17T21:26:45.579 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-common-2:19.3.0-4975.g08cec697.el9.x8 29/35 2024-09-17T21:26:46.579 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.3.0-4975.g08cec697.el9.x86_64 30/35 2024-09-17T21:26:46.615 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-common-2:19.3.0-4975.g08cec697.el9.x86_64 30/35 2024-09-17T21:26:47.189 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.3.0-4975.g08cec697.el9.x86_64 30/35 2024-09-17T21:26:47.228 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-base-2:19.3.0-4975.g08cec697.el9.x86_64 31/35 2024-09-17T21:26:47.318 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-base-2:19.3.0-4975.g08cec697.el9.x86_64 31/35 2024-09-17T21:26:47.318 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T21:26:47.319 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T21:26:47.319 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:26:47.347 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-selinux-2:19.3.0-4975.g08cec697.el9.x86_64 32/35 2024-09-17T21:26:47.951 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.3.0-4975.g08cec697.el9.x86_64 30/35 2024-09-17T21:26:47.993 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-common-2:19.3.0-4975.g08cec697.el9.x86_64 30/35 2024-09-17T21:26:48.556 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.3.0-4975.g08cec697.el9.x86_64 30/35 2024-09-17T21:26:48.597 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-base-2:19.3.0-4975.g08cec697.el9.x86_64 31/35 2024-09-17T21:26:48.694 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-base-2:19.3.0-4975.g08cec697.el9.x86_64 31/35 2024-09-17T21:26:48.694 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T21:26:48.694 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T21:26:48.694 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:26:48.737 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-selinux-2:19.3.0-4975.g08cec697.el9.x86_64 32/35 2024-09-17T21:26:57.605 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4975.g08cec697.el9.x86_64 32/35 2024-09-17T21:26:57.606 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /sys 2024-09-17T21:26:57.606 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /proc 2024-09-17T21:26:57.606 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /mnt 2024-09-17T21:26:57.606 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /var/tmp 2024-09-17T21:26:57.606 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /home 2024-09-17T21:26:57.606 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /root 2024-09-17T21:26:57.606 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /tmp 2024-09-17T21:26:57.606 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:26:58.609 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4975.g08cec697.el9.x86_64 32/35 2024-09-17T21:26:58.609 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /sys 2024-09-17T21:26:58.609 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /proc 2024-09-17T21:26:58.609 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /mnt 2024-09-17T21:26:58.609 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /var/tmp 2024-09-17T21:26:58.610 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /home 2024-09-17T21:26:58.610 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /root 2024-09-17T21:26:58.610 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /tmp 2024-09-17T21:26:58.610 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:26:58.645 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-radosgw-2:19.3.0-4975.g08cec697.el9.x86_64 33/35 2024-09-17T21:26:58.671 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4975.g08cec697.el9.x86_64 33/35 2024-09-17T21:26:58.671 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:26:58.671 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T21:26:58.671 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T21:26:58.671 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T21:26:58.672 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:26:58.673 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T21:26:58.765 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T21:26:58.765 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T21:26:59.640 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-radosgw-2:19.3.0-4975.g08cec697.el9.x86_64 33/35 2024-09-17T21:26:59.665 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4975.g08cec697.el9.x86_64 33/35 2024-09-17T21:26:59.665 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:26:59.665 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T21:26:59.665 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T21:26:59.665 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T21:26:59.665 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:26:59.667 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T21:26:59.750 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T21:26:59.750 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T21:26:59.984 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T21:26:59.984 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-base-2:19.3.0-4975.g08cec697.el9.x86_64 1/35 2024-09-17T21:26:59.984 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-common-2:19.3.0-4975.g08cec697.el9.x86_64 2/35 2024-09-17T21:26:59.985 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-radosgw-2:19.3.0-4975.g08cec697.el9.x86_64 3/35 2024-09-17T21:26:59.985 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-selinux-2:19.3.0-4975.g08cec697.el9.x86_64 4/35 2024-09-17T21:26:59.985 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs2-2:19.3.0-4975.g08cec697.el9.x86_64 5/35 2024-09-17T21:26:59.985 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libradosstriper1-2:19.3.0-4975.g08cec697.el9.x86_6 6/35 2024-09-17T21:26:59.985 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librgw2-2:19.3.0-4975.g08cec697.el9.x86_64 7/35 2024-09-17T21:26:59.985 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4975.g08cec697.el9. 8/35 2024-09-17T21:26:59.985 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-common-2:19.3.0-4975.g08cec697.el9.x8 9/35 2024-09-17T21:26:59.985 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cephfs-2:19.3.0-4975.g08cec697.el9.x86_64 10/35 2024-09-17T21:26:59.985 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rados-2:19.3.0-4975.g08cec697.el9.x86_64 11/35 2024-09-17T21:26:59.988 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rbd-2:19.3.0-4975.g08cec697.el9.x86_64 12/35 2024-09-17T21:26:59.988 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rgw-2:19.3.0-4975.g08cec697.el9.x86_64 13/35 2024-09-17T21:26:59.988 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-17T21:26:59.988 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-17T21:26:59.988 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-17T21:26:59.988 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-17T21:26:59.988 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-17T21:26:59.988 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-17T21:26:59.988 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T21:26:59.988 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-17T21:26:59.989 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-17T21:26:59.989 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-17T21:26:59.989 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-17T21:26:59.989 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-17T21:26:59.989 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-17T21:26:59.989 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-17T21:26:59.990 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-17T21:26:59.990 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-17T21:26:59.990 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-17T21:26:59.990 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-17T21:26:59.990 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:19.3.0-4975.g08cec697.el9.x86_64 32/35 2024-09-17T21:26:59.990 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-17T21:26:59.990 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:19.3.0-4975.g08cec697.el9.x86_64 34/35 2024-09-17T21:27:00.848 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T21:27:00.848 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:00.848 INFO:teuthology.orchestra.run.smithi171.stdout:Upgraded: 2024-09-17T21:27:00.848 INFO:teuthology.orchestra.run.smithi171.stdout: librados2-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:00.848 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:00.849 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T21:27:00.849 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T21:27:00.849 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:00.849 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:00.849 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:00.849 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:00.849 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T21:27:00.849 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T21:27:00.849 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T21:27:00.849 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:00.849 INFO:teuthology.orchestra.run.smithi171.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-17T21:27:00.850 INFO:teuthology.orchestra.run.smithi171.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T21:27:00.850 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T21:27:00.850 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T21:27:00.850 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:00.850 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T21:27:00.850 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:00.850 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T21:27:00.850 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T21:27:00.850 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T21:27:00.850 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:00.850 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:00.850 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:00.850 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T21:27:00.851 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:00.851 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:00.851 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:00.851 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T21:27:00.851 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T21:27:00.851 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T21:27:00.851 INFO:teuthology.orchestra.run.smithi171.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T21:27:00.851 INFO:teuthology.orchestra.run.smithi171.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T21:27:00.851 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:00.851 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:27:00.924 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T21:27:00.924 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-base-2:19.3.0-4975.g08cec697.el9.x86_64 1/35 2024-09-17T21:27:00.924 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-common-2:19.3.0-4975.g08cec697.el9.x86_64 2/35 2024-09-17T21:27:00.924 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-radosgw-2:19.3.0-4975.g08cec697.el9.x86_64 3/35 2024-09-17T21:27:00.924 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-selinux-2:19.3.0-4975.g08cec697.el9.x86_64 4/35 2024-09-17T21:27:00.924 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs2-2:19.3.0-4975.g08cec697.el9.x86_64 5/35 2024-09-17T21:27:00.924 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libradosstriper1-2:19.3.0-4975.g08cec697.el9.x86_6 6/35 2024-09-17T21:27:00.924 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librgw2-2:19.3.0-4975.g08cec697.el9.x86_64 7/35 2024-09-17T21:27:00.924 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4975.g08cec697.el9. 8/35 2024-09-17T21:27:00.924 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-common-2:19.3.0-4975.g08cec697.el9.x8 9/35 2024-09-17T21:27:00.925 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cephfs-2:19.3.0-4975.g08cec697.el9.x86_64 10/35 2024-09-17T21:27:00.925 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rados-2:19.3.0-4975.g08cec697.el9.x86_64 11/35 2024-09-17T21:27:00.925 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rbd-2:19.3.0-4975.g08cec697.el9.x86_64 12/35 2024-09-17T21:27:00.925 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rgw-2:19.3.0-4975.g08cec697.el9.x86_64 13/35 2024-09-17T21:27:00.925 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-17T21:27:00.925 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-17T21:27:00.925 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-17T21:27:00.925 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-17T21:27:00.925 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-17T21:27:00.925 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-17T21:27:00.926 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T21:27:00.926 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-17T21:27:00.926 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-17T21:27:00.926 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-17T21:27:00.926 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-17T21:27:00.926 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-17T21:27:00.926 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-17T21:27:00.926 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-17T21:27:00.926 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-17T21:27:00.926 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-17T21:27:00.926 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-17T21:27:00.927 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-17T21:27:00.927 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:19.3.0-4975.g08cec697.el9.x86_64 32/35 2024-09-17T21:27:00.927 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-17T21:27:00.927 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:19.3.0-4975.g08cec697.el9.x86_64 34/35 2024-09-17T21:27:01.188 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-test 2024-09-17T21:27:01.783 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:28 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:27:01.805 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T21:27:01.805 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:01.806 INFO:teuthology.orchestra.run.smithi129.stdout:Upgraded: 2024-09-17T21:27:01.806 INFO:teuthology.orchestra.run.smithi129.stdout: librados2-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:01.806 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:01.806 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T21:27:01.806 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T21:27:01.806 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:01.806 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:01.806 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:01.806 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:01.806 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T21:27:01.806 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T21:27:01.806 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T21:27:01.807 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:01.807 INFO:teuthology.orchestra.run.smithi129.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-17T21:27:01.807 INFO:teuthology.orchestra.run.smithi129.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T21:27:01.807 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T21:27:01.807 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T21:27:01.807 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:01.807 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T21:27:01.807 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:01.807 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T21:27:01.807 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T21:27:01.807 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T21:27:01.807 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:01.807 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:01.808 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:01.808 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T21:27:01.808 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:01.808 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:01.808 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:01.808 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T21:27:01.808 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T21:27:01.808 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T21:27:01.808 INFO:teuthology.orchestra.run.smithi129.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T21:27:01.808 INFO:teuthology.orchestra.run.smithi129.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T21:27:01.808 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:01.808 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:27:01.916 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:27:01.917 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:27:01.917 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T21:27:01.917 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:27:01.917 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T21:27:01.917 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test x86_64 2:19.3.0-4975.g08cec697.el9 ceph 46 M 2024-09-17T21:27:01.917 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-17T21:27:01.917 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite x86_64 2:19.3.0-4975.g08cec697.el9 ceph 162 k 2024-09-17T21:27:01.917 INFO:teuthology.orchestra.run.smithi171.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T21:27:01.917 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T21:27:01.918 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:01.918 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T21:27:01.918 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:27:01.918 INFO:teuthology.orchestra.run.smithi171.stdout:Install 4 Packages 2024-09-17T21:27:01.918 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:01.918 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 46 M 2024-09-17T21:27:01.919 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 201 M 2024-09-17T21:27:01.919 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T21:27:02.149 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-test 2024-09-17T21:27:02.158 INFO:teuthology.orchestra.run.smithi171.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-17T21:27:02.184 INFO:teuthology.orchestra.run.smithi171.stdout:(2/4): libcephsqlite-19.3.0-4975.g08cec697.el9. 690 kB/s | 162 kB 00:00 2024-09-17T21:27:02.209 INFO:teuthology.orchestra.run.smithi171.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-17T21:27:02.739 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:29 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:27:02.871 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:27:02.872 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:27:02.872 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T21:27:02.872 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:27:02.872 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T21:27:02.872 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test x86_64 2:19.3.0-4975.g08cec697.el9 ceph 46 M 2024-09-17T21:27:02.872 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T21:27:02.872 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite x86_64 2:19.3.0-4975.g08cec697.el9 ceph 162 k 2024-09-17T21:27:02.872 INFO:teuthology.orchestra.run.smithi129.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T21:27:02.872 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T21:27:02.872 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:02.873 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T21:27:02.873 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:27:02.873 INFO:teuthology.orchestra.run.smithi129.stdout:Install 4 Packages 2024-09-17T21:27:02.873 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:02.873 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 46 M 2024-09-17T21:27:02.873 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 201 M 2024-09-17T21:27:02.873 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T21:27:03.216 INFO:teuthology.orchestra.run.smithi129.stdout:(1/4): libcephsqlite-19.3.0-4975.g08cec697.el9. 763 kB/s | 162 kB 00:00 2024-09-17T21:27:03.360 INFO:teuthology.orchestra.run.smithi171.stdout:(4/4): ceph-test-19.3.0-4975.g08cec697.el9.x86_ 32 MB/s | 46 MB 00:01 2024-09-17T21:27:03.362 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:27:03.362 INFO:teuthology.orchestra.run.smithi171.stdout:Total 32 MB/s | 46 MB 00:01 2024-09-17T21:27:03.390 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T21:27:03.410 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T21:27:03.411 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T21:27:03.591 INFO:teuthology.orchestra.run.smithi129.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 170 kB/s | 64 kB 00:00 2024-09-17T21:27:03.641 INFO:teuthology.orchestra.run.smithi129.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 477 kB/s | 304 kB 00:00 2024-09-17T21:27:03.778 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T21:27:03.779 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T21:27:04.305 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T21:27:04.386 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T21:27:04.408 INFO:teuthology.orchestra.run.smithi129.stdout:(4/4): ceph-test-19.3.0-4975.g08cec697.el9.x86_ 32 MB/s | 46 MB 00:01 2024-09-17T21:27:04.410 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:27:04.410 INFO:teuthology.orchestra.run.smithi129.stdout:Total 30 MB/s | 46 MB 00:01 2024-09-17T21:27:04.425 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T21:27:04.438 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T21:27:04.454 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephsqlite-2:19.3.0-4975.g08cec697.el9.x86_64 3/4 2024-09-17T21:27:04.458 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T21:27:04.458 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T21:27:04.816 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T21:27:04.817 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T21:27:05.371 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T21:27:05.451 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T21:27:05.499 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T21:27:05.527 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephsqlite-2:19.3.0-4975.g08cec697.el9.x86_64 3/4 2024-09-17T21:27:08.823 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4975.g08cec697.el9.x86_64 3/4 2024-09-17T21:27:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-test-2:19.3.0-4975.g08cec697.el9.x86_64 4/4 2024-09-17T21:27:09.878 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4975.g08cec697.el9.x86_64 3/4 2024-09-17T21:27:09.952 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-test-2:19.3.0-4975.g08cec697.el9.x86_64 4/4 2024-09-17T21:27:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-test-2:19.3.0-4975.g08cec697.el9.x86_64 4/4 2024-09-17T21:27:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-test-2:19.3.0-4975.g08cec697.el9.x86_64 1/4 2024-09-17T21:27:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephsqlite-2:19.3.0-4975.g08cec697.el9.x86_64 2/4 2024-09-17T21:27:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T21:27:11.717 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T21:27:11.717 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:11.718 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T21:27:11.718 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:11.718 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:11.718 INFO:teuthology.orchestra.run.smithi171.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T21:27:11.718 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T21:27:11.718 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:11.718 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:27:12.000 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph 2024-09-17T21:27:12.369 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-test-2:19.3.0-4975.g08cec697.el9.x86_64 4/4 2024-09-17T21:27:12.369 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-test-2:19.3.0-4975.g08cec697.el9.x86_64 1/4 2024-09-17T21:27:12.370 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephsqlite-2:19.3.0-4975.g08cec697.el9.x86_64 2/4 2024-09-17T21:27:12.370 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T21:27:12.589 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:27:12.713 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T21:27:12.713 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:12.713 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T21:27:12.713 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:12.713 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:12.714 INFO:teuthology.orchestra.run.smithi129.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T21:27:12.714 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T21:27:12.714 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:12.714 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:27:12.736 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:27:12.739 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:27:12.739 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T21:27:12.739 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:27:12.739 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T21:27:12.739 INFO:teuthology.orchestra.run.smithi171.stdout: ceph x86_64 2:19.3.0-4975.g08cec697.el9 ceph 6.6 k 2024-09-17T21:27:12.739 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-17T21:27:12.740 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds x86_64 2:19.3.0-4975.g08cec697.el9 ceph 2.4 M 2024-09-17T21:27:12.740 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr x86_64 2:19.3.0-4975.g08cec697.el9 ceph 1.5 M 2024-09-17T21:27:12.740 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4975.g08cec697.el9 ceph-noarch 291 k 2024-09-17T21:27:12.740 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon x86_64 2:19.3.0-4975.g08cec697.el9 ceph 4.8 M 2024-09-17T21:27:12.740 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd x86_64 2:19.3.0-4975.g08cec697.el9 ceph 17 M 2024-09-17T21:27:12.740 INFO:teuthology.orchestra.run.smithi171.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T21:27:12.740 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T21:27:12.740 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T21:27:12.740 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T21:27:12.740 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T21:27:12.740 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T21:27:12.740 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T21:27:12.741 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T21:27:12.741 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T21:27:12.741 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T21:27:12.741 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections 2024-09-17T21:27:12.741 INFO:teuthology.orchestra.run.smithi171.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-17T21:27:12.741 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T21:27:12.741 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T21:27:12.741 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T21:27:12.741 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T21:27:12.741 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T21:27:12.741 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T21:27:12.741 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T21:27:12.741 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T21:27:12.742 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T21:27:12.742 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T21:27:12.742 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T21:27:12.742 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T21:27:12.742 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T21:27:12.742 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T21:27:12.742 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T21:27:12.742 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T21:27:12.742 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T21:27:12.742 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T21:27:12.742 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T21:27:12.743 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T21:27:12.743 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:12.743 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T21:27:12.743 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:27:12.743 INFO:teuthology.orchestra.run.smithi171.stdout:Install 37 Packages 2024-09-17T21:27:12.743 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:12.743 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 31 M 2024-09-17T21:27:12.744 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 109 M 2024-09-17T21:27:12.744 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T21:27:13.019 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph 2024-09-17T21:27:13.633 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:40 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:27:13.738 INFO:teuthology.orchestra.run.smithi171.stdout:(1/37): ceph-19.3.0-4975.g08cec697.el9.x86_64.r 44 kB/s | 6.6 kB 00:00 2024-09-17T21:27:13.781 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:27:13.783 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:27:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T21:27:13.784 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:27:13.784 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T21:27:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: ceph x86_64 2:19.3.0-4975.g08cec697.el9 ceph 6.6 k 2024-09-17T21:27:13.784 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T21:27:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds x86_64 2:19.3.0-4975.g08cec697.el9 ceph 2.4 M 2024-09-17T21:27:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr x86_64 2:19.3.0-4975.g08cec697.el9 ceph 1.5 M 2024-09-17T21:27:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4975.g08cec697.el9 ceph-noarch 291 k 2024-09-17T21:27:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon x86_64 2:19.3.0-4975.g08cec697.el9 ceph 4.8 M 2024-09-17T21:27:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd x86_64 2:19.3.0-4975.g08cec697.el9 ceph 17 M 2024-09-17T21:27:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T21:27:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T21:27:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T21:27:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T21:27:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T21:27:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T21:27:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T21:27:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T21:27:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T21:27:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T21:27:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections 2024-09-17T21:27:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-17T21:27:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T21:27:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T21:27:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T21:27:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T21:27:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T21:27:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T21:27:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T21:27:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T21:27:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T21:27:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T21:27:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T21:27:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T21:27:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T21:27:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T21:27:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T21:27:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T21:27:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T21:27:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T21:27:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T21:27:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T21:27:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:13.787 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T21:27:13.787 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:27:13.788 INFO:teuthology.orchestra.run.smithi129.stdout:Install 37 Packages 2024-09-17T21:27:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:13.788 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 31 M 2024-09-17T21:27:13.788 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 109 M 2024-09-17T21:27:13.789 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T21:27:13.905 INFO:teuthology.orchestra.run.smithi171.stdout:(2/37): ceph-mgr-19.3.0-4975.g08cec697.el9.x86_ 4.8 MB/s | 1.5 MB 00:00 2024-09-17T21:27:13.956 INFO:teuthology.orchestra.run.smithi171.stdout:(3/37): ceph-mds-19.3.0-4975.g08cec697.el9.x86_ 6.5 MB/s | 2.4 MB 00:00 2024-09-17T21:27:14.048 INFO:teuthology.orchestra.run.smithi171.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4975.g08ce 3.1 MB/s | 291 kB 00:00 2024-09-17T21:27:14.131 INFO:teuthology.orchestra.run.smithi171.stdout:(5/37): ceph-mon-19.3.0-4975.g08cec697.el9.x86_ 12 MB/s | 4.8 MB 00:00 2024-09-17T21:27:14.232 INFO:teuthology.orchestra.run.smithi171.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-17T21:27:14.273 INFO:teuthology.orchestra.run.smithi171.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-17T21:27:14.307 INFO:teuthology.orchestra.run.smithi171.stdout:(8/37): python3-cryptography-36.0.1-4.el9.x86_6 7.0 MB/s | 1.2 MB 00:00 2024-09-17T21:27:14.332 INFO:teuthology.orchestra.run.smithi171.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.2 MB/s | 135 kB 00:00 2024-09-17T21:27:14.358 INFO:teuthology.orchestra.run.smithi171.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-09-17T21:27:14.383 INFO:teuthology.orchestra.run.smithi171.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 4.1 MB/s | 215 kB 00:00 2024-09-17T21:27:14.409 INFO:teuthology.orchestra.run.smithi171.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 3.6 MB/s | 188 kB 00:00 2024-09-17T21:27:14.415 INFO:teuthology.orchestra.run.smithi129.stdout:(1/37): ceph-19.3.0-4975.g08cec697.el9.x86_64.r 43 kB/s | 6.6 kB 00:00 2024-09-17T21:27:14.434 INFO:teuthology.orchestra.run.smithi171.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-17T21:27:14.460 INFO:teuthology.orchestra.run.smithi171.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 692 kB/s | 35 kB 00:00 2024-09-17T21:27:14.485 INFO:teuthology.orchestra.run.smithi171.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.5 MB/s | 77 kB 00:00 2024-09-17T21:27:14.511 INFO:teuthology.orchestra.run.smithi171.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-17T21:27:14.536 INFO:teuthology.orchestra.run.smithi171.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 823 kB/s | 42 kB 00:00 2024-09-17T21:27:14.607 INFO:teuthology.orchestra.run.smithi129.stdout:(2/37): ceph-mgr-19.3.0-4975.g08cec697.el9.x86_ 4.4 MB/s | 1.5 MB 00:00 2024-09-17T21:27:14.666 INFO:teuthology.orchestra.run.smithi129.stdout:(3/37): ceph-mds-19.3.0-4975.g08cec697.el9.x86_ 5.9 MB/s | 2.4 MB 00:00 2024-09-17T21:27:14.712 INFO:teuthology.orchestra.run.smithi171.stdout:(18/37): ceph-osd-19.3.0-4975.g08cec697.el9.x86 21 MB/s | 17 MB 00:00 2024-09-17T21:27:14.750 INFO:teuthology.orchestra.run.smithi129.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4975.g08ce 3.4 MB/s | 291 kB 00:00 2024-09-17T21:27:14.753 INFO:teuthology.orchestra.run.smithi171.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 697 kB/s | 151 kB 00:00 2024-09-17T21:27:14.787 INFO:teuthology.orchestra.run.smithi171.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 591 kB/s | 43 kB 00:00 2024-09-17T21:27:14.812 INFO:teuthology.orchestra.run.smithi171.stdout:(21/37): lua-devel-5.4.4-4.el9.x86_64.rpm 74 kB/s | 22 kB 00:00 2024-09-17T21:27:14.838 INFO:teuthology.orchestra.run.smithi171.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-17T21:27:14.872 INFO:teuthology.orchestra.run.smithi171.stdout:(23/37): python3-cherrypy-18.6.1-2.el9.noarch.r 4.2 MB/s | 358 kB 00:00 2024-09-17T21:27:14.897 INFO:teuthology.orchestra.run.smithi171.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 343 kB/s | 18 kB 00:00 2024-09-17T21:27:14.922 INFO:teuthology.orchestra.run.smithi171.stdout:(25/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 97 kB/s | 11 kB 00:00 2024-09-17T21:27:14.948 INFO:teuthology.orchestra.run.smithi171.stdout:(26/37): python3-jaraco-collections-3.0.0-8.el9 312 kB/s | 23 kB 00:00 2024-09-17T21:27:14.973 INFO:teuthology.orchestra.run.smithi171.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 256 kB/s | 19 kB 00:00 2024-09-17T21:27:14.975 INFO:teuthology.orchestra.run.smithi129.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-17T21:27:14.998 INFO:teuthology.orchestra.run.smithi171.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 259 kB/s | 20 kB 00:00 2024-09-17T21:27:15.024 INFO:teuthology.orchestra.run.smithi171.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 610 kB/s | 46 kB 00:00 2024-09-17T21:27:15.049 INFO:teuthology.orchestra.run.smithi171.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-17T21:27:15.050 INFO:teuthology.orchestra.run.smithi129.stdout:(6/37): ceph-mon-19.3.0-4975.g08cec697.el9.x86_ 7.5 MB/s | 4.8 MB 00:00 2024-09-17T21:27:15.074 INFO:teuthology.orchestra.run.smithi171.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-17T21:27:15.100 INFO:teuthology.orchestra.run.smithi171.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-17T21:27:15.125 INFO:teuthology.orchestra.run.smithi171.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 705 kB/s | 36 kB 00:00 2024-09-17T21:27:15.151 INFO:teuthology.orchestra.run.smithi171.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-17T21:27:15.176 INFO:teuthology.orchestra.run.smithi171.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-17T21:27:15.193 INFO:teuthology.orchestra.run.smithi171.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 475 kB/s | 20 kB 00:00 2024-09-17T21:27:15.218 INFO:teuthology.orchestra.run.smithi129.stdout:(7/37): ceph-osd-19.3.0-4975.g08cec697.el9.x86_ 28 MB/s | 17 MB 00:00 2024-09-17T21:27:15.218 INFO:teuthology.orchestra.run.smithi171.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.5 MB/s | 427 kB 00:00 2024-09-17T21:27:15.222 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:27:15.222 INFO:teuthology.orchestra.run.smithi171.stdout:Total 12 MB/s | 31 MB 00:02 2024-09-17T21:27:15.318 INFO:teuthology.orchestra.run.smithi129.stdout:(8/37): python3-cryptography-36.0.1-4.el9.x86_6 3.6 MB/s | 1.2 MB 00:00 2024-09-17T21:27:15.366 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T21:27:15.368 INFO:teuthology.orchestra.run.smithi129.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-17T21:27:15.393 INFO:teuthology.orchestra.run.smithi129.stdout:(10/37): python3-ply-3.11-14.el9.noarch.rpm 311 kB/s | 106 kB 00:00 2024-09-17T21:27:15.406 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T21:27:15.406 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T21:27:15.419 INFO:teuthology.orchestra.run.smithi129.stdout:(11/37): python3-pycparser-2.20-6.el9.noarch.rp 673 kB/s | 135 kB 00:00 2024-09-17T21:27:15.444 INFO:teuthology.orchestra.run.smithi129.stdout:(12/37): python3-urllib3-1.26.5-6.el9.noarch.rp 2.8 MB/s | 215 kB 00:00 2024-09-17T21:27:15.900 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T21:27:15.900 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T21:27:16.003 INFO:teuthology.orchestra.run.smithi129.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 62 kB/s | 35 kB 00:00 2024-09-17T21:27:16.203 INFO:teuthology.orchestra.run.smithi129.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 220 kB/s | 172 kB 00:00 2024-09-17T21:27:16.228 INFO:teuthology.orchestra.run.smithi129.stdout:(15/37): lua-5.4.4-4.el9.x86_64.rpm 226 kB/s | 188 kB 00:00 2024-09-17T21:27:16.646 INFO:teuthology.orchestra.run.smithi129.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 120 kB/s | 77 kB 00:00 2024-09-17T21:27:16.720 INFO:teuthology.orchestra.run.smithi129.stdout:(17/37): lua-devel-5.4.4-4.el9.x86_64.rpm 299 kB/s | 22 kB 00:00 2024-09-17T21:27:16.771 INFO:teuthology.orchestra.run.smithi129.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-17T21:27:16.796 INFO:teuthology.orchestra.run.smithi129.stdout:(19/37): python3-toml-0.10.2-6.el9.noarch.rpm 73 kB/s | 42 kB 00:00 2024-09-17T21:27:16.821 INFO:teuthology.orchestra.run.smithi129.stdout:(20/37): python3-pytz-2021.1-5.el9.noarch.rpm 83 kB/s | 51 kB 00:00 2024-09-17T21:27:16.855 INFO:teuthology.orchestra.run.smithi129.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 519 kB/s | 43 kB 00:00 2024-09-17T21:27:16.889 INFO:teuthology.orchestra.run.smithi129.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 448 kB/s | 11 kB 00:00 2024-09-17T21:27:16.914 INFO:teuthology.orchestra.run.smithi129.stdout:(23/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-17T21:27:16.947 INFO:teuthology.orchestra.run.smithi129.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 312 kB/s | 18 kB 00:00 2024-09-17T21:27:16.973 INFO:teuthology.orchestra.run.smithi129.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 395 kB/s | 23 kB 00:00 2024-09-17T21:27:16.998 INFO:teuthology.orchestra.run.smithi129.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-09-17T21:27:17.024 INFO:teuthology.orchestra.run.smithi129.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-17T21:27:17.049 INFO:teuthology.orchestra.run.smithi129.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-09-17T21:27:17.074 INFO:teuthology.orchestra.run.smithi129.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-17T21:27:17.100 INFO:teuthology.orchestra.run.smithi129.stdout:(30/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-17T21:27:17.125 INFO:teuthology.orchestra.run.smithi129.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-17T21:27:17.150 INFO:teuthology.orchestra.run.smithi129.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-17T21:27:17.175 INFO:teuthology.orchestra.run.smithi129.stdout:(33/37): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-09-17T21:27:17.201 INFO:teuthology.orchestra.run.smithi129.stdout:(34/37): python3-tempora-5.0.0-2.el9.noarch.rpm 473 kB/s | 36 kB 00:00 2024-09-17T21:27:17.226 INFO:teuthology.orchestra.run.smithi129.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 802 kB/s | 20 kB 00:00 2024-09-17T21:27:17.251 INFO:teuthology.orchestra.run.smithi129.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-17T21:27:17.277 INFO:teuthology.orchestra.run.smithi129.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-17T21:27:17.282 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:27:17.282 INFO:teuthology.orchestra.run.smithi129.stdout:Total 8.8 MB/s | 31 MB 00:03 2024-09-17T21:27:17.295 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T21:27:17.364 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T21:27:17.406 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T21:27:17.437 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T21:27:17.449 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T21:27:17.477 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T21:27:17.477 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T21:27:17.522 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T21:27:17.557 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T21:27:17.590 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T21:27:17.683 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T21:27:17.770 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T21:27:17.810 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T21:27:17.846 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T21:27:17.875 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T21:27:17.918 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T21:27:17.952 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T21:27:17.958 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T21:27:17.958 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T21:27:17.993 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T21:27:18.036 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T21:27:18.071 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T21:27:18.119 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T21:27:18.152 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T21:27:18.225 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T21:27:18.372 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T21:27:18.454 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T21:27:18.507 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T21:27:18.553 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T21:27:18.611 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T21:27:18.666 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T21:27:18.860 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T21:27:18.908 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T21:27:18.989 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T21:27:19.076 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T21:27:19.116 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T21:27:19.246 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T21:27:19.311 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T21:27:19.380 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T21:27:19.407 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4975.g08cec697.el9. 32/37 2024-09-17T21:27:19.430 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T21:27:19.436 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-2:19.3.0-4975.g08cec697.el9.x86_64 33/37 2024-09-17T21:27:19.462 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4975.g08cec697.el9.x86_64 33/37 2024-09-17T21:27:19.463 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:27:19.463 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T21:27:19.463 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T21:27:19.463 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T21:27:19.463 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:19.474 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T21:27:19.552 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T21:27:19.598 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T21:27:19.631 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T21:27:19.721 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T21:27:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T21:27:19.855 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T21:27:19.890 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T21:27:19.920 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T21:27:19.954 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T21:27:19.988 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T21:27:20.037 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T21:27:20.072 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T21:27:20.107 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T21:27:20.155 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T21:27:20.196 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T21:27:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T21:27:20.408 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T21:27:20.496 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T21:27:20.550 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T21:27:20.588 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T21:27:20.646 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T21:27:20.717 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T21:27:20.921 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T21:27:20.937 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-osd-2:19.3.0-4975.g08cec697.el9.x86_64 34/37 2024-09-17T21:27:20.964 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-osd-2:19.3.0-4975.g08cec697.el9.x86_64 34/37 2024-09-17T21:27:20.965 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:27:20.965 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T21:27:20.965 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T21:27:20.965 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T21:27:20.965 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:20.967 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T21:27:21.053 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T21:27:21.141 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T21:27:21.183 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T21:27:21.317 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T21:27:21.430 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mon-2:19.3.0-4975.g08cec697.el9.x86_64 35/37 2024-09-17T21:27:21.454 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mon-2:19.3.0-4975.g08cec697.el9.x86_64 35/37 2024-09-17T21:27:21.454 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:27:21.454 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T21:27:21.454 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T21:27:21.454 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T21:27:21.454 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:21.480 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4975.g08cec697.el9. 32/37 2024-09-17T21:27:21.511 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-2:19.3.0-4975.g08cec697.el9.x86_64 33/37 2024-09-17T21:27:21.538 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4975.g08cec697.el9.x86_64 33/37 2024-09-17T21:27:21.538 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:27:21.538 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T21:27:21.538 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T21:27:21.538 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T21:27:21.538 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mds-2:19.3.0-4975.g08cec697.el9.x86_64 36/37 2024-09-17T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mds-2:19.3.0-4975.g08cec697.el9.x86_64 36/37 2024-09-17T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T21:27:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:21.778 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-2:19.3.0-4975.g08cec697.el9.x86_64 37/37 2024-09-17T21:27:22.995 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-osd-2:19.3.0-4975.g08cec697.el9.x86_64 34/37 2024-09-17T21:27:23.022 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-osd-2:19.3.0-4975.g08cec697.el9.x86_64 34/37 2024-09-17T21:27:23.022 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:27:23.022 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T21:27:23.022 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T21:27:23.022 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T21:27:23.023 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:23.479 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mon-2:19.3.0-4975.g08cec697.el9.x86_64 35/37 2024-09-17T21:27:23.503 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mon-2:19.3.0-4975.g08cec697.el9.x86_64 35/37 2024-09-17T21:27:23.503 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:27:23.503 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T21:27:23.503 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T21:27:23.503 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T21:27:23.503 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:23.746 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mds-2:19.3.0-4975.g08cec697.el9.x86_64 36/37 2024-09-17T21:27:23.770 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mds-2:19.3.0-4975.g08cec697.el9.x86_64 36/37 2024-09-17T21:27:23.770 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:27:23.770 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T21:27:23.770 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T21:27:23.770 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T21:27:23.770 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:23.783 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-2:19.3.0-4975.g08cec697.el9.x86_64 37/37 2024-09-17T21:27:23.783 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-2:19.3.0-4975.g08cec697.el9.x86_64 1/37 2024-09-17T21:27:23.783 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mds-2:19.3.0-4975.g08cec697.el9.x86_64 2/37 2024-09-17T21:27:23.783 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-2:19.3.0-4975.g08cec697.el9.x86_64 3/37 2024-09-17T21:27:23.783 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mon-2:19.3.0-4975.g08cec697.el9.x86_64 4/37 2024-09-17T21:27:23.783 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-osd-2:19.3.0-4975.g08cec697.el9.x86_64 5/37 2024-09-17T21:27:23.783 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4975.g08cec697.el9. 6/37 2024-09-17T21:27:23.784 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T21:27:23.784 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T21:27:23.784 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T21:27:23.784 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T21:27:23.784 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T21:27:23.784 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T21:27:23.784 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T21:27:23.784 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T21:27:23.784 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T21:27:23.784 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T21:27:23.784 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T21:27:23.784 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T21:27:23.784 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T21:27:23.785 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T21:27:23.785 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T21:27:23.785 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T21:27:23.785 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T21:27:23.785 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T21:27:23.785 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T21:27:23.785 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T21:27:23.785 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T21:27:23.786 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T21:27:23.786 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T21:27:23.786 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T21:27:23.786 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T21:27:23.786 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T21:27:23.786 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T21:27:23.787 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T21:27:23.787 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T21:27:23.787 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T21:27:23.837 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-2:19.3.0-4975.g08cec697.el9.x86_64 37/37 2024-09-17T21:27:24.687 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T21:27:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:24.688 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T21:27:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core-2:19.3.0-4975.g08cec697.el9.noarch 2024-09-17T21:27:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T21:27:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T21:27:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T21:27:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T21:27:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T21:27:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T21:27:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T21:27:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T21:27:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T21:27:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T21:27:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T21:27:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T21:27:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T21:27:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T21:27:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T21:27:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T21:27:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T21:27:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T21:27:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T21:27:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T21:27:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T21:27:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T21:27:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T21:27:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T21:27:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T21:27:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T21:27:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T21:27:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T21:27:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T21:27:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T21:27:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T21:27:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:24.690 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:27:25.097 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-base 2024-09-17T21:27:25.685 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:52 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:27:25.745 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-2:19.3.0-4975.g08cec697.el9.x86_64 37/37 2024-09-17T21:27:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-2:19.3.0-4975.g08cec697.el9.x86_64 1/37 2024-09-17T21:27:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mds-2:19.3.0-4975.g08cec697.el9.x86_64 2/37 2024-09-17T21:27:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-2:19.3.0-4975.g08cec697.el9.x86_64 3/37 2024-09-17T21:27:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mon-2:19.3.0-4975.g08cec697.el9.x86_64 4/37 2024-09-17T21:27:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-osd-2:19.3.0-4975.g08cec697.el9.x86_64 5/37 2024-09-17T21:27:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4975.g08cec697.el9. 6/37 2024-09-17T21:27:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T21:27:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T21:27:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T21:27:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T21:27:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T21:27:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T21:27:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T21:27:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T21:27:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T21:27:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T21:27:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T21:27:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T21:27:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T21:27:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T21:27:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T21:27:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T21:27:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T21:27:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T21:27:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T21:27:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T21:27:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T21:27:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T21:27:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T21:27:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T21:27:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T21:27:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T21:27:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T21:27:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T21:27:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T21:27:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T21:27:25.774 INFO:teuthology.orchestra.run.smithi171.stdout:Package ceph-base-2:19.3.0-4975.g08cec697.el9.x86_64 is already installed. 2024-09-17T21:27:25.817 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:27:25.818 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-17T21:27:25.818 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:27:25.897 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install cephadm 2024-09-17T21:27:26.477 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:53 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:27:26.609 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:27:26.610 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:27:26.610 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T21:27:26.610 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:27:26.610 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T21:27:26.610 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm noarch 2:19.3.0-4975.g08cec697.el9 ceph-noarch 784 k 2024-09-17T21:27:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:26.611 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T21:27:26.611 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:27:26.611 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-17T21:27:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:26.611 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 784 k 2024-09-17T21:27:26.611 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 791 k 2024-09-17T21:27:26.611 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T21:27:26.742 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T21:27:26.742 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:26.742 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T21:27:26.742 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:26.742 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:26.742 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:26.742 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core-2:19.3.0-4975.g08cec697.el9.noarch 2024-09-17T21:27:26.742 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:26.742 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:26.742 INFO:teuthology.orchestra.run.smithi129.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T21:27:26.742 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T21:27:26.742 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T21:27:26.743 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T21:27:26.743 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T21:27:26.743 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T21:27:26.743 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T21:27:26.743 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T21:27:26.743 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T21:27:26.743 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T21:27:26.743 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T21:27:26.743 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T21:27:26.743 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T21:27:26.743 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T21:27:26.743 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T21:27:26.743 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T21:27:26.743 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T21:27:26.744 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T21:27:26.744 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T21:27:26.744 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T21:27:26.744 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T21:27:26.744 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T21:27:26.744 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T21:27:26.744 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T21:27:26.744 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T21:27:26.744 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T21:27:26.744 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T21:27:26.744 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T21:27:26.744 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T21:27:26.745 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T21:27:26.745 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T21:27:26.745 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:26.745 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:27:26.885 INFO:teuthology.orchestra.run.smithi171.stdout:cephadm-19.3.0-4975.g08cec697.el9.noarch.rpm 2.8 MB/s | 784 kB 00:00 2024-09-17T21:27:26.886 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:27:26.886 INFO:teuthology.orchestra.run.smithi171.stdout:Total 2.8 MB/s | 784 kB 00:00 2024-09-17T21:27:26.886 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T21:27:26.891 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T21:27:26.892 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T21:27:26.897 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T21:27:26.897 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T21:27:26.970 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T21:27:27.136 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-base 2024-09-17T21:27:27.457 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephadm-2:19.3.0-4975.g08cec697.el9.noarch 1/1 2024-09-17T21:27:27.540 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephadm-2:19.3.0-4975.g08cec697.el9.noarch 1/1 2024-09-17T21:27:27.721 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:54 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:27:27.809 INFO:teuthology.orchestra.run.smithi129.stdout:Package ceph-base-2:19.3.0-4975.g08cec697.el9.x86_64 is already installed. 2024-09-17T21:27:27.853 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:27:27.853 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T21:27:27.853 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:27:27.893 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephadm-2:19.3.0-4975.g08cec697.el9.noarch 1/1 2024-09-17T21:27:27.922 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install cephadm 2024-09-17T21:27:28.140 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephadm-2:19.3.0-4975.g08cec697.el9.noarch 1/1 2024-09-17T21:27:28.140 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:28.140 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T21:27:28.140 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm-2:19.3.0-4975.g08cec697.el9.noarch 2024-09-17T21:27:28.140 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:28.140 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:27:28.342 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T21:27:28.496 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:55 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:27:28.626 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:27:28.627 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:27:28.627 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T21:27:28.627 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:27:28.627 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T21:27:28.627 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm noarch 2:19.3.0-4975.g08cec697.el9 ceph-noarch 784 k 2024-09-17T21:27:28.627 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:28.627 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T21:27:28.627 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:27:28.627 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T21:27:28.627 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:28.627 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 784 k 2024-09-17T21:27:28.627 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 791 k 2024-09-17T21:27:28.628 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T21:27:28.889 INFO:teuthology.orchestra.run.smithi129.stdout:cephadm-19.3.0-4975.g08cec697.el9.noarch.rpm 2.9 MB/s | 784 kB 00:00 2024-09-17T21:27:28.889 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:27:28.890 INFO:teuthology.orchestra.run.smithi129.stdout:Total 2.9 MB/s | 784 kB 00:00 2024-09-17T21:27:28.890 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T21:27:28.895 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T21:27:28.895 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T21:27:28.901 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T21:27:28.901 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T21:27:28.928 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:55 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:27:28.977 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T21:27:29.065 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:27:29.065 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:27:29.065 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repo Size 2024-09-17T21:27:29.065 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:27:29.065 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T21:27:29.066 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4975.g08cec697.el9 ceph 145 k 2024-09-17T21:27:29.066 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:29.066 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T21:27:29.066 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:27:29.066 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-17T21:27:29.066 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:29.066 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 145 k 2024-09-17T21:27:29.066 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 432 k 2024-09-17T21:27:29.067 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T21:27:29.271 INFO:teuthology.orchestra.run.smithi171.stdout:ceph-immutable-object-cache-19.3.0-4975.g08cec6 708 kB/s | 145 kB 00:00 2024-09-17T21:27:29.272 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:27:29.272 INFO:teuthology.orchestra.run.smithi171.stdout:Total 703 kB/s | 145 kB 00:00 2024-09-17T21:27:29.272 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T21:27:29.280 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T21:27:29.280 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T21:27:29.338 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T21:27:29.338 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T21:27:29.378 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephadm-2:19.3.0-4975.g08cec697.el9.noarch 1/1 2024-09-17T21:27:29.470 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephadm-2:19.3.0-4975.g08cec697.el9.noarch 1/1 2024-09-17T21:27:29.489 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T21:27:29.556 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4975.g08cec697. 1/1 2024-09-17T21:27:29.581 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4975.g08cec697. 1/1 2024-09-17T21:27:29.581 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:27:29.581 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T21:27:29.581 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:29.830 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephadm-2:19.3.0-4975.g08cec697.el9.noarch 1/1 2024-09-17T21:27:30.105 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephadm-2:19.3.0-4975.g08cec697.el9.noarch 1/1 2024-09-17T21:27:30.105 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:30.105 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T21:27:30.105 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm-2:19.3.0-4975.g08cec697.el9.noarch 2024-09-17T21:27:30.105 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:30.105 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:27:30.296 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T21:27:30.476 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4975.g08cec697. 1/1 2024-09-17T21:27:30.476 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:30.476 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T21:27:30.477 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:30.477 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:30.477 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:27:30.694 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-mgr 2024-09-17T21:27:30.886 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:57 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:27:31.020 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:27:31.021 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:27:31.021 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-17T21:27:31.021 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:27:31.021 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T21:27:31.021 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4975.g08cec697.el9 ceph 145 k 2024-09-17T21:27:31.021 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:31.022 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T21:27:31.022 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:27:31.022 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T21:27:31.022 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:31.022 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 145 k 2024-09-17T21:27:31.022 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 432 k 2024-09-17T21:27:31.023 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T21:27:31.231 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-immutable-object-cache-19.3.0-4975.g08cec6 694 kB/s | 145 kB 00:00 2024-09-17T21:27:31.232 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:27:31.232 INFO:teuthology.orchestra.run.smithi129.stdout:Total 688 kB/s | 145 kB 00:00 2024-09-17T21:27:31.233 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T21:27:31.240 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T21:27:31.240 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T21:27:31.288 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:27:31.297 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T21:27:31.297 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T21:27:31.383 INFO:teuthology.orchestra.run.smithi171.stdout:Package ceph-mgr-2:19.3.0-4975.g08cec697.el9.x86_64 is already installed. 2024-09-17T21:27:31.427 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:27:31.429 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-17T21:27:31.429 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:27:31.448 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T21:27:31.491 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T21:27:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4975.g08cec697. 1/1 2024-09-17T21:27:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4975.g08cec697. 1/1 2024-09-17T21:27:31.549 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:27:31.549 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T21:27:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:32.064 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:59 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:27:32.195 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:27:32.196 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:27:32.197 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T21:27:32.197 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:27:32.197 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T21:27:32.197 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4975.g08cec697.el9 ceph-noarch 14 M 2024-09-17T21:27:32.197 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-17T21:27:32.197 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T21:27:32.197 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4975.g08cec697.el9 ceph-noarch 34 k 2024-09-17T21:27:32.197 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4975.g08cec697.el9 ceph-noarch 16 k 2024-09-17T21:27:32.197 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T21:27:32.197 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T21:27:32.198 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T21:27:32.198 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T21:27:32.198 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T21:27:32.198 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T21:27:32.198 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T21:27:32.198 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T21:27:32.198 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:32.198 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T21:27:32.198 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:27:32.198 INFO:teuthology.orchestra.run.smithi171.stdout:Install 12 Packages 2024-09-17T21:27:32.198 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:32.199 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 19 M 2024-09-17T21:27:32.199 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 159 M 2024-09-17T21:27:32.199 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T21:27:32.496 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4975.g08cec697. 1/1 2024-09-17T21:27:32.496 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:32.496 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T21:27:32.496 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:27:32.496 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:32.496 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:27:32.738 INFO:teuthology.orchestra.run.smithi171.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4975.g08 232 kB/s | 34 kB 00:00 2024-09-17T21:27:32.744 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr 2024-09-17T21:27:32.763 INFO:teuthology.orchestra.run.smithi171.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4975.g08c 95 kB/s | 16 kB 00:00 2024-09-17T21:27:32.964 INFO:teuthology.orchestra.run.smithi171.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.3 MB/s | 269 kB 00:00 2024-09-17T21:27:33.089 INFO:teuthology.orchestra.run.smithi171.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.9 MB/s | 1.0 MB 00:00 2024-09-17T21:27:33.256 INFO:teuthology.orchestra.run.smithi171.stdout:(5/12): ceph-mgr-dashboard-19.3.0-4975.g08cec69 21 MB/s | 14 MB 00:00 2024-09-17T21:27:33.281 INFO:teuthology.orchestra.run.smithi171.stdout:(6/12): protobuf-compiler-3.14.0-13.el9.x86_64. 2.7 MB/s | 863 kB 00:00 2024-09-17T21:27:33.323 INFO:teuthology.orchestra.run.smithi171.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-09-17T21:27:33.339 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:00 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:27:33.357 INFO:teuthology.orchestra.run.smithi171.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.0 MB/s | 548 kB 00:00 2024-09-17T21:27:33.391 INFO:teuthology.orchestra.run.smithi171.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 921 kB/s | 31 kB 00:00 2024-09-17T21:27:33.416 INFO:teuthology.orchestra.run.smithi171.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 1.6 MB/s | 144 kB 00:00 2024-09-17T21:27:33.432 INFO:teuthology.orchestra.run.smithi129.stdout:Package ceph-mgr-2:19.3.0-4975.g08cec697.el9.x86_64 is already installed. 2024-09-17T21:27:33.441 INFO:teuthology.orchestra.run.smithi171.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-17T21:27:33.474 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:27:33.475 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T21:27:33.476 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:27:33.533 INFO:teuthology.orchestra.run.smithi171.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 8.3 MB/s | 2.0 MB 00:00 2024-09-17T21:27:33.537 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:27:33.537 INFO:teuthology.orchestra.run.smithi171.stdout:Total 14 MB/s | 19 MB 00:01 2024-09-17T21:27:33.539 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T21:27:33.584 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T21:27:33.603 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T21:27:33.603 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T21:27:33.879 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T21:27:33.879 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T21:27:34.123 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:01 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:27:34.257 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:27:34.259 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:27:34.259 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T21:27:34.259 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:27:34.259 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T21:27:34.259 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4975.g08cec697.el9 ceph-noarch 14 M 2024-09-17T21:27:34.259 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T21:27:34.259 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T21:27:34.259 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4975.g08cec697.el9 ceph-noarch 34 k 2024-09-17T21:27:34.259 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4975.g08cec697.el9 ceph-noarch 16 k 2024-09-17T21:27:34.259 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T21:27:34.260 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T21:27:34.260 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T21:27:34.260 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T21:27:34.260 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T21:27:34.260 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T21:27:34.260 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T21:27:34.260 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T21:27:34.260 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:34.260 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T21:27:34.260 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:27:34.260 INFO:teuthology.orchestra.run.smithi129.stdout:Install 12 Packages 2024-09-17T21:27:34.261 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:34.261 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 19 M 2024-09-17T21:27:34.261 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 159 M 2024-09-17T21:27:34.261 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T21:27:34.474 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T21:27:34.551 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T21:27:34.586 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T21:27:34.628 INFO:teuthology.orchestra.run.smithi129.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4975.g08 217 kB/s | 34 kB 00:00 2024-09-17T21:27:34.634 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T21:27:34.645 INFO:teuthology.orchestra.run.smithi129.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4975.g08c 94 kB/s | 16 kB 00:00 2024-09-17T21:27:34.664 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T21:27:34.762 INFO:teuthology.orchestra.run.smithi129.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.3 MB/s | 269 kB 00:00 2024-09-17T21:27:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T21:27:34.812 INFO:teuthology.orchestra.run.smithi129.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.5 MB/s | 1.0 MB 00:00 2024-09-17T21:27:34.871 INFO:teuthology.orchestra.run.smithi129.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 9.2 MB/s | 548 kB 00:00 2024-09-17T21:27:34.892 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T21:27:34.896 INFO:teuthology.orchestra.run.smithi129.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 787 kB/s | 19 kB 00:00 2024-09-17T21:27:34.985 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T21:27:34.989 INFO:teuthology.orchestra.run.smithi129.stdout:(7/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.7 MB/s | 863 kB 00:00 2024-09-17T21:27:35.037 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T21:27:35.082 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T21:27:35.115 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4975.g08cec697.el9 10/12 2024-09-17T21:27:35.147 INFO:teuthology.orchestra.run.smithi129.stdout:(8/12): ceph-mgr-dashboard-19.3.0-4975.g08cec69 21 MB/s | 14 MB 00:00 2024-09-17T21:27:35.189 INFO:teuthology.orchestra.run.smithi129.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 722 kB/s | 144 kB 00:00 2024-09-17T21:27:35.239 INFO:teuthology.orchestra.run.smithi129.stdout:(10/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 5.9 MB/s | 2.0 MB 00:00 2024-09-17T21:27:35.264 INFO:teuthology.orchestra.run.smithi129.stdout:(11/12): python3-repoze-lru-0.7-16.el9.noarch.r 267 kB/s | 31 kB 00:00 2024-09-17T21:27:35.290 INFO:teuthology.orchestra.run.smithi129.stdout:(12/12): python3-routes-2.5.1-5.el9.noarch.rpm 1.9 MB/s | 188 kB 00:00 2024-09-17T21:27:35.293 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:27:35.294 INFO:teuthology.orchestra.run.smithi129.stdout:Total 19 MB/s | 19 MB 00:01 2024-09-17T21:27:35.342 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T21:27:35.361 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T21:27:35.361 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T21:27:35.629 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T21:27:35.629 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T21:27:36.217 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T21:27:36.307 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T21:27:36.350 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T21:27:36.397 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T21:27:36.436 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T21:27:36.540 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T21:27:36.670 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T21:27:36.755 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T21:27:36.808 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T21:27:36.837 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T21:27:36.869 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4975.g08cec697.el9 10/12 2024-09-17T21:27:38.304 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4975.g08cec697.el 11/12 2024-09-17T21:27:38.355 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4975.g08cec697.el9.noa 12/12 2024-09-17T21:27:40.061 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4975.g08cec697.el 11/12 2024-09-17T21:27:40.116 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4975.g08cec697.el9.noa 12/12 2024-09-17T21:27:40.363 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4975.g08cec697.el9.noa 12/12 2024-09-17T21:27:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4975.g08cec697.el 1/12 2024-09-17T21:27:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4975.g08cec697.el9.noa 2/12 2024-09-17T21:27:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4975.g08cec697.el9 3/12 2024-09-17T21:27:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T21:27:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T21:27:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T21:27:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T21:27:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T21:27:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T21:27:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T21:27:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T21:27:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T21:27:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:40.692 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T21:27:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T21:27:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards-2:19.3.0-4975.g08cec697.el9.noarch 2024-09-17T21:27:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard-2:19.3.0-4975.g08cec697.el9.noarch 2024-09-17T21:27:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts-2:19.3.0-4975.g08cec697.el9.noarch 2024-09-17T21:27:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T21:27:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T21:27:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T21:27:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T21:27:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T21:27:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T21:27:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T21:27:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T21:27:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:40.693 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:27:40.950 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T21:27:41.525 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:08 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:27:41.658 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:27:41.659 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-09-17T21:27:41.659 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T21:27:41.659 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-09-17T21:27:41.660 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T21:27:41.660 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4975.g08cec697.el9 ceph-noarch 7.4 M 2024-09-17T21:27:41.660 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-17T21:27:41.660 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T21:27:41.660 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T21:27:41.660 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T21:27:41.660 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T21:27:41.660 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T21:27:41.660 INFO:teuthology.orchestra.run.smithi171.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T21:27:41.660 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T21:27:41.660 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T21:27:41.660 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T21:27:41.660 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T21:27:41.660 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T21:27:41.661 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:41.661 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T21:27:41.661 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-09-17T21:27:41.661 INFO:teuthology.orchestra.run.smithi171.stdout:Install 12 Packages 2024-09-17T21:27:41.661 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:41.661 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 38 M 2024-09-17T21:27:41.661 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 212 M 2024-09-17T21:27:41.661 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T21:27:41.986 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4975.g08cec697.el9.noa 12/12 2024-09-17T21:27:41.986 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4975.g08cec697.el 1/12 2024-09-17T21:27:41.986 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4975.g08cec697.el9.noa 2/12 2024-09-17T21:27:41.986 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4975.g08cec697.el9 3/12 2024-09-17T21:27:41.986 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T21:27:41.986 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T21:27:41.986 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T21:27:41.986 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T21:27:41.986 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T21:27:41.986 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T21:27:41.986 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T21:27:41.987 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T21:27:42.081 INFO:teuthology.orchestra.run.smithi171.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-17T21:27:42.140 INFO:teuthology.orchestra.run.smithi171.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.4 MB/s | 801 kB 00:00 2024-09-17T21:27:42.320 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T21:27:42.320 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:42.320 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T21:27:42.320 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T21:27:42.320 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards-2:19.3.0-4975.g08cec697.el9.noarch 2024-09-17T21:27:42.320 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard-2:19.3.0-4975.g08cec697.el9.noarch 2024-09-17T21:27:42.320 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts-2:19.3.0-4975.g08cec697.el9.noarch 2024-09-17T21:27:42.321 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T21:27:42.321 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T21:27:42.321 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T21:27:42.321 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T21:27:42.321 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T21:27:42.321 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T21:27:42.321 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T21:27:42.321 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T21:27:42.321 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:42.321 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:27:42.448 INFO:teuthology.orchestra.run.smithi171.stdout:(3/12): ceph-mgr-diskprediction-local-19.3.0-49 15 MB/s | 7.4 MB 00:00 2024-09-17T21:27:42.574 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T21:27:42.707 INFO:teuthology.orchestra.run.smithi171.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 52 kB/s | 33 kB 00:00 2024-09-17T21:27:42.974 INFO:teuthology.orchestra.run.smithi171.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 35 kB/s | 18 kB 00:00 2024-09-17T21:27:43.158 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:10 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:27:43.291 INFO:teuthology.orchestra.run.smithi171.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.6 MB/s | 3.0 MB 00:01 2024-09-17T21:27:43.292 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:27:43.294 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-09-17T21:27:43.294 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T21:27:43.294 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-09-17T21:27:43.294 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T21:27:43.294 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4975.g08cec697.el9 ceph-noarch 7.4 M 2024-09-17T21:27:43.294 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T21:27:43.294 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T21:27:43.294 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T21:27:43.294 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T21:27:43.294 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T21:27:43.294 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T21:27:43.294 INFO:teuthology.orchestra.run.smithi129.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T21:27:43.295 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T21:27:43.295 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T21:27:43.295 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T21:27:43.295 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T21:27:43.295 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T21:27:43.295 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:43.295 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T21:27:43.295 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-09-17T21:27:43.295 INFO:teuthology.orchestra.run.smithi129.stdout:Install 12 Packages 2024-09-17T21:27:43.295 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:43.296 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 38 M 2024-09-17T21:27:43.296 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 212 M 2024-09-17T21:27:43.296 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T21:27:43.308 INFO:teuthology.orchestra.run.smithi171.stdout:(7/12): openblas-0.3.26-2.el9.x86_64.rpm 65 kB/s | 38 kB 00:00 2024-09-17T21:27:43.666 INFO:teuthology.orchestra.run.smithi129.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 880 kB/s | 191 kB 00:00 2024-09-17T21:27:43.733 INFO:teuthology.orchestra.run.smithi129.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 490 kB/s | 33 kB 00:00 2024-09-17T21:27:43.783 INFO:teuthology.orchestra.run.smithi129.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 801 kB 00:00 2024-09-17T21:27:43.925 INFO:teuthology.orchestra.run.smithi129.stdout:(4/12): ceph-mgr-diskprediction-local-19.3.0-49 15 MB/s | 7.4 MB 00:00 2024-09-17T21:27:43.950 INFO:teuthology.orchestra.run.smithi129.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 109 kB/s | 18 kB 00:00 2024-09-17T21:27:43.975 INFO:teuthology.orchestra.run.smithi129.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-17T21:27:44.058 INFO:teuthology.orchestra.run.smithi171.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 320 kB/s | 245 kB 00:00 2024-09-17T21:27:44.076 INFO:teuthology.orchestra.run.smithi129.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-17T21:27:44.142 INFO:teuthology.orchestra.run.smithi171.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 4.2 MB/s | 4.9 MB 00:01 2024-09-17T21:27:44.402 INFO:teuthology.orchestra.run.smithi129.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.5 MB/s | 3.0 MB 00:00 2024-09-17T21:27:44.485 INFO:teuthology.orchestra.run.smithi129.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.9 MB/s | 250 kB 00:00 2024-09-17T21:27:44.509 INFO:teuthology.orchestra.run.smithi171.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.2 MB/s | 5.1 MB 00:01 2024-09-17T21:27:44.909 INFO:teuthology.orchestra.run.smithi171.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 295 kB/s | 250 kB 00:00 2024-09-17T21:27:45.252 INFO:teuthology.orchestra.run.smithi129.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.3 MB/s | 5.1 MB 00:01 2024-09-17T21:27:45.327 INFO:teuthology.orchestra.run.smithi129.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 3.6 MB/s | 4.9 MB 00:01 2024-09-17T21:27:45.643 INFO:teuthology.orchestra.run.smithi171.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-17T21:27:45.644 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:27:45.645 INFO:teuthology.orchestra.run.smithi171.stdout:Total 9.6 MB/s | 38 MB 00:03 2024-09-17T21:27:46.003 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T21:27:46.030 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T21:27:46.030 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T21:27:46.454 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T21:27:46.454 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T21:27:46.585 INFO:teuthology.orchestra.run.smithi129.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.7 MB/s | 16 MB 00:02 2024-09-17T21:27:46.586 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:27:46.587 INFO:teuthology.orchestra.run.smithi129.stdout:Total 12 MB/s | 38 MB 00:03 2024-09-17T21:27:46.944 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T21:27:46.970 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T21:27:46.970 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T21:27:47.213 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T21:27:47.281 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T21:27:47.325 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T21:27:47.395 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T21:27:47.395 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T21:27:47.426 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T21:27:47.460 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T21:27:47.646 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T21:27:47.675 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T21:27:47.752 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T21:27:48.170 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T21:27:48.240 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T21:27:48.284 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T21:27:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T21:27:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T21:27:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T21:27:48.534 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T21:27:48.619 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T21:27:48.651 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T21:27:48.727 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T21:27:49.376 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T21:27:49.498 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T21:27:49.816 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T21:27:50.760 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T21:27:50.859 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T21:27:50.889 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4975.g08cec 12/12 2024-09-17T21:27:51.806 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T21:27:51.844 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4975.g08cec 12/12 2024-09-17T21:27:53.695 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4975.g08cec 12/12 2024-09-17T21:27:53.695 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4975.g08cec 1/12 2024-09-17T21:27:53.695 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T21:27:53.695 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T21:27:53.695 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T21:27:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T21:27:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T21:27:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T21:27:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T21:27:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T21:27:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T21:27:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T21:27:54.153 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T21:27:54.153 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:54.153 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T21:27:54.153 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4975.g08cec697.el9.noarch 2024-09-17T21:27:54.153 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T21:27:54.153 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T21:27:54.154 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T21:27:54.154 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T21:27:54.154 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T21:27:54.154 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T21:27:54.154 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T21:27:54.154 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T21:27:54.154 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T21:27:54.154 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T21:27:54.154 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T21:27:54.154 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:54.154 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:27:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4975.g08cec 12/12 2024-09-17T21:27:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4975.g08cec 1/12 2024-09-17T21:27:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T21:27:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T21:27:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T21:27:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T21:27:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T21:27:54.413 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T21:27:54.413 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T21:27:54.413 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T21:27:54.413 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T21:27:54.413 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T21:27:54.576 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-mgr-rook 2024-09-17T21:27:54.852 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T21:27:54.852 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:54.852 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T21:27:54.852 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4975.g08cec697.el9.noarch 2024-09-17T21:27:54.852 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T21:27:54.852 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T21:27:54.852 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T21:27:54.852 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T21:27:54.852 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T21:27:54.853 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T21:27:54.853 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T21:27:54.853 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T21:27:54.853 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T21:27:54.853 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T21:27:54.853 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T21:27:54.853 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:54.853 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:27:55.155 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:22 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:27:55.243 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-rook 2024-09-17T21:27:55.291 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:27:55.291 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:27:55.291 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T21:27:55.292 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:27:55.292 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T21:27:55.292 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook noarch 2:19.3.0-4975.g08cec697.el9 ceph-noarch 49 k 2024-09-17T21:27:55.292 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-17T21:27:55.292 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T21:27:55.292 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T21:27:55.292 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T21:27:55.292 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T21:27:55.292 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T21:27:55.292 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T21:27:55.292 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T21:27:55.293 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T21:27:55.293 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T21:27:55.293 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T21:27:55.293 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T21:27:55.293 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T21:27:55.293 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:55.293 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T21:27:55.293 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:27:55.293 INFO:teuthology.orchestra.run.smithi171.stdout:Install 13 Packages 2024-09-17T21:27:55.293 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:27:55.294 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 2.2 M 2024-09-17T21:27:55.294 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 26 M 2024-09-17T21:27:55.294 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T21:27:55.834 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:22 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:27:55.922 INFO:teuthology.orchestra.run.smithi171.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 88 kB/s | 26 kB 00:00 2024-09-17T21:27:55.947 INFO:teuthology.orchestra.run.smithi171.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 61 kB/s | 19 kB 00:00 2024-09-17T21:27:55.969 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:27:55.970 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:27:55.970 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T21:27:55.970 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:27:55.970 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T21:27:55.970 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook noarch 2:19.3.0-4975.g08cec697.el9 ceph-noarch 49 k 2024-09-17T21:27:55.970 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T21:27:55.970 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T21:27:55.970 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T21:27:55.971 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T21:27:55.971 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T21:27:55.971 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T21:27:55.971 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T21:27:55.971 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T21:27:55.971 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T21:27:55.971 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T21:27:55.971 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T21:27:55.971 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T21:27:55.971 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T21:27:55.971 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:55.971 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T21:27:55.971 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:27:55.972 INFO:teuthology.orchestra.run.smithi129.stdout:Install 13 Packages 2024-09-17T21:27:55.972 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:27:55.972 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 2.2 M 2024-09-17T21:27:55.972 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 26 M 2024-09-17T21:27:55.972 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T21:27:55.973 INFO:teuthology.orchestra.run.smithi171.stdout:(3/13): ceph-mgr-rook-19.3.0-4975.g08cec697.el9 144 kB/s | 49 kB 00:00 2024-09-17T21:27:56.323 INFO:teuthology.orchestra.run.smithi171.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 423 kB/s | 159 kB 00:00 2024-09-17T21:27:56.381 INFO:teuthology.orchestra.run.smithi171.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 483 kB/s | 222 kB 00:00 2024-09-17T21:27:56.423 INFO:teuthology.orchestra.run.smithi171.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 771 kB/s | 32 kB 00:00 2024-09-17T21:27:56.441 INFO:teuthology.orchestra.run.smithi171.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 456 kB/s | 54 kB 00:00 2024-09-17T21:27:56.457 INFO:teuthology.orchestra.run.smithi171.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 420 kB/s | 14 kB 00:00 2024-09-17T21:27:56.497 INFO:teuthology.orchestra.run.smithi129.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 348 kB/s | 26 kB 00:00 2024-09-17T21:27:56.514 INFO:teuthology.orchestra.run.smithi129.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 212 kB/s | 19 kB 00:00 2024-09-17T21:27:56.524 INFO:teuthology.orchestra.run.smithi171.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 2.6 MB/s | 220 kB 00:00 2024-09-17T21:27:56.558 INFO:teuthology.orchestra.run.smithi171.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-09-17T21:27:56.584 INFO:teuthology.orchestra.run.smithi171.stdout:(11/13): python3-pyasn1-modules-0.4.8-6.el9.noa 456 kB/s | 279 kB 00:00 2024-09-17T21:27:56.598 INFO:teuthology.orchestra.run.smithi129.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 159 kB 00:00 2024-09-17T21:27:56.617 INFO:teuthology.orchestra.run.smithi171.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-09-17T21:27:56.623 INFO:teuthology.orchestra.run.smithi129.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.1 MB/s | 222 kB 00:00 2024-09-17T21:27:56.640 INFO:teuthology.orchestra.run.smithi129.stdout:(5/13): ceph-mgr-rook-19.3.0-4975.g08cec697.el9 228 kB/s | 49 kB 00:00 2024-09-17T21:27:56.651 INFO:teuthology.orchestra.run.smithi171.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.5 MB/s | 1.0 MB 00:00 2024-09-17T21:27:56.653 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:27:56.653 INFO:teuthology.orchestra.run.smithi171.stdout:Total 1.6 MB/s | 2.2 MB 00:01 2024-09-17T21:27:56.665 INFO:teuthology.orchestra.run.smithi129.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.0 MB/s | 279 kB 00:00 2024-09-17T21:27:56.682 INFO:teuthology.orchestra.run.smithi129.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 912 kB/s | 54 kB 00:00 2024-09-17T21:27:56.721 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T21:27:56.724 INFO:teuthology.orchestra.run.smithi129.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 338 kB/s | 14 kB 00:00 2024-09-17T21:27:56.736 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T21:27:56.737 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T21:27:56.749 INFO:teuthology.orchestra.run.smithi129.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 295 kB/s | 32 kB 00:00 2024-09-17T21:27:56.774 INFO:teuthology.orchestra.run.smithi129.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.3 MB/s | 220 kB 00:00 2024-09-17T21:27:56.800 INFO:teuthology.orchestra.run.smithi129.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-17T21:27:56.825 INFO:teuthology.orchestra.run.smithi129.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-17T21:27:56.829 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T21:27:56.829 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T21:27:56.884 INFO:teuthology.orchestra.run.smithi129.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.5 MB/s | 1.0 MB 00:00 2024-09-17T21:27:56.887 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:27:56.887 INFO:teuthology.orchestra.run.smithi129.stdout:Total 2.4 MB/s | 2.2 MB 00:00 2024-09-17T21:27:56.952 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T21:27:56.967 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T21:27:56.967 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T21:27:57.057 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T21:27:57.057 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T21:27:57.347 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T21:27:57.523 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T21:27:57.564 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T21:27:57.586 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T21:27:57.650 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T21:27:57.687 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T21:27:57.729 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T21:27:57.738 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T21:27:57.794 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T21:27:57.839 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T21:27:57.849 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T21:27:57.886 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T21:27:57.928 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T21:27:57.941 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T21:27:57.998 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T21:27:58.037 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T21:27:58.148 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T21:27:58.205 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T21:27:58.715 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T21:27:58.772 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T21:27:58.805 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T21:27:58.854 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T21:27:58.876 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-rook-2:19.3.0-4975.g08cec697.el9.noarch 13/13 2024-09-17T21:27:58.911 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T21:27:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T21:27:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T21:27:59.043 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T21:27:59.065 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-rook-2:19.3.0-4975.g08cec697.el9.noarch 13/13 2024-09-17T21:27:59.673 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4975.g08cec697.el9.noarch 13/13 2024-09-17T21:27:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4975.g08cec697.el9.noarch 1/13 2024-09-17T21:27:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T21:27:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T21:27:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T21:27:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T21:27:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T21:27:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T21:27:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T21:27:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T21:27:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T21:27:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T21:27:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T21:27:59.880 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4975.g08cec697.el9.noarch 13/13 2024-09-17T21:27:59.880 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4975.g08cec697.el9.noarch 1/13 2024-09-17T21:27:59.880 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T21:27:59.880 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T21:27:59.880 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T21:27:59.880 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T21:27:59.880 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T21:27:59.881 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T21:27:59.881 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T21:27:59.881 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T21:27:59.881 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T21:27:59.881 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T21:27:59.881 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T21:28:00.002 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T21:28:00.002 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:00.002 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T21:28:00.002 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook-2:19.3.0-4975.g08cec697.el9.noarch 2024-09-17T21:28:00.002 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T21:28:00.002 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T21:28:00.002 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T21:28:00.003 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T21:28:00.003 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T21:28:00.003 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T21:28:00.003 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T21:28:00.003 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T21:28:00.003 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T21:28:00.003 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T21:28:00.003 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T21:28:00.003 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T21:28:00.003 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:00.003 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:28:00.217 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T21:28:00.217 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:00.217 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T21:28:00.217 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook-2:19.3.0-4975.g08cec697.el9.noarch 2024-09-17T21:28:00.217 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T21:28:00.217 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T21:28:00.217 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T21:28:00.218 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T21:28:00.218 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T21:28:00.218 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T21:28:00.218 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T21:28:00.218 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T21:28:00.218 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T21:28:00.218 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T21:28:00.218 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T21:28:00.218 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T21:28:00.218 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:00.218 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:28:00.237 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T21:28:00.467 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T21:28:00.815 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:27 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:00.951 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:28:00.951 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:00.952 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T21:28:00.952 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:00.952 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T21:28:00.952 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4975.g08cec697.el9 ceph-noarch 154 k 2024-09-17T21:28:00.952 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-17T21:28:00.952 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T21:28:00.952 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T21:28:00.952 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T21:28:00.952 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T21:28:00.952 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T21:28:00.952 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:00.953 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T21:28:00.953 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:00.953 INFO:teuthology.orchestra.run.smithi171.stdout:Install 6 Packages 2024-09-17T21:28:00.953 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:00.953 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 7.0 M 2024-09-17T21:28:00.953 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 33 M 2024-09-17T21:28:00.953 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T21:28:01.053 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:28 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:01.168 INFO:teuthology.orchestra.run.smithi171.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.8 MB/s | 248 kB 00:00 2024-09-17T21:28:01.190 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:28:01.191 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:01.191 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T21:28:01.191 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:01.191 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T21:28:01.191 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4975.g08cec697.el9 ceph-noarch 154 k 2024-09-17T21:28:01.191 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T21:28:01.191 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T21:28:01.191 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T21:28:01.191 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T21:28:01.191 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T21:28:01.191 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T21:28:01.192 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:01.192 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T21:28:01.192 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:01.192 INFO:teuthology.orchestra.run.smithi129.stdout:Install 6 Packages 2024-09-17T21:28:01.192 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:01.192 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 7.0 M 2024-09-17T21:28:01.192 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 33 M 2024-09-17T21:28:01.193 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T21:28:01.226 INFO:teuthology.orchestra.run.smithi171.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-17T21:28:01.252 INFO:teuthology.orchestra.run.smithi171.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-17T21:28:01.277 INFO:teuthology.orchestra.run.smithi171.stdout:(4/6): ceph-mgr-cephadm-19.3.0-4975.g08cec697.e 625 kB/s | 154 kB 00:00 2024-09-17T21:28:01.303 INFO:teuthology.orchestra.run.smithi171.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-17T21:28:01.395 INFO:teuthology.orchestra.run.smithi171.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-17T21:28:01.396 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:28:01.397 INFO:teuthology.orchestra.run.smithi171.stdout:Total 16 MB/s | 7.0 MB 00:00 2024-09-17T21:28:01.467 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T21:28:01.478 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T21:28:01.479 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T21:28:01.496 INFO:teuthology.orchestra.run.smithi129.stdout:(1/6): ceph-mgr-cephadm-19.3.0-4975.g08cec697.e 788 kB/s | 154 kB 00:00 2024-09-17T21:28:01.522 INFO:teuthology.orchestra.run.smithi129.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-09-17T21:28:01.579 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T21:28:01.579 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T21:28:01.580 INFO:teuthology.orchestra.run.smithi129.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.4 MB/s | 548 kB 00:00 2024-09-17T21:28:01.606 INFO:teuthology.orchestra.run.smithi129.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 689 kB/s | 58 kB 00:00 2024-09-17T21:28:01.631 INFO:teuthology.orchestra.run.smithi129.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-17T21:28:01.903 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T21:28:02.069 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T21:28:02.123 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T21:28:02.653 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T21:28:02.742 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T21:28:02.811 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T21:28:02.836 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4975.g08cec697.el9.noarch 6/6 2024-09-17T21:28:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4975.g08cec697.el9.noarch 6/6 2024-09-17T21:28:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4975.g08cec697.el9.noarch 1/6 2024-09-17T21:28:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T21:28:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T21:28:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T21:28:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T21:28:03.881 INFO:teuthology.orchestra.run.smithi129.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.3 MB/s | 6.0 MB 00:02 2024-09-17T21:28:03.883 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:28:03.884 INFO:teuthology.orchestra.run.smithi129.stdout:Total 2.6 MB/s | 7.0 MB 00:02 2024-09-17T21:28:03.894 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T21:28:03.894 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:03.894 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T21:28:03.894 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm-2:19.3.0-4975.g08cec697.el9.noarch 2024-09-17T21:28:03.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T21:28:03.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T21:28:03.895 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T21:28:03.895 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T21:28:03.895 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T21:28:03.895 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:03.895 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:28:03.955 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T21:28:03.966 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T21:28:03.966 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T21:28:04.066 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T21:28:04.067 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T21:28:04.155 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-fuse 2024-09-17T21:28:04.377 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T21:28:04.552 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T21:28:04.608 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T21:28:04.733 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:31 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:04.865 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:28:04.865 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:04.865 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T21:28:04.865 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:04.865 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T21:28:04.866 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse x86_64 2:19.3.0-4975.g08cec697.el9 ceph 900 k 2024-09-17T21:28:04.866 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-17T21:28:04.866 INFO:teuthology.orchestra.run.smithi171.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T21:28:04.866 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:04.866 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T21:28:04.866 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:04.866 INFO:teuthology.orchestra.run.smithi171.stdout:Install 2 Packages 2024-09-17T21:28:04.866 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:04.867 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 980 k 2024-09-17T21:28:04.867 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 2.9 M 2024-09-17T21:28:04.867 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T21:28:05.075 INFO:teuthology.orchestra.run.smithi171.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 501 kB/s | 80 kB 00:00 2024-09-17T21:28:05.122 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T21:28:05.208 INFO:teuthology.orchestra.run.smithi171.stdout:(2/2): ceph-fuse-19.3.0-4975.g08cec697.el9.x86_ 3.0 MB/s | 900 kB 00:00 2024-09-17T21:28:05.209 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:28:05.209 INFO:teuthology.orchestra.run.smithi171.stdout:Total 2.8 MB/s | 980 kB 00:00 2024-09-17T21:28:05.211 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T21:28:05.221 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T21:28:05.243 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T21:28:05.243 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T21:28:05.278 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T21:28:05.304 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4975.g08cec697.el9.noarch 6/6 2024-09-17T21:28:05.324 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T21:28:05.324 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T21:28:05.506 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T21:28:05.653 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T21:28:05.716 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-fuse-2:19.3.0-4975.g08cec697.el9.x86_64 2/2 2024-09-17T21:28:06.059 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4975.g08cec697.el9.noarch 6/6 2024-09-17T21:28:06.059 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4975.g08cec697.el9.noarch 1/6 2024-09-17T21:28:06.059 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T21:28:06.060 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T21:28:06.060 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T21:28:06.060 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T21:28:06.349 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T21:28:06.349 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:06.349 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T21:28:06.350 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm-2:19.3.0-4975.g08cec697.el9.noarch 2024-09-17T21:28:06.350 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T21:28:06.350 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T21:28:06.350 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T21:28:06.350 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T21:28:06.350 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T21:28:06.350 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:06.350 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:28:06.441 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4975.g08cec697.el9.x86_64 2/2 2024-09-17T21:28:06.442 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-fuse-2:19.3.0-4975.g08cec697.el9.x86_64 1/2 2024-09-17T21:28:06.604 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-fuse 2024-09-17T21:28:06.713 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T21:28:06.713 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:06.713 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T21:28:06.713 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse-2:19.3.0-4975.g08cec697.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T21:28:06.713 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:06.713 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:28:06.965 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-volume 2024-09-17T21:28:07.182 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:34 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:07.314 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:28:07.315 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:07.315 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T21:28:07.315 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:07.315 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T21:28:07.315 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse x86_64 2:19.3.0-4975.g08cec697.el9 ceph 900 k 2024-09-17T21:28:07.315 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T21:28:07.315 INFO:teuthology.orchestra.run.smithi129.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T21:28:07.315 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:07.315 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T21:28:07.315 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:07.315 INFO:teuthology.orchestra.run.smithi129.stdout:Install 2 Packages 2024-09-17T21:28:07.316 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:07.316 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 980 k 2024-09-17T21:28:07.316 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 2.9 M 2024-09-17T21:28:07.316 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T21:28:07.553 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:34 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:07.690 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:28:07.691 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:07.691 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T21:28:07.691 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:07.691 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T21:28:07.691 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume noarch 2:19.3.0-4975.g08cec697.el9 ceph-noarch 285 k 2024-09-17T21:28:07.691 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:07.691 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T21:28:07.691 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:07.691 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-17T21:28:07.691 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:07.691 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 285 k 2024-09-17T21:28:07.692 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 1.3 M 2024-09-17T21:28:07.692 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T21:28:07.842 INFO:teuthology.orchestra.run.smithi129.stdout:(1/2): ceph-fuse-19.3.0-4975.g08cec697.el9.x86_ 3.4 MB/s | 900 kB 00:00 2024-09-17T21:28:07.918 INFO:teuthology.orchestra.run.smithi171.stdout:ceph-volume-19.3.0-4975.g08cec697.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-17T21:28:07.918 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:28:07.919 INFO:teuthology.orchestra.run.smithi171.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-17T21:28:07.919 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T21:28:07.927 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T21:28:07.927 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T21:28:07.956 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T21:28:07.956 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T21:28:08.042 INFO:teuthology.orchestra.run.smithi129.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 174 kB/s | 80 kB 00:00 2024-09-17T21:28:08.043 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:28:08.043 INFO:teuthology.orchestra.run.smithi129.stdout:Total 1.3 MB/s | 980 kB 00:00 2024-09-17T21:28:08.055 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T21:28:08.078 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T21:28:08.078 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T21:28:08.160 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T21:28:08.160 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T21:28:08.173 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T21:28:08.237 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-volume-2:19.3.0-4975.g08cec697.el9.noarch 1/1 2024-09-17T21:28:08.253 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-volume-2:19.3.0-4975.g08cec697.el9.noarch 1/1 2024-09-17T21:28:08.254 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:28:08.254 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T21:28:08.254 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:08.351 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T21:28:08.504 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T21:28:08.575 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-fuse-2:19.3.0-4975.g08cec697.el9.x86_64 2/2 2024-09-17T21:28:09.255 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4975.g08cec697.el9.x86_64 2/2 2024-09-17T21:28:09.255 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-fuse-2:19.3.0-4975.g08cec697.el9.x86_64 1/2 2024-09-17T21:28:09.288 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-volume-2:19.3.0-4975.g08cec697.el9.noarch 1/1 2024-09-17T21:28:09.288 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:09.288 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T21:28:09.288 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume-2:19.3.0-4975.g08cec697.el9.noarch 2024-09-17T21:28:09.288 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:09.288 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:28:09.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T21:28:09.540 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:09.540 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T21:28:09.540 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse-2:19.3.0-4975.g08cec697.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T21:28:09.540 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:09.540 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:28:09.595 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install librados-devel 2024-09-17T21:28:09.797 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-volume 2024-09-17T21:28:10.191 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:37 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:10.329 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:28:10.330 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:10.330 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repo Size 2024-09-17T21:28:10.330 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:10.330 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T21:28:10.330 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel x86_64 2:19.3.0-4975.g08cec697.el9 ceph 126 k 2024-09-17T21:28:10.330 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:10.330 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T21:28:10.330 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:10.330 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-17T21:28:10.331 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:10.331 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 126 k 2024-09-17T21:28:10.331 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 456 k 2024-09-17T21:28:10.331 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T21:28:10.395 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:37 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:10.536 INFO:teuthology.orchestra.run.smithi171.stdout:librados-devel-19.3.0-4975.g08cec697.el9.x86_64 618 kB/s | 126 kB 00:00 2024-09-17T21:28:10.537 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:28:10.537 INFO:teuthology.orchestra.run.smithi171.stdout:Total 614 kB/s | 126 kB 00:00 2024-09-17T21:28:10.537 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T21:28:10.537 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:28:10.537 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:10.538 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T21:28:10.538 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:10.538 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T21:28:10.538 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume noarch 2:19.3.0-4975.g08cec697.el9 ceph-noarch 285 k 2024-09-17T21:28:10.538 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:10.538 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T21:28:10.538 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:10.538 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T21:28:10.538 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:10.538 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 285 k 2024-09-17T21:28:10.538 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 1.3 M 2024-09-17T21:28:10.538 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T21:28:10.542 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T21:28:10.542 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T21:28:10.606 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T21:28:10.606 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T21:28:10.771 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T21:28:10.774 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-volume-19.3.0-4975.g08cec697.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-17T21:28:10.775 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:28:10.775 INFO:teuthology.orchestra.run.smithi129.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-17T21:28:10.775 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T21:28:10.783 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T21:28:10.783 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T21:28:10.815 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T21:28:10.815 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T21:28:10.875 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librados-devel-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:11.037 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T21:28:11.110 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-volume-2:19.3.0-4975.g08cec697.el9.noarch 1/1 2024-09-17T21:28:11.126 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-volume-2:19.3.0-4975.g08cec697.el9.noarch 1/1 2024-09-17T21:28:11.127 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:28:11.127 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T21:28:11.127 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados-devel-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:11.568 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados-devel-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:11.568 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:11.568 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T21:28:11.568 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:28:11.568 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:11.568 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:28:11.834 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install libcephfs2 2024-09-17T21:28:12.106 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-volume-2:19.3.0-4975.g08cec697.el9.noarch 1/1 2024-09-17T21:28:12.107 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:12.107 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T21:28:12.107 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume-2:19.3.0-4975.g08cec697.el9.noarch 2024-09-17T21:28:12.107 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:12.107 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:28:12.427 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install librados-devel 2024-09-17T21:28:12.434 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:39 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:12.526 INFO:teuthology.orchestra.run.smithi171.stdout:Package libcephfs2-2:19.3.0-4975.g08cec697.el9.x86_64 is already installed. 2024-09-17T21:28:12.571 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:28:12.572 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-17T21:28:12.572 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:28:12.645 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install libcephfs-devel 2024-09-17T21:28:13.029 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:40 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:13.167 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:28:13.167 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:13.167 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-17T21:28:13.167 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:13.167 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T21:28:13.167 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel x86_64 2:19.3.0-4975.g08cec697.el9 ceph 126 k 2024-09-17T21:28:13.167 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:13.167 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T21:28:13.167 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:13.168 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T21:28:13.168 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:13.168 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 126 k 2024-09-17T21:28:13.168 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 456 k 2024-09-17T21:28:13.168 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T21:28:13.233 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:40 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:13.369 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:28:13.370 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:13.370 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repo Size 2024-09-17T21:28:13.370 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:13.370 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T21:28:13.370 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel x86_64 2:19.3.0-4975.g08cec697.el9 ceph 32 k 2024-09-17T21:28:13.370 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:13.371 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T21:28:13.371 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:13.371 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-17T21:28:13.371 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:13.371 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 32 k 2024-09-17T21:28:13.371 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 142 k 2024-09-17T21:28:13.371 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T21:28:13.373 INFO:teuthology.orchestra.run.smithi129.stdout:librados-devel-19.3.0-4975.g08cec697.el9.x86_64 618 kB/s | 126 kB 00:00 2024-09-17T21:28:13.373 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:28:13.374 INFO:teuthology.orchestra.run.smithi129.stdout:Total 614 kB/s | 126 kB 00:00 2024-09-17T21:28:13.374 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T21:28:13.379 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T21:28:13.379 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T21:28:13.443 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T21:28:13.443 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T21:28:13.526 INFO:teuthology.orchestra.run.smithi171.stdout:libcephfs-devel-19.3.0-4975.g08cec697.el9.x86_6 205 kB/s | 32 kB 00:00 2024-09-17T21:28:13.526 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:28:13.527 INFO:teuthology.orchestra.run.smithi171.stdout:Total 203 kB/s | 32 kB 00:00 2024-09-17T21:28:13.527 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T21:28:13.530 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T21:28:13.531 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T21:28:13.551 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T21:28:13.551 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T21:28:13.592 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T21:28:13.645 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T21:28:13.704 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librados-devel-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:13.747 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs-devel-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:14.091 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:14.159 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados-devel-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:14.324 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs-devel-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:14.324 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:14.324 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T21:28:14.324 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:28:14.324 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:14.324 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:28:14.418 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados-devel-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:14.418 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:14.419 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T21:28:14.419 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:28:14.419 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:14.419 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:28:14.485 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install librados2 2024-09-17T21:28:14.668 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install libcephfs2 2024-09-17T21:28:15.063 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:15.153 INFO:teuthology.orchestra.run.smithi171.stdout:Package librados2-2:19.3.0-4975.g08cec697.el9.x86_64 is already installed. 2024-09-17T21:28:15.196 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:28:15.197 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-17T21:28:15.197 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:28:15.259 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:15.261 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install librbd1 2024-09-17T21:28:15.350 INFO:teuthology.orchestra.run.smithi129.stdout:Package libcephfs2-2:19.3.0-4975.g08cec697.el9.x86_64 is already installed. 2024-09-17T21:28:15.394 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:28:15.395 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T21:28:15.395 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:28:15.465 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install libcephfs-devel 2024-09-17T21:28:15.841 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:15.931 INFO:teuthology.orchestra.run.smithi171.stdout:Package librbd1-2:19.3.0-4975.g08cec697.el9.x86_64 is already installed. 2024-09-17T21:28:15.974 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:28:15.975 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-17T21:28:15.975 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:28:16.050 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:43 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:16.054 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install python3-rados 2024-09-17T21:28:16.185 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:28:16.185 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:16.185 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-17T21:28:16.185 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:16.186 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T21:28:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel x86_64 2:19.3.0-4975.g08cec697.el9 ceph 32 k 2024-09-17T21:28:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:16.186 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T21:28:16.186 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:16.186 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T21:28:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:16.187 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 32 k 2024-09-17T21:28:16.187 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 142 k 2024-09-17T21:28:16.187 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T21:28:16.344 INFO:teuthology.orchestra.run.smithi129.stdout:libcephfs-devel-19.3.0-4975.g08cec697.el9.x86_6 202 kB/s | 32 kB 00:00 2024-09-17T21:28:16.344 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:28:16.345 INFO:teuthology.orchestra.run.smithi129.stdout:Total 200 kB/s | 32 kB 00:00 2024-09-17T21:28:16.345 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T21:28:16.348 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T21:28:16.349 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T21:28:16.369 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T21:28:16.370 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T21:28:16.449 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T21:28:16.574 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs-devel-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:16.639 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:43 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:16.730 INFO:teuthology.orchestra.run.smithi171.stdout:Package python3-rados-2:19.3.0-4975.g08cec697.el9.x86_64 is already installed. 2024-09-17T21:28:16.772 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:28:16.774 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-17T21:28:16.774 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:28:16.844 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install python3-rgw 2024-09-17T21:28:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:17.212 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs-devel-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:17.212 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:17.212 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T21:28:17.212 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:28:17.212 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:17.212 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:28:17.384 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install librados2 2024-09-17T21:28:17.428 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:17.518 INFO:teuthology.orchestra.run.smithi171.stdout:Package python3-rgw-2:19.3.0-4975.g08cec697.el9.x86_64 is already installed. 2024-09-17T21:28:17.562 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:28:17.562 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-17T21:28:17.562 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:28:17.631 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install python3-cephfs 2024-09-17T21:28:17.965 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:18.055 INFO:teuthology.orchestra.run.smithi129.stdout:Package librados2-2:19.3.0-4975.g08cec697.el9.x86_64 is already installed. 2024-09-17T21:28:18.099 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:28:18.100 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T21:28:18.100 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:28:18.175 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install librbd1 2024-09-17T21:28:18.213 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:18.304 INFO:teuthology.orchestra.run.smithi171.stdout:Package python3-cephfs-2:19.3.0-4975.g08cec697.el9.x86_64 is already installed. 2024-09-17T21:28:18.348 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:28:18.349 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-17T21:28:18.349 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:28:18.409 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install python3-rbd 2024-09-17T21:28:18.757 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:18.846 INFO:teuthology.orchestra.run.smithi129.stdout:Package librbd1-2:19.3.0-4975.g08cec697.el9.x86_64 is already installed. 2024-09-17T21:28:18.891 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:28:18.892 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T21:28:18.892 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:28:18.958 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-rados 2024-09-17T21:28:18.983 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:19.072 INFO:teuthology.orchestra.run.smithi171.stdout:Package python3-rbd-2:19.3.0-4975.g08cec697.el9.x86_64 is already installed. 2024-09-17T21:28:19.115 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:28:19.116 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-17T21:28:19.116 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:28:19.190 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install rbd-fuse 2024-09-17T21:28:19.540 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:46 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:19.629 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-rados-2:19.3.0-4975.g08cec697.el9.x86_64 is already installed. 2024-09-17T21:28:19.672 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:28:19.673 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T21:28:19.673 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:28:19.748 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-rgw 2024-09-17T21:28:19.768 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:46 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:19.901 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:28:19.902 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:19.902 INFO:teuthology.orchestra.run.smithi171.stdout: Package Architecture Version Repository Size 2024-09-17T21:28:19.902 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:19.902 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T21:28:19.902 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse x86_64 2:19.3.0-4975.g08cec697.el9 ceph 85 k 2024-09-17T21:28:19.902 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:19.902 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T21:28:19.902 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:19.902 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-17T21:28:19.903 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:19.903 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 85 k 2024-09-17T21:28:19.903 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 231 k 2024-09-17T21:28:19.903 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T21:28:20.122 INFO:teuthology.orchestra.run.smithi171.stdout:rbd-fuse-19.3.0-4975.g08cec697.el9.x86_64.rpm 388 kB/s | 85 kB 00:00 2024-09-17T21:28:20.123 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:28:20.123 INFO:teuthology.orchestra.run.smithi171.stdout:Total 385 kB/s | 85 kB 00:00 2024-09-17T21:28:20.123 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T21:28:20.130 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T21:28:20.130 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T21:28:20.193 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T21:28:20.193 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T21:28:20.338 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:20.339 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T21:28:20.429 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-rgw-2:19.3.0-4975.g08cec697.el9.x86_64 is already installed. 2024-09-17T21:28:20.444 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-fuse-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:20.473 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:28:20.474 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T21:28:20.474 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:28:20.542 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-cephfs 2024-09-17T21:28:20.837 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:21.099 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-fuse-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:21.099 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:21.099 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T21:28:21.099 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:28:21.099 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:21.099 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:28:21.122 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:48 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:21.212 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-cephfs-2:19.3.0-4975.g08cec697.el9.x86_64 is already installed. 2024-09-17T21:28:21.256 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:28:21.257 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T21:28:21.257 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:28:21.332 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-rbd 2024-09-17T21:28:21.348 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install rbd-mirror 2024-09-17T21:28:21.945 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:48 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:21.957 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:48 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:22.038 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-rbd-2:19.3.0-4975.g08cec697.el9.x86_64 is already installed. 2024-09-17T21:28:22.084 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:28:22.085 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T21:28:22.085 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:28:22.119 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:28:22.120 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:22.120 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repo Size 2024-09-17T21:28:22.120 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:22.120 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T21:28:22.120 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror x86_64 2:19.3.0-4975.g08cec697.el9 ceph 3.2 M 2024-09-17T21:28:22.120 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:22.120 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T21:28:22.120 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:22.120 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-17T21:28:22.120 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:22.121 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 3.2 M 2024-09-17T21:28:22.121 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 14 M 2024-09-17T21:28:22.121 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T21:28:22.169 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install rbd-fuse 2024-09-17T21:28:22.449 INFO:teuthology.orchestra.run.smithi171.stdout:rbd-mirror-19.3.0-4975.g08cec697.el9.x86_64.rpm 9.7 MB/s | 3.2 MB 00:00 2024-09-17T21:28:22.450 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:28:22.450 INFO:teuthology.orchestra.run.smithi171.stdout:Total 9.7 MB/s | 3.2 MB 00:00 2024-09-17T21:28:22.450 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T21:28:22.459 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T21:28:22.459 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T21:28:22.543 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T21:28:22.544 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T21:28:22.778 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:49 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:22.916 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:28:22.917 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:22.917 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-09-17T21:28:22.917 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:22.917 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T21:28:22.917 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse x86_64 2:19.3.0-4975.g08cec697.el9 ceph 85 k 2024-09-17T21:28:22.917 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:22.917 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T21:28:22.917 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:22.917 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T21:28:22.917 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:22.917 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 85 k 2024-09-17T21:28:22.917 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 231 k 2024-09-17T21:28:22.918 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T21:28:22.990 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T21:28:23.058 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-mirror-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:23.084 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:23.084 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:28:23.084 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T21:28:23.084 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T21:28:23.085 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T21:28:23.085 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:23.107 INFO:teuthology.orchestra.run.smithi129.stdout:rbd-fuse-19.3.0-4975.g08cec697.el9.x86_64.rpm 450 kB/s | 85 kB 00:00 2024-09-17T21:28:23.108 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:28:23.108 INFO:teuthology.orchestra.run.smithi129.stdout:Total 446 kB/s | 85 kB 00:00 2024-09-17T21:28:23.108 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T21:28:23.115 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T21:28:23.115 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T21:28:23.179 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T21:28:23.180 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T21:28:23.329 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T21:28:23.439 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-fuse-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:23.837 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:24.088 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-fuse-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:24.088 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:24.088 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T21:28:24.088 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:28:24.089 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:24.089 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:28:24.290 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-mirror-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:24.290 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:24.290 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T21:28:24.290 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:28:24.290 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:24.290 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:28:24.375 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install rbd-mirror 2024-09-17T21:28:24.526 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install rbd-nbd 2024-09-17T21:28:24.974 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:51 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:25.113 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:28:25.113 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:25.114 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-17T21:28:25.114 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:25.114 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T21:28:25.114 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror x86_64 2:19.3.0-4975.g08cec697.el9 ceph 3.2 M 2024-09-17T21:28:25.114 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:25.114 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T21:28:25.114 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:25.114 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T21:28:25.114 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:25.115 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 3.2 M 2024-09-17T21:28:25.115 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 14 M 2024-09-17T21:28:25.115 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T21:28:25.127 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:52 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:25.262 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T21:28:25.262 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:25.262 INFO:teuthology.orchestra.run.smithi171.stdout: Package Architecture Version Repository Size 2024-09-17T21:28:25.263 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:25.263 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T21:28:25.263 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd x86_64 2:19.3.0-4975.g08cec697.el9 ceph 170 k 2024-09-17T21:28:25.263 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:25.263 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T21:28:25.263 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T21:28:25.263 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-17T21:28:25.263 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:25.263 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 170 k 2024-09-17T21:28:25.263 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 490 k 2024-09-17T21:28:25.263 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T21:28:25.437 INFO:teuthology.orchestra.run.smithi129.stdout:rbd-mirror-19.3.0-4975.g08cec697.el9.x86_64.rpm 9.9 MB/s | 3.2 MB 00:00 2024-09-17T21:28:25.437 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:28:25.438 INFO:teuthology.orchestra.run.smithi129.stdout:Total 9.9 MB/s | 3.2 MB 00:00 2024-09-17T21:28:25.438 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T21:28:25.447 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T21:28:25.447 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T21:28:25.477 INFO:teuthology.orchestra.run.smithi171.stdout:rbd-nbd-19.3.0-4975.g08cec697.el9.x86_64.rpm 798 kB/s | 170 kB 00:00 2024-09-17T21:28:25.477 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:28:25.478 INFO:teuthology.orchestra.run.smithi171.stdout:Total 792 kB/s | 170 kB 00:00 2024-09-17T21:28:25.478 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T21:28:25.485 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T21:28:25.485 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T21:28:25.528 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T21:28:25.529 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T21:28:25.550 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T21:28:25.550 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T21:28:25.705 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T21:28:25.816 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-nbd-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:25.972 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T21:28:26.037 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-mirror-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:26.063 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:26.063 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:28:26.063 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T21:28:26.064 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T21:28:26.064 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T21:28:26.064 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:26.235 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:26.497 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-nbd-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:26.497 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:26.497 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T21:28:26.498 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:28:26.498 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:26.498 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T21:28:26.757 DEBUG:teuthology.parallel:result is None 2024-09-17T21:28:27.180 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-mirror-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:27.180 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:27.180 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T21:28:27.180 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:28:27.180 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:27.181 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:28:27.432 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install rbd-nbd 2024-09-17T21:28:28.019 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:55 ago on Tue 17 Sep 2024 09:26:33 PM UTC. 2024-09-17T21:28:28.154 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T21:28:28.154 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:28.155 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-09-17T21:28:28.155 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:28.155 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T21:28:28.155 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd x86_64 2:19.3.0-4975.g08cec697.el9 ceph 170 k 2024-09-17T21:28:28.155 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:28.155 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T21:28:28.155 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T21:28:28.155 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T21:28:28.155 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:28.156 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 170 k 2024-09-17T21:28:28.156 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 490 k 2024-09-17T21:28:28.156 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T21:28:28.567 INFO:teuthology.orchestra.run.smithi129.stdout:rbd-nbd-19.3.0-4975.g08cec697.el9.x86_64.rpm 415 kB/s | 170 kB 00:00 2024-09-17T21:28:28.567 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:28:28.568 INFO:teuthology.orchestra.run.smithi129.stdout:Total 413 kB/s | 170 kB 00:00 2024-09-17T21:28:28.568 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T21:28:28.576 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T21:28:28.576 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T21:28:28.640 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T21:28:28.641 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T21:28:28.793 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T21:28:28.915 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-nbd-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:29.316 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:29.563 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-nbd-2:19.3.0-4975.g08cec697.el9.x86_64 1/1 2024-09-17T21:28:29.563 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:29.563 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T21:28:29.563 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd-2:19.3.0-4975.g08cec697.el9.x86_64 2024-09-17T21:28:29.563 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:29.563 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T21:28:29.810 DEBUG:teuthology.parallel:result is None 2024-09-17T21:28:29.811 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=08cec6974d86f81dc304360b31113f30b4085bf8 2024-09-17T21:28:29.955 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T21:28:29.976 INFO:teuthology.orchestra.run.smithi129.stdout:19.3.0-4975.g08cec697.el9 2024-09-17T21:28:29.977 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4975.g08cec697.el9 2024-09-17T21:28:29.977 INFO:teuthology.task.install:The correct ceph version 19.3.0-4975.g08cec697 is installed. 2024-09-17T21:28:29.979 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=08cec6974d86f81dc304360b31113f30b4085bf8 2024-09-17T21:28:30.112 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T21:28:30.137 INFO:teuthology.orchestra.run.smithi171.stdout:19.3.0-4975.g08cec697.el9 2024-09-17T21:28:30.138 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4975.g08cec697.el9 2024-09-17T21:28:30.138 INFO:teuthology.task.install:The correct ceph version 19.3.0-4975.g08cec697 is installed. 2024-09-17T21:28:30.140 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T21:28:30.140 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:28:30.140 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T21:28:30.165 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:28:30.165 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T21:28:30.207 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T21:28:30.207 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:28:30.207 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T21:28:30.231 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T21:28:30.295 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:28:30.296 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T21:28:30.323 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T21:28:30.390 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T21:28:30.390 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:28:30.390 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T21:28:30.414 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T21:28:30.479 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:28:30.479 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T21:28:30.505 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T21:28:30.572 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T21:28:30.572 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:28:30.572 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T21:28:30.597 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T21:28:30.661 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:28:30.661 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T21:28:30.687 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T21:28:30.754 INFO:teuthology.run_tasks:Running task ceph... 2024-09-17T21:28:30.828 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-17T21:28:30.828 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 777 /var/log/ceph 2024-09-17T21:28:30.830 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 777 /var/log/ceph 2024-09-17T21:28:30.855 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-17T21:28:30.855 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T21:28:30.895 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T21:28:30.920 INFO:tasks.ceph:Creating extra log directories... 2024-09-17T21:28:30.920 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T21:28:30.960 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T21:28:30.993 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-17T21:28:30.994 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-17T21:28:30.994 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909352', 'branch': 'main', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7909352', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '08cec6974d86f81dc304360b31113f30b4085bf8'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '08cec6974d86f81dc304360b31113f30b4085bf8'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '5ecc7400dbbb63ce990b663d2a92e87dbc17f30d'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 950, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 4954, 'sha1': '08cec6974d86f81dc304360b31113f30b4085bf8', 'sleep_before_teardown': 0, 'subset': '16/128', 'suite': 'rbd', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_5ecc7400dbbb63ce990b663d2a92e87dbc17f30d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '5ecc7400dbbb63ce990b663d2a92e87dbc17f30d', 'targets': {'smithi129.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBlWv4OLCblN2HmVxOdsCPfdJzju+eEOhxtK05GoDQrZU3Evcgpcndl85WagAwpxVwaCM1lfrIAuDXQh3sPgfC4=', 'smithi171.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLNBgFzCjxfas+H7+r/VLB4Bz3W0KiEyLYSc3gPE5/Vc7TQhda/3+qG5jUiIgzsLXfCs6Ba3jPJMSb/J2J9aBTA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_20:16:19', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-17T21:28:30.994 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T21:28:31.024 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T21:28:31.040 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T21:28:31.078 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T21:28:31.105 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:28:31.105 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T21:28:31.157 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T21:28:31.157 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_1 2024-09-17T21:28:31.212 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T21:28:31.212 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:28:31.212 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-17T21:28:31.213 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:28:31.213 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T21:28:31.213 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-17 21:28:26.314770690 +0000 2024-09-17T21:28:31.213 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-17 21:25:52.675629801 +0000 2024-09-17T21:28:31.213 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-17 21:25:52.675629801 +0000 2024-09-17T21:28:31.213 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-17 21:25:52.675629801 +0000 2024-09-17T21:28:31.213 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T21:28:31.275 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-17T21:28:31.275 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-17T21:28:31.275 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.00018527 s, 2.8 MB/s 2024-09-17T21:28:31.277 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T21:28:31.333 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_2 2024-09-17T21:28:31.389 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T21:28:31.389 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:28:31.389 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-17T21:28:31.389 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:28:31.389 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T21:28:31.389 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-17 21:28:26.315770667 +0000 2024-09-17T21:28:31.389 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-17 21:25:53.002622356 +0000 2024-09-17T21:28:31.389 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-17 21:25:53.002622356 +0000 2024-09-17T21:28:31.389 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-17 21:25:53.002622356 +0000 2024-09-17T21:28:31.390 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T21:28:31.452 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-17T21:28:31.452 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-17T21:28:31.452 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000189392 s, 2.7 MB/s 2024-09-17T21:28:31.454 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T21:28:31.510 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_3 2024-09-17T21:28:31.565 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T21:28:31.566 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:28:31.566 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-17T21:28:31.566 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:28:31.566 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T21:28:31.566 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-17 21:28:26.315770667 +0000 2024-09-17T21:28:31.566 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-17 21:25:53.330614888 +0000 2024-09-17T21:28:31.566 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-17 21:25:53.330614888 +0000 2024-09-17T21:28:31.566 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-17 21:25:53.330614888 +0000 2024-09-17T21:28:31.566 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T21:28:31.628 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-17T21:28:31.629 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-17T21:28:31.629 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000243636 s, 2.1 MB/s 2024-09-17T21:28:31.630 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T21:28:31.687 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_4 2024-09-17T21:28:31.742 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T21:28:31.742 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:28:31.742 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-17T21:28:31.742 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:28:31.742 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T21:28:31.742 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-17 21:28:26.315770667 +0000 2024-09-17T21:28:31.743 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-17 21:25:53.655607489 +0000 2024-09-17T21:28:31.743 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-17 21:25:53.655607489 +0000 2024-09-17T21:28:31.743 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-17 21:25:53.655607489 +0000 2024-09-17T21:28:31.743 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T21:28:31.806 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-17T21:28:31.806 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-17T21:28:31.806 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.00018482 s, 2.8 MB/s 2024-09-17T21:28:31.807 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T21:28:31.863 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T21:28:31.863 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:28:31.863 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T21:28:31.880 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T21:28:31.880 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_1 2024-09-17T21:28:31.937 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T21:28:31.938 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:28:31.938 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-17T21:28:31.938 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:28:31.938 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T21:28:31.938 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-17 21:28:23.342347192 +0000 2024-09-17T21:28:31.938 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-17 21:25:52.676646688 +0000 2024-09-17T21:28:31.938 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-17 21:25:52.676646688 +0000 2024-09-17T21:28:31.938 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-17 21:25:52.676646688 +0000 2024-09-17T21:28:31.938 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T21:28:32.003 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-17T21:28:32.003 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-17T21:28:32.003 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000199317 s, 2.6 MB/s 2024-09-17T21:28:32.004 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T21:28:32.061 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_2 2024-09-17T21:28:32.117 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T21:28:32.117 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:28:32.117 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-17T21:28:32.117 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:28:32.117 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T21:28:32.117 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-17 21:28:23.343347174 +0000 2024-09-17T21:28:32.117 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-17 21:25:53.002640600 +0000 2024-09-17T21:28:32.117 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-17 21:25:53.002640600 +0000 2024-09-17T21:28:32.117 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-17 21:25:53.002640600 +0000 2024-09-17T21:28:32.118 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T21:28:32.181 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-17T21:28:32.181 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-17T21:28:32.181 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000214373 s, 2.4 MB/s 2024-09-17T21:28:32.183 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T21:28:32.239 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_3 2024-09-17T21:28:32.295 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T21:28:32.295 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:28:32.295 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-17T21:28:32.295 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:28:32.295 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T21:28:32.296 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-17 21:28:23.343347174 +0000 2024-09-17T21:28:32.296 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-17 21:25:53.326634549 +0000 2024-09-17T21:28:32.296 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-17 21:25:53.326634549 +0000 2024-09-17T21:28:32.296 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-17 21:25:53.326634549 +0000 2024-09-17T21:28:32.297 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T21:28:32.358 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-17T21:28:32.358 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-17T21:28:32.359 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000241334 s, 2.1 MB/s 2024-09-17T21:28:32.360 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T21:28:32.416 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_4 2024-09-17T21:28:32.471 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T21:28:32.471 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:28:32.471 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-17T21:28:32.471 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:28:32.472 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T21:28:32.472 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-17 21:28:23.343347174 +0000 2024-09-17T21:28:32.472 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-17 21:25:53.644628610 +0000 2024-09-17T21:28:32.472 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-17 21:25:53.644628610 +0000 2024-09-17T21:28:32.472 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-17 21:25:53.644628610 +0000 2024-09-17T21:28:32.472 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T21:28:32.535 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-17T21:28:32.535 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-17T21:28:32.535 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000236 s, 2.2 MB/s 2024-09-17T21:28:32.536 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T21:28:32.592 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T21:28:32.593 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi129.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi171.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T21:28:32.593 INFO:tasks.ceph:Generating config... 2024-09-17T21:28:32.594 INFO:tasks.ceph:[client] rbd cache = True 2024-09-17T21:28:32.594 INFO:tasks.ceph:[client] rbd cache policy = writearound 2024-09-17T21:28:32.594 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-17T21:28:32.594 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-17T21:28:32.594 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-17T21:28:32.594 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-17T21:28:32.594 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-17T21:28:32.595 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-17T21:28:32.595 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-17T21:28:32.595 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-17T21:28:32.595 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-17T21:28:32.595 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-09-17T21:28:32.595 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-17T21:28:32.595 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-17T21:28:32.595 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-17T21:28:32.595 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-17T21:28:32.595 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-17T21:28:32.595 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-17T21:28:32.595 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-17T21:28:32.595 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-17T21:28:32.596 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-17T21:28:32.596 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-17T21:28:32.596 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-17T21:28:32.596 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-17T21:28:32.596 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-17T21:28:32.596 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-17T21:28:32.596 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-17T21:28:32.596 INFO:tasks.ceph:Setting up mon.a... 2024-09-17T21:28:32.596 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-17T21:28:32.645 INFO:teuthology.orchestra.run.smithi129.stdout:creating /etc/ceph/ceph.keyring 2024-09-17T21:28:32.647 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-17T21:28:32.690 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T21:28:32.755 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.129'), ('mon.c', '[v2:172.21.15.129:3301,v1:172.21.15.129:6790]'), ('mon.b', '172.21.15.171')] 2024-09-17T21:28:32.755 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.129,[v2:172.21.15.129:3301,v1:172.21.15.129:6790],172.21.15.171', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache policy': 'writearound'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-17T21:28:32.756 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:28:32.756 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T21:28:32.810 DEBUG:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.129 --addv c '[v2:172.21.15.129:3301,v1:172.21.15.129:6790]' --add b 172.21.15.171 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:28:32.896 INFO:teuthology.orchestra.run.smithi129.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:28:32.896 INFO:teuthology.orchestra.run.smithi129.stdout:monmaptool: generated fsid f6d3a80c-8a1f-4716-b25d-e97b28890c0d 2024-09-17T21:28:32.896 INFO:teuthology.orchestra.run.smithi129.stdout:setting min_mon_release = quincy 2024-09-17T21:28:32.897 INFO:teuthology.orchestra.run.smithi129.stdout:epoch 0 2024-09-17T21:28:32.897 INFO:teuthology.orchestra.run.smithi129.stdout:fsid f6d3a80c-8a1f-4716-b25d-e97b28890c0d 2024-09-17T21:28:32.897 INFO:teuthology.orchestra.run.smithi129.stdout:last_changed 2024-09-17T21:28:32.895304+0000 2024-09-17T21:28:32.897 INFO:teuthology.orchestra.run.smithi129.stdout:created 2024-09-17T21:28:32.895304+0000 2024-09-17T21:28:32.897 INFO:teuthology.orchestra.run.smithi129.stdout:min_mon_release 17 (quincy) 2024-09-17T21:28:32.897 INFO:teuthology.orchestra.run.smithi129.stdout:election_strategy: 1 2024-09-17T21:28:32.897 INFO:teuthology.orchestra.run.smithi129.stdout:0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-09-17T21:28:32.897 INFO:teuthology.orchestra.run.smithi129.stdout:1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.b 2024-09-17T21:28:32.897 INFO:teuthology.orchestra.run.smithi129.stdout:2: [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] mon.c 2024-09-17T21:28:32.897 INFO:teuthology.orchestra.run.smithi129.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-17T21:28:32.898 DEBUG:teuthology.orchestra.run.smithi129:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T21:28:32.913 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f6d3a80c-8a1f-4716-b25d-e97b28890c0d... 2024-09-17T21:28:32.914 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T21:28:32.955 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T21:28:32.998 INFO:teuthology.orchestra.run.smithi129.stdout:[global] 2024-09-17T21:28:32.998 INFO:teuthology.orchestra.run.smithi129.stdout: chdir = "" 2024-09-17T21:28:32.998 INFO:teuthology.orchestra.run.smithi129.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T21:28:32.999 INFO:teuthology.orchestra.run.smithi129.stdout: auth supported = cephx 2024-09-17T21:28:32.999 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:32.999 INFO:teuthology.orchestra.run.smithi129.stdout: filestore xattr use omap = true 2024-09-17T21:28:32.999 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:32.999 INFO:teuthology.orchestra.run.smithi129.stdout: mon clock drift allowed = 1.000 2024-09-17T21:28:32.999 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:32.999 INFO:teuthology.orchestra.run.smithi129.stdout: osd crush chooseleaf type = 0 2024-09-17T21:28:32.999 INFO:teuthology.orchestra.run.smithi129.stdout: auth debug = true 2024-09-17T21:28:32.999 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:32.999 INFO:teuthology.orchestra.run.smithi129.stdout: ms die on old message = true 2024-09-17T21:28:32.999 INFO:teuthology.orchestra.run.smithi129.stdout: ms die on bug = true 2024-09-17T21:28:32.999 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:32.999 INFO:teuthology.orchestra.run.smithi129.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T21:28:33.000 INFO:teuthology.orchestra.run.smithi129.stdout: mon pg warn max object skew = 0 2024-09-17T21:28:33.000 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.000 INFO:teuthology.orchestra.run.smithi129.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T21:28:33.000 INFO:teuthology.orchestra.run.smithi129.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T21:28:33.000 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.000 INFO:teuthology.orchestra.run.smithi129.stdout: osd pool default size = 2 2024-09-17T21:28:33.000 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.000 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd allow primary affinity = true 2024-09-17T21:28:33.000 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd allow pg remap = true 2024-09-17T21:28:33.000 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on legacy crush tunables = false 2024-09-17T21:28:33.000 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on crush straw calc version zero = false 2024-09-17T21:28:33.000 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on no sortbitwise = false 2024-09-17T21:28:33.000 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on osd down out interval zero = false 2024-09-17T21:28:33.001 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on too few osds = false 2024-09-17T21:28:33.001 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T21:28:33.001 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T21:28:33.001 INFO:teuthology.orchestra.run.smithi129.stdout: mon_allow_pool_size_one = true 2024-09-17T21:28:33.001 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.001 INFO:teuthology.orchestra.run.smithi129.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T21:28:33.001 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.001 INFO:teuthology.orchestra.run.smithi129.stdout: osd default data pool replay window = 5 2024-09-17T21:28:33.001 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.001 INFO:teuthology.orchestra.run.smithi129.stdout: mon allow pool delete = true 2024-09-17T21:28:33.001 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.001 INFO:teuthology.orchestra.run.smithi129.stdout: mon cluster log file level = debug 2024-09-17T21:28:33.001 INFO:teuthology.orchestra.run.smithi129.stdout: debug asserts on shutdown = true 2024-09-17T21:28:33.001 INFO:teuthology.orchestra.run.smithi129.stdout: mon health detail to clog = false 2024-09-17T21:28:33.002 INFO:teuthology.orchestra.run.smithi129.stdout: mon host = "172.21.15.129,[v2:172.21.15.129:3301,v1:172.21.15.129:6790],172.21.15.171" 2024-09-17T21:28:33.002 INFO:teuthology.orchestra.run.smithi129.stdout: mon client directed command retry = 5 2024-09-17T21:28:33.002 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on pool no app = False 2024-09-17T21:28:33.002 INFO:teuthology.orchestra.run.smithi129.stdout: ms inject socket failures = 5000 2024-09-17T21:28:33.002 INFO:teuthology.orchestra.run.smithi129.stdout: fsid = f6d3a80c-8a1f-4716-b25d-e97b28890c0d 2024-09-17T21:28:33.002 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.002 INFO:teuthology.orchestra.run.smithi129.stdout:[osd] 2024-09-17T21:28:33.002 INFO:teuthology.orchestra.run.smithi129.stdout: osd journal size = 100 2024-09-17T21:28:33.002 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.002 INFO:teuthology.orchestra.run.smithi129.stdout: osd scrub load threshold = 5.0 2024-09-17T21:28:33.002 INFO:teuthology.orchestra.run.smithi129.stdout: osd scrub max interval = 600 2024-09-17T21:28:33.002 INFO:teuthology.orchestra.run.smithi129.stdout: osd mclock profile = high_recovery_ops 2024-09-17T21:28:33.002 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.002 INFO:teuthology.orchestra.run.smithi129.stdout: osd recover clone overlap = true 2024-09-17T21:28:33.003 INFO:teuthology.orchestra.run.smithi129.stdout: osd recovery max chunk = 1048576 2024-09-17T21:28:33.003 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.003 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug shutdown = true 2024-09-17T21:28:33.003 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug op order = true 2024-09-17T21:28:33.003 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug verify stray on activate = true 2024-09-17T21:28:33.003 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug trim objects = true 2024-09-17T21:28:33.003 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.003 INFO:teuthology.orchestra.run.smithi129.stdout: osd open classes on start = true 2024-09-17T21:28:33.003 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug pg log writeout = true 2024-09-17T21:28:33.003 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.003 INFO:teuthology.orchestra.run.smithi129.stdout: osd deep scrub update digest min age = 30 2024-09-17T21:28:33.003 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.003 INFO:teuthology.orchestra.run.smithi129.stdout: osd map max advance = 10 2024-09-17T21:28:33.003 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.004 INFO:teuthology.orchestra.run.smithi129.stdout: journal zero on create = true 2024-09-17T21:28:33.004 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.004 INFO:teuthology.orchestra.run.smithi129.stdout: filestore ondisk finisher threads = 3 2024-09-17T21:28:33.004 INFO:teuthology.orchestra.run.smithi129.stdout: filestore apply finisher threads = 3 2024-09-17T21:28:33.004 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.004 INFO:teuthology.orchestra.run.smithi129.stdout: bdev debug aio = true 2024-09-17T21:28:33.004 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug misdirected ops = true 2024-09-17T21:28:33.004 INFO:teuthology.orchestra.run.smithi129.stdout: bluestore block size = 96636764160 2024-09-17T21:28:33.004 INFO:teuthology.orchestra.run.smithi129.stdout: bluestore compression algorithm = zstd 2024-09-17T21:28:33.004 INFO:teuthology.orchestra.run.smithi129.stdout: bluestore compression mode = aggressive 2024-09-17T21:28:33.004 INFO:teuthology.orchestra.run.smithi129.stdout: bluestore fsck on mount = True 2024-09-17T21:28:33.004 INFO:teuthology.orchestra.run.smithi129.stdout: debug bluefs = 1/20 2024-09-17T21:28:33.004 INFO:teuthology.orchestra.run.smithi129.stdout: debug bluestore = 1/20 2024-09-17T21:28:33.005 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-09-17T21:28:33.005 INFO:teuthology.orchestra.run.smithi129.stdout: debug osd = 20 2024-09-17T21:28:33.005 INFO:teuthology.orchestra.run.smithi129.stdout: debug rocksdb = 4/10 2024-09-17T21:28:33.005 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T21:28:33.005 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd full ratio = 0.9 2024-09-17T21:28:33.005 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd nearfull ratio = 0.8 2024-09-17T21:28:33.005 INFO:teuthology.orchestra.run.smithi129.stdout: osd blocked scrub grace period = 3600 2024-09-17T21:28:33.005 INFO:teuthology.orchestra.run.smithi129.stdout: osd failsafe full ratio = 0.95 2024-09-17T21:28:33.005 INFO:teuthology.orchestra.run.smithi129.stdout: osd max markdown count = 1000 2024-09-17T21:28:33.005 INFO:teuthology.orchestra.run.smithi129.stdout: osd objectstore = bluestore 2024-09-17T21:28:33.005 INFO:teuthology.orchestra.run.smithi129.stdout: osd shutdown pgref assert = True 2024-09-17T21:28:33.005 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.005 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr] 2024-09-17T21:28:33.005 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-09-17T21:28:33.005 INFO:teuthology.orchestra.run.smithi129.stdout: debug mgr = 20 2024-09-17T21:28:33.005 INFO:teuthology.orchestra.run.smithi129.stdout: debug mon = 20 2024-09-17T21:28:33.006 INFO:teuthology.orchestra.run.smithi129.stdout: debug auth = 20 2024-09-17T21:28:33.006 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min pgs per osd = 4 2024-09-17T21:28:33.006 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min bytes per osd = 10 2024-09-17T21:28:33.006 INFO:teuthology.orchestra.run.smithi129.stdout: mgr/telemetry/nag = false 2024-09-17T21:28:33.006 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.006 INFO:teuthology.orchestra.run.smithi129.stdout:[mon] 2024-09-17T21:28:33.006 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-09-17T21:28:33.006 INFO:teuthology.orchestra.run.smithi129.stdout: debug mon = 20 2024-09-17T21:28:33.006 INFO:teuthology.orchestra.run.smithi129.stdout: debug paxos = 20 2024-09-17T21:28:33.006 INFO:teuthology.orchestra.run.smithi129.stdout: debug auth = 20 2024-09-17T21:28:33.006 INFO:teuthology.orchestra.run.smithi129.stdout: mon data avail warn = 5 2024-09-17T21:28:33.006 INFO:teuthology.orchestra.run.smithi129.stdout: mon mgr mkfs grace = 240 2024-09-17T21:28:33.006 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min pgs per osd = 4 2024-09-17T21:28:33.006 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd reporter subtree level = osd 2024-09-17T21:28:33.006 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd prime pg temp = true 2024-09-17T21:28:33.006 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min bytes per osd = 10 2024-09-17T21:28:33.007 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.007 INFO:teuthology.orchestra.run.smithi129.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T21:28:33.007 INFO:teuthology.orchestra.run.smithi129.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T21:28:33.007 INFO:teuthology.orchestra.run.smithi129.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T21:28:33.007 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.007 INFO:teuthology.orchestra.run.smithi129.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T21:28:33.007 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T21:28:33.007 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T21:28:33.007 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.007 INFO:teuthology.orchestra.run.smithi129.stdout: # 1m isn't quite enough 2024-09-17T21:28:33.007 INFO:teuthology.orchestra.run.smithi129.stdout: mon_down_mkfs_grace = 2m 2024-09-17T21:28:33.007 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.007 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_filestore_osds = false 2024-09-17T21:28:33.007 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:28:33.007 INFO:teuthology.orchestra.run.smithi129.stdout:[client] 2024-09-17T21:28:33.007 INFO:teuthology.orchestra.run.smithi129.stdout: rgw cache enabled = true 2024-09-17T21:28:33.008 INFO:teuthology.orchestra.run.smithi129.stdout: rgw enable ops log = true 2024-09-17T21:28:33.008 INFO:teuthology.orchestra.run.smithi129.stdout: rgw enable usage log = true 2024-09-17T21:28:33.008 INFO:teuthology.orchestra.run.smithi129.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T21:28:33.008 INFO:teuthology.orchestra.run.smithi129.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T21:28:33.008 INFO:teuthology.orchestra.run.smithi129.stdout: rbd cache = True 2024-09-17T21:28:33.008 INFO:teuthology.orchestra.run.smithi129.stdout: rbd cache policy = writearound 2024-09-17T21:28:33.008 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.a] 2024-09-17T21:28:33.008 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.c] 2024-09-17T21:28:33.008 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.b] 2024-09-17T21:28:33.008 INFO:teuthology.orchestra.run.smithi171.stdout:[global] 2024-09-17T21:28:33.009 INFO:teuthology.orchestra.run.smithi171.stdout: chdir = "" 2024-09-17T21:28:33.009 INFO:teuthology.orchestra.run.smithi171.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T21:28:33.009 INFO:teuthology.orchestra.run.smithi171.stdout: auth supported = cephx 2024-09-17T21:28:33.009 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.009 INFO:teuthology.orchestra.run.smithi171.stdout: filestore xattr use omap = true 2024-09-17T21:28:33.009 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.009 INFO:teuthology.orchestra.run.smithi171.stdout: mon clock drift allowed = 1.000 2024-09-17T21:28:33.009 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.009 INFO:teuthology.orchestra.run.smithi171.stdout: osd crush chooseleaf type = 0 2024-09-17T21:28:33.009 INFO:teuthology.orchestra.run.smithi171.stdout: auth debug = true 2024-09-17T21:28:33.009 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.009 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on old message = true 2024-09-17T21:28:33.009 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on bug = true 2024-09-17T21:28:33.010 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.010 INFO:teuthology.orchestra.run.smithi171.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T21:28:33.010 INFO:teuthology.orchestra.run.smithi171.stdout: mon pg warn max object skew = 0 2024-09-17T21:28:33.010 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.010 INFO:teuthology.orchestra.run.smithi171.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T21:28:33.010 INFO:teuthology.orchestra.run.smithi171.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T21:28:33.010 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.010 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default size = 2 2024-09-17T21:28:33.010 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.010 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow primary affinity = true 2024-09-17T21:28:33.010 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow pg remap = true 2024-09-17T21:28:33.010 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on legacy crush tunables = false 2024-09-17T21:28:33.010 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on crush straw calc version zero = false 2024-09-17T21:28:33.010 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on no sortbitwise = false 2024-09-17T21:28:33.011 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on osd down out interval zero = false 2024-09-17T21:28:33.011 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on too few osds = false 2024-09-17T21:28:33.011 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T21:28:33.011 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T21:28:33.011 INFO:teuthology.orchestra.run.smithi171.stdout: mon_allow_pool_size_one = true 2024-09-17T21:28:33.011 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.011 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T21:28:33.011 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.011 INFO:teuthology.orchestra.run.smithi171.stdout: osd default data pool replay window = 5 2024-09-17T21:28:33.012 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.012 INFO:teuthology.orchestra.run.smithi171.stdout: mon allow pool delete = true 2024-09-17T21:28:33.012 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.012 INFO:teuthology.orchestra.run.smithi171.stdout: mon cluster log file level = debug 2024-09-17T21:28:33.012 INFO:teuthology.orchestra.run.smithi171.stdout: debug asserts on shutdown = true 2024-09-17T21:28:33.012 INFO:teuthology.orchestra.run.smithi171.stdout: mon health detail to clog = false 2024-09-17T21:28:33.012 INFO:teuthology.orchestra.run.smithi171.stdout: mon host = "172.21.15.129,[v2:172.21.15.129:3301,v1:172.21.15.129:6790],172.21.15.171" 2024-09-17T21:28:33.012 INFO:teuthology.orchestra.run.smithi171.stdout: mon client directed command retry = 5 2024-09-17T21:28:33.012 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on pool no app = False 2024-09-17T21:28:33.012 INFO:teuthology.orchestra.run.smithi171.stdout: ms inject socket failures = 5000 2024-09-17T21:28:33.012 INFO:teuthology.orchestra.run.smithi171.stdout: fsid = f6d3a80c-8a1f-4716-b25d-e97b28890c0d 2024-09-17T21:28:33.012 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.012 INFO:teuthology.orchestra.run.smithi171.stdout:[osd] 2024-09-17T21:28:33.013 INFO:teuthology.orchestra.run.smithi171.stdout: osd journal size = 100 2024-09-17T21:28:33.013 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.013 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub load threshold = 5.0 2024-09-17T21:28:33.013 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub max interval = 600 2024-09-17T21:28:33.013 INFO:teuthology.orchestra.run.smithi171.stdout: osd mclock profile = high_recovery_ops 2024-09-17T21:28:33.013 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.013 INFO:teuthology.orchestra.run.smithi171.stdout: osd recover clone overlap = true 2024-09-17T21:28:33.013 INFO:teuthology.orchestra.run.smithi171.stdout: osd recovery max chunk = 1048576 2024-09-17T21:28:33.013 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.013 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug shutdown = true 2024-09-17T21:28:33.013 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug op order = true 2024-09-17T21:28:33.013 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug verify stray on activate = true 2024-09-17T21:28:33.013 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug trim objects = true 2024-09-17T21:28:33.013 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.014 INFO:teuthology.orchestra.run.smithi171.stdout: osd open classes on start = true 2024-09-17T21:28:33.014 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug pg log writeout = true 2024-09-17T21:28:33.014 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.014 INFO:teuthology.orchestra.run.smithi171.stdout: osd deep scrub update digest min age = 30 2024-09-17T21:28:33.014 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.014 INFO:teuthology.orchestra.run.smithi171.stdout: osd map max advance = 10 2024-09-17T21:28:33.014 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.014 INFO:teuthology.orchestra.run.smithi171.stdout: journal zero on create = true 2024-09-17T21:28:33.014 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.014 INFO:teuthology.orchestra.run.smithi171.stdout: filestore ondisk finisher threads = 3 2024-09-17T21:28:33.014 INFO:teuthology.orchestra.run.smithi171.stdout: filestore apply finisher threads = 3 2024-09-17T21:28:33.014 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.014 INFO:teuthology.orchestra.run.smithi171.stdout: bdev debug aio = true 2024-09-17T21:28:33.014 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug misdirected ops = true 2024-09-17T21:28:33.015 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore block size = 96636764160 2024-09-17T21:28:33.015 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore compression algorithm = zstd 2024-09-17T21:28:33.015 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore compression mode = aggressive 2024-09-17T21:28:33.015 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore fsck on mount = True 2024-09-17T21:28:33.015 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluefs = 1/20 2024-09-17T21:28:33.015 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluestore = 1/20 2024-09-17T21:28:33.015 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-17T21:28:33.015 INFO:teuthology.orchestra.run.smithi171.stdout: debug osd = 20 2024-09-17T21:28:33.015 INFO:teuthology.orchestra.run.smithi171.stdout: debug rocksdb = 4/10 2024-09-17T21:28:33.015 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T21:28:33.015 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd full ratio = 0.9 2024-09-17T21:28:33.015 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd nearfull ratio = 0.8 2024-09-17T21:28:33.015 INFO:teuthology.orchestra.run.smithi171.stdout: osd blocked scrub grace period = 3600 2024-09-17T21:28:33.016 INFO:teuthology.orchestra.run.smithi171.stdout: osd failsafe full ratio = 0.95 2024-09-17T21:28:33.016 INFO:teuthology.orchestra.run.smithi171.stdout: osd max markdown count = 1000 2024-09-17T21:28:33.016 INFO:teuthology.orchestra.run.smithi171.stdout: osd objectstore = bluestore 2024-09-17T21:28:33.016 INFO:teuthology.orchestra.run.smithi171.stdout: osd shutdown pgref assert = True 2024-09-17T21:28:33.016 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.016 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr] 2024-09-17T21:28:33.016 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-17T21:28:33.016 INFO:teuthology.orchestra.run.smithi171.stdout: debug mgr = 20 2024-09-17T21:28:33.016 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-09-17T21:28:33.016 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-09-17T21:28:33.016 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-09-17T21:28:33.016 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-09-17T21:28:33.016 INFO:teuthology.orchestra.run.smithi171.stdout: mgr/telemetry/nag = false 2024-09-17T21:28:33.016 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.016 INFO:teuthology.orchestra.run.smithi171.stdout:[mon] 2024-09-17T21:28:33.017 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-17T21:28:33.017 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-09-17T21:28:33.017 INFO:teuthology.orchestra.run.smithi171.stdout: debug paxos = 20 2024-09-17T21:28:33.017 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-09-17T21:28:33.017 INFO:teuthology.orchestra.run.smithi171.stdout: mon data avail warn = 5 2024-09-17T21:28:33.017 INFO:teuthology.orchestra.run.smithi171.stdout: mon mgr mkfs grace = 240 2024-09-17T21:28:33.017 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-09-17T21:28:33.017 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd reporter subtree level = osd 2024-09-17T21:28:33.017 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd prime pg temp = true 2024-09-17T21:28:33.017 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-09-17T21:28:33.017 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.017 INFO:teuthology.orchestra.run.smithi171.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T21:28:33.017 INFO:teuthology.orchestra.run.smithi171.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T21:28:33.017 INFO:teuthology.orchestra.run.smithi171.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T21:28:33.018 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.018 INFO:teuthology.orchestra.run.smithi171.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T21:28:33.018 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T21:28:33.018 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T21:28:33.018 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.018 INFO:teuthology.orchestra.run.smithi171.stdout: # 1m isn't quite enough 2024-09-17T21:28:33.018 INFO:teuthology.orchestra.run.smithi171.stdout: mon_down_mkfs_grace = 2m 2024-09-17T21:28:33.018 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.018 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_filestore_osds = false 2024-09-17T21:28:33.018 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T21:28:33.018 INFO:teuthology.orchestra.run.smithi171.stdout:[client] 2024-09-17T21:28:33.018 INFO:teuthology.orchestra.run.smithi171.stdout: rgw cache enabled = true 2024-09-17T21:28:33.018 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable ops log = true 2024-09-17T21:28:33.018 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable usage log = true 2024-09-17T21:28:33.018 INFO:teuthology.orchestra.run.smithi171.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T21:28:33.019 INFO:teuthology.orchestra.run.smithi171.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T21:28:33.019 INFO:teuthology.orchestra.run.smithi171.stdout: rbd cache = True 2024-09-17T21:28:33.019 INFO:teuthology.orchestra.run.smithi171.stdout: rbd cache policy = writearound 2024-09-17T21:28:33.019 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.a] 2024-09-17T21:28:33.019 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.c] 2024-09-17T21:28:33.019 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.b] 2024-09-17T21:28:33.020 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-17T21:28:33.020 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-17T21:28:33.115 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-17T21:28:33.116 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:28:33.116 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-17T21:28:33.130 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:28:33.130 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-17T21:28:33.185 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T21:28:33.186 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:28:33.186 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T21:28:33.186 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T21:28:33.259 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:28:33.260 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:28:33.315 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T21:28:33.315 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:28:33.315 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T21:28:33.315 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T21:28:33.350 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:28:33.350 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:28:33.405 INFO:tasks.ceph:Setting up mon nodes... 2024-09-17T21:28:33.406 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-17T21:28:33.406 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T21:28:33.456 INFO:teuthology.orchestra.run.smithi129.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T21:28:33.459 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T21:28:33.516 INFO:teuthology.orchestra.run.smithi171.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T21:28:33.519 INFO:tasks.ceph:Setting up mds nodes... 2024-09-17T21:28:33.519 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-17T21:28:33.519 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T21:28:33.562 INFO:teuthology.orchestra.run.smithi129.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-17T21:28:33.575 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-17T21:28:33.616 INFO:teuthology.orchestra.run.smithi171.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-17T21:28:33.629 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-17T21:28:33.629 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi129.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi171.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T21:28:33.629 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-17T21:28:33.653 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T21:28:33.653 INFO:tasks.ceph:role: osd.0 2024-09-17T21:28:33.654 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T21:28:33.654 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T21:28:33.725 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:28:33.725 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:28:33.725 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:28:33.725 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T21:28:33.726 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:28:33.726 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:28:33.726 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:28:33.726 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T21:28:33.726 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:28:33.726 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:28:34.232 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-09-17T21:28:34.234 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-09-17T21:28:34.234 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-17T21:28:34.338 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-17T21:28:34.369 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-17T21:28:34.435 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T21:28:34.435 INFO:tasks.ceph:role: osd.1 2024-09-17T21:28:34.435 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T21:28:34.436 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T21:28:34.508 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:28:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:28:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:28:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T21:28:34.509 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:28:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:28:34.509 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:28:34.509 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T21:28:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:28:34.509 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:28:34.974 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-09-17T21:28:34.976 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-09-17T21:28:34.976 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-17T21:28:35.068 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-17T21:28:35.097 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-17T21:28:35.162 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T21:28:35.162 INFO:tasks.ceph:role: osd.2 2024-09-17T21:28:35.162 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T21:28:35.162 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T21:28:35.235 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:28:35.235 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:28:35.235 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:28:35.235 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T21:28:35.235 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:28:35.235 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:28:35.236 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:28:35.236 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T21:28:35.236 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:28:35.236 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:28:35.715 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-09-17T21:28:35.717 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-09-17T21:28:35.718 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-17T21:28:35.871 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-17T21:28:35.902 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-17T21:28:35.967 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T21:28:35.967 INFO:tasks.ceph:role: osd.3 2024-09-17T21:28:35.967 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T21:28:35.967 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T21:28:36.039 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:28:36.039 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:28:36.039 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:28:36.039 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T21:28:36.039 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:28:36.039 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:28:36.040 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:28:36.040 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T21:28:36.040 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:28:36.040 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:28:36.547 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-09-17T21:28:36.549 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-09-17T21:28:36.550 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-17T21:28:36.702 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-17T21:28:36.738 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:28:36.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T21:28:36.832+0000 7f37e3c13640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-17T21:28:36.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T21:28:36.833+0000 7f37e3c13640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-17T21:28:36.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T21:28:36.833+0000 7f37e3c13640 -1 bdev(0x558c5b479800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-17T21:28:36.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T21:28:36.833+0000 7f37e3c13640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-17T21:28:40.237 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-17T21:28:40.267 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:28:40.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T21:28:40.354+0000 7f14d5d45640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-17T21:28:40.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T21:28:40.355+0000 7f14d5d45640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-17T21:28:40.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T21:28:40.355+0000 7f14d5d45640 -1 bdev(0x5557d734b800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-17T21:28:40.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T21:28:40.355+0000 7f14d5d45640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-17T21:28:43.765 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-17T21:28:43.794 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:28:43.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T21:28:43.882+0000 7f700d00b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-17T21:28:43.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T21:28:43.883+0000 7f700d00b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-17T21:28:43.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T21:28:43.883+0000 7f700d00b640 -1 bdev(0x5580cc8fb800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-17T21:28:43.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T21:28:43.883+0000 7f700d00b640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-17T21:28:46.793 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T21:28:46.820 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:28:46.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T21:28:46.906+0000 7f645fa16640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-17T21:28:46.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T21:28:46.906+0000 7f645fa16640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-17T21:28:46.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T21:28:46.906+0000 7f645fa16640 -1 bdev(0x55f22a6cf800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-17T21:28:46.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T21:28:46.906+0000 7f645fa16640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-17T21:28:49.818 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-17T21:28:49.848 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-17T21:28:49.878 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T21:28:49.878 INFO:tasks.ceph:role: osd.4 2024-09-17T21:28:49.879 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T21:28:49.879 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T21:28:49.955 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:28:49.955 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:28:49.955 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:28:49.955 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T21:28:49.955 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:28:49.955 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:28:49.955 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:28:49.956 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T21:28:49.956 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:28:49.956 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:28:50.483 INFO:teuthology.orchestra.run.smithi171.stdout:Discarding blocks...Done. 2024-09-17T21:28:50.485 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-09-17T21:28:50.485 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-17T21:28:50.605 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-17T21:28:50.640 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-17T21:28:50.707 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T21:28:50.707 INFO:tasks.ceph:role: osd.5 2024-09-17T21:28:50.707 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T21:28:50.707 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T21:28:50.780 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:28:50.780 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:28:50.781 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:28:50.781 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T21:28:50.781 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:28:50.781 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:28:50.781 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:28:50.781 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T21:28:50.781 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:28:50.781 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:28:51.237 INFO:teuthology.orchestra.run.smithi171.stdout:Discarding blocks...Done. 2024-09-17T21:28:51.239 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-09-17T21:28:51.239 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-17T21:28:51.334 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-17T21:28:51.363 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-17T21:28:51.428 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T21:28:51.428 INFO:tasks.ceph:role: osd.6 2024-09-17T21:28:51.428 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T21:28:51.428 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T21:28:51.500 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:28:51.500 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:28:51.500 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:28:51.500 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T21:28:51.501 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:28:51.501 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:28:51.501 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:28:51.501 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T21:28:51.501 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:28:51.501 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:28:51.977 INFO:teuthology.orchestra.run.smithi171.stdout:Discarding blocks...Done. 2024-09-17T21:28:51.979 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-09-17T21:28:51.979 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-17T21:28:52.114 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-17T21:28:52.143 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-17T21:28:52.208 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T21:28:52.209 INFO:tasks.ceph:role: osd.7 2024-09-17T21:28:52.209 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T21:28:52.209 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T21:28:52.283 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:28:52.283 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:28:52.283 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:28:52.283 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T21:28:52.283 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:28:52.283 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:28:52.283 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:28:52.283 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T21:28:52.284 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:28:52.284 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:28:52.770 INFO:teuthology.orchestra.run.smithi171.stdout:Discarding blocks...Done. 2024-09-17T21:28:52.772 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-09-17T21:28:52.772 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-17T21:28:52.917 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-17T21:28:52.953 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:28:53.054 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T21:28:53.051+0000 7fe6b7dfd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-17T21:28:53.054 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T21:28:53.052+0000 7fe6b7dfd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-17T21:28:53.054 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T21:28:53.052+0000 7fe6b7dfd640 -1 bdev(0x5647ec3c5800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-17T21:28:53.054 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T21:28:53.052+0000 7fe6b7dfd640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-17T21:28:55.973 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-17T21:28:56.003 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:28:56.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T21:28:56.092+0000 7fe52f018640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-17T21:28:56.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T21:28:56.092+0000 7fe52f018640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-17T21:28:56.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T21:28:56.092+0000 7fe52f018640 -1 bdev(0x55bab306d800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-17T21:28:56.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T21:28:56.093+0000 7fe52f018640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-17T21:28:59.262 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-17T21:28:59.292 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:28:59.383 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T21:28:59.380+0000 7fbf12433640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-17T21:28:59.383 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T21:28:59.381+0000 7fbf12433640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-17T21:28:59.383 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T21:28:59.381+0000 7fbf12433640 -1 bdev(0x557b64e7d800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-17T21:28:59.383 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T21:28:59.381+0000 7fbf12433640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-17T21:29:02.319 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-17T21:29:02.344 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:29:02.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T21:29:02.428+0000 7f002ea0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-17T21:29:02.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T21:29:02.428+0000 7f002ea0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-17T21:29:02.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T21:29:02.428+0000 7f002ea0f640 -1 bdev(0x55b25958d800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-17T21:29:02.431 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T21:29:02.428+0000 7f002ea0f640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-17T21:29:05.335 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-17T21:29:05.361 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-17T21:29:05.361 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:29:05.361 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-17T21:29:05.392 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:29:05.392 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-17T21:29:05.462 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:29:05.463 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-17T21:29:05.531 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:29:05.531 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-17T21:29:05.598 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:29:05.598 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-17T21:29:05.665 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:29:05.665 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-17T21:29:05.695 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:29:05.695 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-17T21:29:05.763 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:29:05.763 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-17T21:29:05.831 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:29:05.831 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-17T21:29:05.898 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:29:05.898 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-17T21:29:05.964 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:29:05.964 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-17T21:29:05.983 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:29:05.983 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-17T21:29:06.020 INFO:tasks.ceph:Adding keys to all mons... 2024-09-17T21:29:06.021 DEBUG:teuthology.orchestra.run.smithi129:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T21:29:06.026 DEBUG:teuthology.orchestra.run.smithi171:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T21:29:06.064 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr.y] 2024-09-17T21:29:06.064 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCB9OlmsTQoGxAATImQfhfj5GO5t4/ALP7y3A== 2024-09-17T21:29:06.064 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.0] 2024-09-17T21:29:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCE9Olm0T63MRAAYYoL8o5OE3LH8byqOC+nSg== 2024-09-17T21:29:06.065 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.1] 2024-09-17T21:29:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCI9OlmeXA3FRAAbPdMRn68RJWomCfojgU9vA== 2024-09-17T21:29:06.065 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.2] 2024-09-17T21:29:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCL9OlmO1KxNBAAYkmH5U0fan0T8Zk+uG5KaQ== 2024-09-17T21:29:06.065 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.3] 2024-09-17T21:29:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCO9OlmBccUNhAAXi4/KNS4E2gt3npFIH8T9w== 2024-09-17T21:29:06.065 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr.x] 2024-09-17T21:29:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCB9OlmEi+6HhAAYyrRrXe8e9YQXV2fIXhxGw== 2024-09-17T21:29:06.065 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.4] 2024-09-17T21:29:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCV9OlmNQgkAxAAVEPgADkzK/2c2GJIoty3Zw== 2024-09-17T21:29:06.065 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.5] 2024-09-17T21:29:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCY9OlmDY+QBRAAZqkAmZ8HjmCJETMlgiO8PA== 2024-09-17T21:29:06.066 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.6] 2024-09-17T21:29:06.066 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCb9OlmK6TAFhAASIQBd6EIjPD1Z2EPxmLGlg== 2024-09-17T21:29:06.066 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.7] 2024-09-17T21:29:06.066 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCe9OlmLMmTGRAAntJpZhXnkpefjGV5TiUJlQ== 2024-09-17T21:29:06.066 INFO:teuthology.orchestra.run.smithi129.stdout:[client.0] 2024-09-17T21:29:06.066 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCB9OlmXwZtIRAAgptCcjHPQPFsM85JOfQTAg== 2024-09-17T21:29:06.066 INFO:teuthology.orchestra.run.smithi129.stdout:[client.1] 2024-09-17T21:29:06.066 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCB9OlmMa2oJBAAz+Fby8gg+c0oCuN24AleDg== 2024-09-17T21:29:06.085 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr.y] 2024-09-17T21:29:06.085 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCB9OlmsTQoGxAATImQfhfj5GO5t4/ALP7y3A== 2024-09-17T21:29:06.085 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.0] 2024-09-17T21:29:06.085 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCE9Olm0T63MRAAYYoL8o5OE3LH8byqOC+nSg== 2024-09-17T21:29:06.085 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.1] 2024-09-17T21:29:06.085 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCI9OlmeXA3FRAAbPdMRn68RJWomCfojgU9vA== 2024-09-17T21:29:06.086 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.2] 2024-09-17T21:29:06.086 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCL9OlmO1KxNBAAYkmH5U0fan0T8Zk+uG5KaQ== 2024-09-17T21:29:06.086 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.3] 2024-09-17T21:29:06.086 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCO9OlmBccUNhAAXi4/KNS4E2gt3npFIH8T9w== 2024-09-17T21:29:06.086 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr.x] 2024-09-17T21:29:06.086 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCB9OlmEi+6HhAAYyrRrXe8e9YQXV2fIXhxGw== 2024-09-17T21:29:06.086 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.4] 2024-09-17T21:29:06.086 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCV9OlmNQgkAxAAVEPgADkzK/2c2GJIoty3Zw== 2024-09-17T21:29:06.086 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.5] 2024-09-17T21:29:06.086 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCY9OlmDY+QBRAAZqkAmZ8HjmCJETMlgiO8PA== 2024-09-17T21:29:06.086 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.6] 2024-09-17T21:29:06.086 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCb9OlmK6TAFhAASIQBd6EIjPD1Z2EPxmLGlg== 2024-09-17T21:29:06.086 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.7] 2024-09-17T21:29:06.086 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCe9OlmLMmTGRAAntJpZhXnkpefjGV5TiUJlQ== 2024-09-17T21:29:06.087 INFO:teuthology.orchestra.run.smithi171.stdout:[client.0] 2024-09-17T21:29:06.087 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCB9OlmXwZtIRAAgptCcjHPQPFsM85JOfQTAg== 2024-09-17T21:29:06.087 INFO:teuthology.orchestra.run.smithi171.stdout:[client.1] 2024-09-17T21:29:06.087 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCB9OlmMa2oJBAAz+Fby8gg+c0oCuN24AleDg== 2024-09-17T21:29:06.087 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T21:29:06.112 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T21:29:06.180 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:29:06.182 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:29:06.233 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:29:06.235 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:29:06.286 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:29:06.288 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:29:06.339 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:29:06.341 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:29:06.392 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T21:29:06.395 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T21:29:06.446 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:29:06.448 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:29:06.499 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:29:06.501 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:29:06.552 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:29:06.554 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:29:06.606 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:29:06.608 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:29:06.660 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T21:29:06.662 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T21:29:06.717 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T21:29:06.719 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T21:29:06.773 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-17T21:29:06.773 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-17T21:29:06.798 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T21:29:06.893 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-17T21:29:06.918 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-17T21:29:06.981 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T21:29:07.077 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-17T21:29:07.101 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-17T21:29:07.127 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T21:29:07.225 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-17T21:29:07.249 DEBUG:teuthology.orchestra.run.smithi129:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:29:07.251 DEBUG:teuthology.orchestra.run.smithi171:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:29:07.305 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-17T21:29:07.305 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T21:29:07.305 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T21:29:07.307 INFO:tasks.ceph.mon.a:Started 2024-09-17T21:29:07.307 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T21:29:07.308 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T21:29:07.309 INFO:tasks.ceph.mon.c:Started 2024-09-17T21:29:07.310 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T21:29:07.310 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T21:29:07.347 INFO:tasks.ceph.mon.b:Started 2024-09-17T21:29:07.347 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-17T21:29:07.348 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-17T21:29:07.348 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-17T21:29:07.350 INFO:tasks.ceph.mgr.y:Started 2024-09-17T21:29:07.350 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-17T21:29:07.350 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-17T21:29:07.352 INFO:tasks.ceph.mgr.x:Started 2024-09-17T21:29:07.353 DEBUG:tasks.ceph:set 0 configs 2024-09-17T21:29:07.353 INFO:tasks.ceph:Setting crush tunables to default 2024-09-17T21:29:07.353 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd crush tunables default 2024-09-17T21:29:12.705 INFO:teuthology.orchestra.run.smithi129.stderr:adjusted tunables profile to default 2024-09-17T21:29:12.957 INFO:tasks.ceph:check_enable_crimson: False 2024-09-17T21:29:12.958 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-17T21:29:12.958 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:29:12.958 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-17T21:29:12.983 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:29:12.984 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-17T21:29:13.049 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:29:13.049 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-17T21:29:13.114 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T21:29:13.114 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-17T21:29:13.179 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:29:13.179 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-17T21:29:13.204 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:29:13.204 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-17T21:29:13.269 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:29:13.269 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-17T21:29:13.333 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T21:29:13.333 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-17T21:29:13.399 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new ad067b51-9402-4f69-a0f0-08fbf068dcfc 0 2024-09-17T21:29:13.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T21:29:13.737+0000 7f42837fe640 0 --2- 172.21.15.171:0/2186678972 >> v2:172.21.15.171:3300/0 conn(0x7f4284071b80 0x7f4284071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-17T21:29:13.825 INFO:teuthology.orchestra.run.smithi171.stdout:0 2024-09-17T21:29:13.837 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new 30385fc0-ef5f-45f5-a3ec-37e890d4826e 1 2024-09-17T21:29:14.021 INFO:teuthology.orchestra.run.smithi171.stdout:1 2024-09-17T21:29:14.033 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new 76be9b2b-b63f-402a-9ab5-2564bd671ac3 2 2024-09-17T21:29:14.218 INFO:teuthology.orchestra.run.smithi171.stdout:2 2024-09-17T21:29:14.230 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new d6c34e3b-1ac6-415c-9c56-38ae77a5c9b0 3 2024-09-17T21:29:14.414 INFO:teuthology.orchestra.run.smithi171.stdout:3 2024-09-17T21:29:14.425 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new 7d722e8c-c497-4f0c-9f61-d9826f8c93d8 4 2024-09-17T21:29:14.608 INFO:teuthology.orchestra.run.smithi171.stdout:4 2024-09-17T21:29:14.619 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new 954f801b-3ac9-410a-a375-eb1c533744de 5 2024-09-17T21:29:14.799 INFO:teuthology.orchestra.run.smithi171.stdout:5 2024-09-17T21:29:14.811 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new 21837515-65a2-4f21-96d6-67cfc4589358 6 2024-09-17T21:29:14.991 INFO:teuthology.orchestra.run.smithi171.stdout:6 2024-09-17T21:29:15.002 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new 67af7eea-8803-425d-ae50-30a6ab91a711 7 2024-09-17T21:29:15.186 INFO:teuthology.orchestra.run.smithi171.stdout:7 2024-09-17T21:29:15.197 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T21:29:15.197 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T21:29:15.199 INFO:tasks.ceph.osd.0:Started 2024-09-17T21:29:15.200 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T21:29:15.200 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T21:29:15.202 INFO:tasks.ceph.osd.1:Started 2024-09-17T21:29:15.202 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T21:29:15.202 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T21:29:15.204 INFO:tasks.ceph.osd.2:Started 2024-09-17T21:29:15.204 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T21:29:15.204 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T21:29:15.207 INFO:tasks.ceph.osd.3:Started 2024-09-17T21:29:15.207 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-17T21:29:15.207 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-17T21:29:15.209 INFO:tasks.ceph.osd.4:Started 2024-09-17T21:29:15.209 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T21:29:15.209 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T21:29:15.211 INFO:tasks.ceph.osd.5:Started 2024-09-17T21:29:15.212 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-17T21:29:15.212 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-17T21:29:15.214 INFO:tasks.ceph.osd.6:Started 2024-09-17T21:29:15.214 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-17T21:29:15.214 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-17T21:29:15.216 INFO:tasks.ceph.osd.7:Started 2024-09-17T21:29:15.216 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T21:29:15.443 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:29:15.443 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":10,"fsid":"f6d3a80c-8a1f-4716-b25d-e97b28890c0d","created":"2024-09-17T21:29:12.606637+0000","modified":"2024-09-17T21:29:15.180249+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T21:29:15.180249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ad067b51-9402-4f69-a0f0-08fbf068dcfc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"30385fc0-ef5f-45f5-a3ec-37e890d4826e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"76be9b2b-b63f-402a-9ab5-2564bd671ac3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d6c34e3b-1ac6-415c-9c56-38ae77a5c9b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7d722e8c-c497-4f0c-9f61-d9826f8c93d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"954f801b-3ac9-410a-a375-eb1c533744de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"21837515-65a2-4f21-96d6-67cfc4589358","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"67af7eea-8803-425d-ae50-30a6ab91a711","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:29:15.458 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-17T21:29:15.459 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-17T21:29:16.263 DEBUG:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T21:29:16.423 INFO:teuthology.misc.health.smithi129.stdout: 2024-09-17T21:29:16.423 INFO:teuthology.misc.health.smithi129.stdout:{"epoch":10,"fsid":"f6d3a80c-8a1f-4716-b25d-e97b28890c0d","created":"2024-09-17T21:29:12.606637+0000","modified":"2024-09-17T21:29:15.180249+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T21:29:15.180249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ad067b51-9402-4f69-a0f0-08fbf068dcfc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"30385fc0-ef5f-45f5-a3ec-37e890d4826e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"76be9b2b-b63f-402a-9ab5-2564bd671ac3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d6c34e3b-1ac6-415c-9c56-38ae77a5c9b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7d722e8c-c497-4f0c-9f61-d9826f8c93d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"954f801b-3ac9-410a-a375-eb1c533744de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"21837515-65a2-4f21-96d6-67cfc4589358","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"67af7eea-8803-425d-ae50-30a6ab91a711","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:29:16.433 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-17T21:29:16.650 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:16.647+0000 7fd45521d640 -1 Falling back to public interface 2024-09-17T21:29:16.674 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:16.671+0000 7f11cc420640 -1 Falling back to public interface 2024-09-17T21:29:16.738 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:16.736+0000 7f673781a640 -1 Falling back to public interface 2024-09-17T21:29:16.773 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:16.771+0000 7f6003c16640 -1 Falling back to public interface 2024-09-17T21:29:16.917 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:16.914+0000 7fbbc47bd640 -1 Falling back to public interface 2024-09-17T21:29:16.981 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:16.979+0000 7f581ea0d640 -1 Falling back to public interface 2024-09-17T21:29:17.005 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:17.003+0000 7fbc0e687640 -1 Falling back to public interface 2024-09-17T21:29:17.165 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:17.162+0000 7f06308d8640 -1 Falling back to public interface 2024-09-17T21:29:17.866 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-09-17T21:29:17.863+0000 7fbe442c6640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T21:29:17.866 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-09-17T21:29:17.863+0000 7fbe442c6640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-17T21:29:20.449 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:20.447+0000 7fd45521d640 -1 osd.0 0 log_to_monitors true 2024-09-17T21:29:20.984 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:20.982+0000 7f581ea0d640 -1 osd.5 0 log_to_monitors true 2024-09-17T21:29:21.005 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:21.003+0000 7f673781a640 -1 osd.7 0 log_to_monitors true 2024-09-17T21:29:21.008 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:21.005+0000 7f11cc420640 -1 osd.2 0 log_to_monitors true 2024-09-17T21:29:21.059 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:21.056+0000 7f6003c16640 -1 osd.4 0 log_to_monitors true 2024-09-17T21:29:21.066 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:21.063+0000 7fbc0e687640 -1 osd.6 0 log_to_monitors true 2024-09-17T21:29:21.227 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:21.225+0000 7fbbc47bd640 -1 osd.1 0 log_to_monitors true 2024-09-17T21:29:21.385 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:21.383+0000 7f06308d8640 -1 osd.3 0 log_to_monitors true 2024-09-17T21:29:22.593 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:22.590+0000 7fd450c08640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T21:29:22.602 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:22.600+0000 7fd448a27640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:29:23.239 DEBUG:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T21:29:23.670 INFO:teuthology.misc.health.smithi129.stdout: 2024-09-17T21:29:23.670 INFO:teuthology.misc.health.smithi129.stdout:{"epoch":13,"fsid":"f6d3a80c-8a1f-4716-b25d-e97b28890c0d","created":"2024-09-17T21:29:12.606637+0000","modified":"2024-09-17T21:29:22.883451+0000","last_up_change":"2024-09-17T21:29:22.883451+0000","last_in_change":"2024-09-17T21:29:15.180249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ad067b51-9402-4f69-a0f0-08fbf068dcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6801","nonce":3635367421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6803","nonce":3635367421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6807","nonce":3635367421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6805","nonce":3635367421}]},"public_addr":"172.21.15.129:6801/3635367421","cluster_addr":"172.21.15.129:6803/3635367421","heartbeat_back_addr":"172.21.15.129:6807/3635367421","heartbeat_front_addr":"172.21.15.129:6805/3635367421","state":["exists","up"]},{"osd":1,"uuid":"30385fc0-ef5f-45f5-a3ec-37e890d4826e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"76be9b2b-b63f-402a-9ab5-2564bd671ac3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d6c34e3b-1ac6-415c-9c56-38ae77a5c9b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7d722e8c-c497-4f0c-9f61-d9826f8c93d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"954f801b-3ac9-410a-a375-eb1c533744de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"21837515-65a2-4f21-96d6-67cfc4589358","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"67af7eea-8803-425d-ae50-30a6ab91a711","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:29:23.684 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-17T21:29:23.913 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:23.910+0000 7fbbc074c640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T21:29:23.921 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:23.919+0000 7fbbb856b640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:29:23.961 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:23.959+0000 7f062c867640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T21:29:23.973 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:23.970+0000 7f11c7d11640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T21:29:23.976 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:23.974+0000 7f0624686640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:29:23.984 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:23.982+0000 7f67331a1640 -1 osd.7 0 waiting for initial osdmap 2024-09-17T21:29:23.989 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:23.987+0000 7f11bfb30640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:29:23.998 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:23.996+0000 7f672afc0640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:29:24.002 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:24.000+0000 7f581ac5f640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T21:29:24.015 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:24.013+0000 7f581226a640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:29:24.025 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:24.023+0000 7f5fff624640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T21:29:24.038 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:24.036+0000 7f5ff7443640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:29:24.040 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:24.038+0000 7fbc0a616640 -1 osd.6 0 waiting for initial osdmap 2024-09-17T21:29:24.053 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:24.051+0000 7fbc02435640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:29:30.490 DEBUG:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T21:29:30.778 INFO:teuthology.misc.health.smithi129.stdout: 2024-09-17T21:29:30.778 INFO:teuthology.misc.health.smithi129.stdout:{"epoch":19,"fsid":"f6d3a80c-8a1f-4716-b25d-e97b28890c0d","created":"2024-09-17T21:29:12.606637+0000","modified":"2024-09-17T21:29:28.939746+0000","last_up_change":"2024-09-17T21:29:24.913767+0000","last_in_change":"2024-09-17T21:29:15.180249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T21:29:26.876763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ad067b51-9402-4f69-a0f0-08fbf068dcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6801","nonce":3635367421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6803","nonce":3635367421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6807","nonce":3635367421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6805","nonce":3635367421}]},"public_addr":"172.21.15.129:6801/3635367421","cluster_addr":"172.21.15.129:6803/3635367421","heartbeat_back_addr":"172.21.15.129:6807/3635367421","heartbeat_front_addr":"172.21.15.129:6805/3635367421","state":["exists","up"]},{"osd":1,"uuid":"30385fc0-ef5f-45f5-a3ec-37e890d4826e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":176704438},{"type":"v1","addr":"172.21.15.129:6819","nonce":176704438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":176704438},{"type":"v1","addr":"172.21.15.129:6821","nonce":176704438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":176704438},{"type":"v1","addr":"172.21.15.129:6825","nonce":176704438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":176704438},{"type":"v1","addr":"172.21.15.129:6823","nonce":176704438}]},"public_addr":"172.21.15.129:6819/176704438","cluster_addr":"172.21.15.129:6821/176704438","heartbeat_back_addr":"172.21.15.129:6825/176704438","heartbeat_front_addr":"172.21.15.129:6823/176704438","state":["exists","up"]},{"osd":2,"uuid":"76be9b2b-b63f-402a-9ab5-2564bd671ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6809","nonce":1790706405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6811","nonce":1790706405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6815","nonce":1790706405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6813","nonce":1790706405}]},"public_addr":"172.21.15.129:6809/1790706405","cluster_addr":"172.21.15.129:6811/1790706405","heartbeat_back_addr":"172.21.15.129:6815/1790706405","heartbeat_front_addr":"172.21.15.129:6813/1790706405","state":["exists","up"]},{"osd":3,"uuid":"d6c34e3b-1ac6-415c-9c56-38ae77a5c9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6827","nonce":393361453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6829","nonce":393361453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6833","nonce":393361453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6831","nonce":393361453}]},"public_addr":"172.21.15.129:6827/393361453","cluster_addr":"172.21.15.129:6829/393361453","heartbeat_back_addr":"172.21.15.129:6833/393361453","heartbeat_front_addr":"172.21.15.129:6831/393361453","state":["exists","up"]},{"osd":4,"uuid":"7d722e8c-c497-4f0c-9f61-d9826f8c93d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6809","nonce":2337546183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6811","nonce":2337546183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6815","nonce":2337546183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6813","nonce":2337546183}]},"public_addr":"172.21.15.171:6809/2337546183","cluster_addr":"172.21.15.171:6811/2337546183","heartbeat_back_addr":"172.21.15.171:6815/2337546183","heartbeat_front_addr":"172.21.15.171:6813/2337546183","state":["exists","up"]},{"osd":5,"uuid":"954f801b-3ac9-410a-a375-eb1c533744de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6817","nonce":1510320351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6819","nonce":1510320351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6823","nonce":1510320351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6821","nonce":1510320351}]},"public_addr":"172.21.15.171:6817/1510320351","cluster_addr":"172.21.15.171:6819/1510320351","heartbeat_back_addr":"172.21.15.171:6823/1510320351","heartbeat_front_addr":"172.21.15.171:6821/1510320351","state":["exists","up"]},{"osd":6,"uuid":"21837515-65a2-4f21-96d6-67cfc4589358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6825","nonce":471043930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6827","nonce":471043930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6831","nonce":471043930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6829","nonce":471043930}]},"public_addr":"172.21.15.171:6825/471043930","cluster_addr":"172.21.15.171:6827/471043930","heartbeat_back_addr":"172.21.15.171:6831/471043930","heartbeat_front_addr":"172.21.15.171:6829/471043930","state":["exists","up"]},{"osd":7,"uuid":"67af7eea-8803-425d-ae50-30a6ab91a711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6801","nonce":1770619177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6803","nonce":1770619177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6807","nonce":1770619177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6805","nonce":1770619177}]},"public_addr":"172.21.15.171:6801/1770619177","cluster_addr":"172.21.15.171:6803/1770619177","heartbeat_back_addr":"172.21.15.171:6807/1770619177","heartbeat_front_addr":"172.21.15.171:6805/1770619177","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:21.445879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.240967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.039097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.372014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.049543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.009384+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.048598+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.046345+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:29:30.790 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-17T21:29:30.790 INFO:tasks.ceph:Creating RBD pool 2024-09-17T21:29:30.790 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-17T21:29:31.962 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'rbd' created 2024-09-17T21:29:31.976 DEBUG:teuthology.orchestra.run.smithi129:> rbd --cluster ceph pool init rbd 2024-09-17T21:29:34.993 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-17T21:29:34.994 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-17T21:29:34.994 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-17T21:29:35.285 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:29:35.298 INFO:teuthology.orchestra.run.smithi129.stdout:[{"version":9,"timestamp":"2024-09-17T21:29:24.039595+0000","name":"","changes":[{"name":"osd.6/osd_mclock_max_capacity_iops_ssd","new_value":"4589.167920"}]}] 2024-09-17T21:29:35.298 INFO:tasks.ceph_manager:config epoch is 9 2024-09-17T21:29:35.298 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T21:29:35.299 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-17T21:29:35.299 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-17T21:29:35.635 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:29:35.645 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":2122901865},{"type":"v1","addr":"172.21.15.129:6817","nonce":2122901865}]},"active_addr":"172.21.15.129:6817/2122901865","active_change":"2024-09-17T21:29:16.852688+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":4143263225}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":3386296386}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":1823999170}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":130746405}]}]} 2024-09-17T21:29:35.650 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-17T21:29:35.650 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T21:29:35.650 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T21:29:35.933 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:29:35.933 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":23,"fsid":"f6d3a80c-8a1f-4716-b25d-e97b28890c0d","created":"2024-09-17T21:29:12.606637+0000","modified":"2024-09-17T21:29:34.968974+0000","last_up_change":"2024-09-17T21:29:24.913767+0000","last_in_change":"2024-09-17T21:29:15.180249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T21:29:26.876763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T21:29:31.078466+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ad067b51-9402-4f69-a0f0-08fbf068dcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6801","nonce":3635367421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6803","nonce":3635367421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6807","nonce":3635367421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6805","nonce":3635367421}]},"public_addr":"172.21.15.129:6801/3635367421","cluster_addr":"172.21.15.129:6803/3635367421","heartbeat_back_addr":"172.21.15.129:6807/3635367421","heartbeat_front_addr":"172.21.15.129:6805/3635367421","state":["exists","up"]},{"osd":1,"uuid":"30385fc0-ef5f-45f5-a3ec-37e890d4826e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":176704438},{"type":"v1","addr":"172.21.15.129:6819","nonce":176704438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":176704438},{"type":"v1","addr":"172.21.15.129:6821","nonce":176704438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":176704438},{"type":"v1","addr":"172.21.15.129:6825","nonce":176704438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":176704438},{"type":"v1","addr":"172.21.15.129:6823","nonce":176704438}]},"public_addr":"172.21.15.129:6819/176704438","cluster_addr":"172.21.15.129:6821/176704438","heartbeat_back_addr":"172.21.15.129:6825/176704438","heartbeat_front_addr":"172.21.15.129:6823/176704438","state":["exists","up"]},{"osd":2,"uuid":"76be9b2b-b63f-402a-9ab5-2564bd671ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6809","nonce":1790706405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6811","nonce":1790706405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6815","nonce":1790706405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6813","nonce":1790706405}]},"public_addr":"172.21.15.129:6809/1790706405","cluster_addr":"172.21.15.129:6811/1790706405","heartbeat_back_addr":"172.21.15.129:6815/1790706405","heartbeat_front_addr":"172.21.15.129:6813/1790706405","state":["exists","up"]},{"osd":3,"uuid":"d6c34e3b-1ac6-415c-9c56-38ae77a5c9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6827","nonce":393361453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6829","nonce":393361453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6833","nonce":393361453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6831","nonce":393361453}]},"public_addr":"172.21.15.129:6827/393361453","cluster_addr":"172.21.15.129:6829/393361453","heartbeat_back_addr":"172.21.15.129:6833/393361453","heartbeat_front_addr":"172.21.15.129:6831/393361453","state":["exists","up"]},{"osd":4,"uuid":"7d722e8c-c497-4f0c-9f61-d9826f8c93d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6809","nonce":2337546183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6811","nonce":2337546183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6815","nonce":2337546183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6813","nonce":2337546183}]},"public_addr":"172.21.15.171:6809/2337546183","cluster_addr":"172.21.15.171:6811/2337546183","heartbeat_back_addr":"172.21.15.171:6815/2337546183","heartbeat_front_addr":"172.21.15.171:6813/2337546183","state":["exists","up"]},{"osd":5,"uuid":"954f801b-3ac9-410a-a375-eb1c533744de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6817","nonce":1510320351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6819","nonce":1510320351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6823","nonce":1510320351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6821","nonce":1510320351}]},"public_addr":"172.21.15.171:6817/1510320351","cluster_addr":"172.21.15.171:6819/1510320351","heartbeat_back_addr":"172.21.15.171:6823/1510320351","heartbeat_front_addr":"172.21.15.171:6821/1510320351","state":["exists","up"]},{"osd":6,"uuid":"21837515-65a2-4f21-96d6-67cfc4589358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6825","nonce":471043930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6827","nonce":471043930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6831","nonce":471043930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6829","nonce":471043930}]},"public_addr":"172.21.15.171:6825/471043930","cluster_addr":"172.21.15.171:6827/471043930","heartbeat_back_addr":"172.21.15.171:6831/471043930","heartbeat_front_addr":"172.21.15.171:6829/471043930","state":["exists","up"]},{"osd":7,"uuid":"67af7eea-8803-425d-ae50-30a6ab91a711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6801","nonce":1770619177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6803","nonce":1770619177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6807","nonce":1770619177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6805","nonce":1770619177}]},"public_addr":"172.21.15.171:6801/1770619177","cluster_addr":"172.21.15.171:6803/1770619177","heartbeat_back_addr":"172.21.15.171:6807/1770619177","heartbeat_front_addr":"172.21.15.171:6805/1770619177","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:21.445879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.240967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.039097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.372014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.049543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.009384+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.048598+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.046345+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:29:35.946 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T21:29:35.946 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T21:29:36.230 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:29:36.230 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":23,"fsid":"f6d3a80c-8a1f-4716-b25d-e97b28890c0d","created":"2024-09-17T21:29:12.606637+0000","modified":"2024-09-17T21:29:34.968974+0000","last_up_change":"2024-09-17T21:29:24.913767+0000","last_in_change":"2024-09-17T21:29:15.180249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T21:29:26.876763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T21:29:31.078466+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ad067b51-9402-4f69-a0f0-08fbf068dcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6801","nonce":3635367421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6803","nonce":3635367421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6807","nonce":3635367421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6805","nonce":3635367421}]},"public_addr":"172.21.15.129:6801/3635367421","cluster_addr":"172.21.15.129:6803/3635367421","heartbeat_back_addr":"172.21.15.129:6807/3635367421","heartbeat_front_addr":"172.21.15.129:6805/3635367421","state":["exists","up"]},{"osd":1,"uuid":"30385fc0-ef5f-45f5-a3ec-37e890d4826e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":176704438},{"type":"v1","addr":"172.21.15.129:6819","nonce":176704438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":176704438},{"type":"v1","addr":"172.21.15.129:6821","nonce":176704438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":176704438},{"type":"v1","addr":"172.21.15.129:6825","nonce":176704438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":176704438},{"type":"v1","addr":"172.21.15.129:6823","nonce":176704438}]},"public_addr":"172.21.15.129:6819/176704438","cluster_addr":"172.21.15.129:6821/176704438","heartbeat_back_addr":"172.21.15.129:6825/176704438","heartbeat_front_addr":"172.21.15.129:6823/176704438","state":["exists","up"]},{"osd":2,"uuid":"76be9b2b-b63f-402a-9ab5-2564bd671ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6809","nonce":1790706405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6811","nonce":1790706405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6815","nonce":1790706405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6813","nonce":1790706405}]},"public_addr":"172.21.15.129:6809/1790706405","cluster_addr":"172.21.15.129:6811/1790706405","heartbeat_back_addr":"172.21.15.129:6815/1790706405","heartbeat_front_addr":"172.21.15.129:6813/1790706405","state":["exists","up"]},{"osd":3,"uuid":"d6c34e3b-1ac6-415c-9c56-38ae77a5c9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6827","nonce":393361453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6829","nonce":393361453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6833","nonce":393361453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6831","nonce":393361453}]},"public_addr":"172.21.15.129:6827/393361453","cluster_addr":"172.21.15.129:6829/393361453","heartbeat_back_addr":"172.21.15.129:6833/393361453","heartbeat_front_addr":"172.21.15.129:6831/393361453","state":["exists","up"]},{"osd":4,"uuid":"7d722e8c-c497-4f0c-9f61-d9826f8c93d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6809","nonce":2337546183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6811","nonce":2337546183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6815","nonce":2337546183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6813","nonce":2337546183}]},"public_addr":"172.21.15.171:6809/2337546183","cluster_addr":"172.21.15.171:6811/2337546183","heartbeat_back_addr":"172.21.15.171:6815/2337546183","heartbeat_front_addr":"172.21.15.171:6813/2337546183","state":["exists","up"]},{"osd":5,"uuid":"954f801b-3ac9-410a-a375-eb1c533744de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6817","nonce":1510320351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6819","nonce":1510320351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6823","nonce":1510320351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6821","nonce":1510320351}]},"public_addr":"172.21.15.171:6817/1510320351","cluster_addr":"172.21.15.171:6819/1510320351","heartbeat_back_addr":"172.21.15.171:6823/1510320351","heartbeat_front_addr":"172.21.15.171:6821/1510320351","state":["exists","up"]},{"osd":6,"uuid":"21837515-65a2-4f21-96d6-67cfc4589358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6825","nonce":471043930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6827","nonce":471043930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6831","nonce":471043930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6829","nonce":471043930}]},"public_addr":"172.21.15.171:6825/471043930","cluster_addr":"172.21.15.171:6827/471043930","heartbeat_back_addr":"172.21.15.171:6831/471043930","heartbeat_front_addr":"172.21.15.171:6829/471043930","state":["exists","up"]},{"osd":7,"uuid":"67af7eea-8803-425d-ae50-30a6ab91a711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6801","nonce":1770619177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6803","nonce":1770619177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6807","nonce":1770619177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6805","nonce":1770619177}]},"public_addr":"172.21.15.171:6801/1770619177","cluster_addr":"172.21.15.171:6803/1770619177","heartbeat_back_addr":"172.21.15.171:6807/1770619177","heartbeat_front_addr":"172.21.15.171:6805/1770619177","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:21.445879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.240967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.039097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.372014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.049543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.009384+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.048598+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.046345+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:29:36.243 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T21:29:36.244 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T21:29:36.244 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T21:29:36.245 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T21:29:36.245 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T21:29:36.245 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T21:29:36.246 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-17T21:29:36.246 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-17T21:29:36.488 INFO:teuthology.orchestra.run.smithi129.stdout:64424509446 2024-09-17T21:29:36.488 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T21:29:36.490 INFO:teuthology.orchestra.run.smithi129.stdout:64424509445 2024-09-17T21:29:36.490 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T21:29:36.491 INFO:teuthology.orchestra.run.smithi129.stdout:55834574854 2024-09-17T21:29:36.491 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T21:29:36.493 INFO:teuthology.orchestra.run.smithi129.stdout:64424509445 2024-09-17T21:29:36.493 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T21:29:36.494 INFO:teuthology.orchestra.run.smithi129.stdout:64424509445 2024-09-17T21:29:36.494 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T21:29:36.495 INFO:teuthology.orchestra.run.smithi129.stdout:64424509445 2024-09-17T21:29:36.495 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T21:29:36.496 INFO:teuthology.orchestra.run.smithi129.stdout:64424509445 2024-09-17T21:29:36.496 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T21:29:36.498 INFO:teuthology.orchestra.run.smithi129.stdout:64424509446 2024-09-17T21:29:36.499 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T21:29:37.032 INFO:teuthology.orchestra.run.smithi129.stdout:55834574854 2024-09-17T21:29:37.052 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-09-17T21:29:37.052 DEBUG:teuthology.parallel:result is None 2024-09-17T21:29:37.054 INFO:teuthology.orchestra.run.smithi129.stdout:64424509445 2024-09-17T21:29:37.063 INFO:teuthology.orchestra.run.smithi129.stdout:64424509446 2024-09-17T21:29:37.072 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.7 2024-09-17T21:29:37.072 DEBUG:teuthology.parallel:result is None 2024-09-17T21:29:37.082 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.3 2024-09-17T21:29:37.082 DEBUG:teuthology.parallel:result is None 2024-09-17T21:29:37.096 INFO:teuthology.orchestra.run.smithi129.stdout:64424509445 2024-09-17T21:29:37.106 INFO:teuthology.orchestra.run.smithi129.stdout:64424509445 2024-09-17T21:29:37.109 INFO:teuthology.orchestra.run.smithi129.stdout:64424509445 2024-09-17T21:29:37.110 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.1 2024-09-17T21:29:37.110 DEBUG:teuthology.parallel:result is None 2024-09-17T21:29:37.122 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.6 2024-09-17T21:29:37.122 DEBUG:teuthology.parallel:result is None 2024-09-17T21:29:37.123 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.4 2024-09-17T21:29:37.124 DEBUG:teuthology.parallel:result is None 2024-09-17T21:29:37.135 INFO:teuthology.orchestra.run.smithi129.stdout:64424509445 2024-09-17T21:29:37.152 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.2 2024-09-17T21:29:37.153 DEBUG:teuthology.parallel:result is None 2024-09-17T21:29:37.175 INFO:teuthology.orchestra.run.smithi129.stdout:64424509446 2024-09-17T21:29:37.188 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.5 2024-09-17T21:29:37.188 DEBUG:teuthology.parallel:result is None 2024-09-17T21:29:37.188 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T21:29:37.188 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:29:37.513 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:29:37.513 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-17T21:29:37.525 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-17T21:29:37.391631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218208,"kb_used_data":1932,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756512,"statfs":{"total":773094113280,"available":772870668288,"internally_reserved":0,"allocated":1978368,"data_stored":1808518,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.531803"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:35.002656+0000","last_change":"2024-09-17T21:29:35.002656+0000","last_active":"2024-09-17T21:29:35.002656+0000","last_peered":"2024-09-17T21:29:35.002656+0000","last_clean":"2024-09-17T21:29:35.002656+0000","last_became_active":"2024-09-17T21:29:32.971677+0000","last_became_peered":"2024-09-17T21:29:32.971677+0000","last_unstale":"2024-09-17T21:29:35.002656+0000","last_undegraded":"2024-09-17T21:29:35.002656+0000","last_fullsized":"2024-09-17T21:29:35.002656+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:53.781657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041121300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.980789+0000","last_change":"2024-09-17T21:29:34.980789+0000","last_active":"2024-09-17T21:29:34.980789+0000","last_peered":"2024-09-17T21:29:34.980789+0000","last_clean":"2024-09-17T21:29:34.980789+0000","last_became_active":"2024-09-17T21:29:32.970760+0000","last_became_peered":"2024-09-17T21:29:32.970760+0000","last_unstale":"2024-09-17T21:29:34.980789+0000","last_undegraded":"2024-09-17T21:29:34.980789+0000","last_fullsized":"2024-09-17T21:29:34.980789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:06:15.964358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049926799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:35.078854+0000","last_change":"2024-09-17T21:29:35.078854+0000","last_active":"2024-09-17T21:29:35.078854+0000","last_peered":"2024-09-17T21:29:35.078854+0000","last_clean":"2024-09-17T21:29:35.078854+0000","last_became_active":"2024-09-17T21:29:33.071665+0000","last_became_peered":"2024-09-17T21:29:33.071665+0000","last_unstale":"2024-09-17T21:29:35.078854+0000","last_undegraded":"2024-09-17T21:29:35.078854+0000","last_fullsized":"2024-09-17T21:29:35.078854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.415687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047969899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.980718+0000","last_change":"2024-09-17T21:29:34.980718+0000","last_active":"2024-09-17T21:29:34.980718+0000","last_peered":"2024-09-17T21:29:34.980718+0000","last_clean":"2024-09-17T21:29:34.980718+0000","last_became_active":"2024-09-17T21:29:33.071515+0000","last_became_peered":"2024-09-17T21:29:33.071515+0000","last_unstale":"2024-09-17T21:29:34.980718+0000","last_undegraded":"2024-09-17T21:29:34.980718+0000","last_fullsized":"2024-09-17T21:29:34.980718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:03:09.322560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051127799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.986925+0000","last_change":"2024-09-17T21:29:34.985098+0000","last_active":"2024-09-17T21:29:34.986925+0000","last_peered":"2024-09-17T21:29:34.986925+0000","last_clean":"2024-09-17T21:29:34.986925+0000","last_became_active":"2024-09-17T21:29:32.971754+0000","last_became_peered":"2024-09-17T21:29:32.971754+0000","last_unstale":"2024-09-17T21:29:34.986925+0000","last_undegraded":"2024-09-17T21:29:34.986925+0000","last_fullsized":"2024-09-17T21:29:34.986925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:55.964827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001139555,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.981930+0000","last_change":"2024-09-17T21:29:34.981930+0000","last_active":"2024-09-17T21:29:34.981930+0000","last_peered":"2024-09-17T21:29:34.981930+0000","last_clean":"2024-09-17T21:29:34.981930+0000","last_became_active":"2024-09-17T21:29:33.071313+0000","last_became_peered":"2024-09-17T21:29:33.071313+0000","last_unstale":"2024-09-17T21:29:34.981930+0000","last_undegraded":"2024-09-17T21:29:34.981930+0000","last_fullsized":"2024-09-17T21:29:34.981930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:11:51.460374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000527295,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:35.078843+0000","last_change":"2024-09-17T21:29:35.078843+0000","last_active":"2024-09-17T21:29:35.078843+0000","last_peered":"2024-09-17T21:29:35.078843+0000","last_clean":"2024-09-17T21:29:35.078843+0000","last_became_active":"2024-09-17T21:29:32.971797+0000","last_became_peered":"2024-09-17T21:29:32.971797+0000","last_unstale":"2024-09-17T21:29:35.078843+0000","last_undegraded":"2024-09-17T21:29:35.078843+0000","last_fullsized":"2024-09-17T21:29:35.078843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:05:48.959445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044115199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.984547+0000","last_change":"2024-09-17T21:29:34.984547+0000","last_active":"2024-09-17T21:29:34.984547+0000","last_peered":"2024-09-17T21:29:34.984547+0000","last_clean":"2024-09-17T21:29:34.984547+0000","last_became_active":"2024-09-17T21:29:32.971816+0000","last_became_peered":"2024-09-17T21:29:32.971816+0000","last_unstale":"2024-09-17T21:29:34.984547+0000","last_undegraded":"2024-09-17T21:29:34.984547+0000","last_fullsized":"2024-09-17T21:29:34.984547+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:16.578606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052285799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:35.078303+0000","last_change":"2024-09-17T21:29:28.554672+0000","last_active":"2024-09-17T21:29:35.078303+0000","last_peered":"2024-09-17T21:29:35.078303+0000","last_clean":"2024-09-17T21:29:35.078303+0000","last_became_active":"2024-09-17T21:29:28.553733+0000","last_became_peered":"2024-09-17T21:29:28.553733+0000","last_unstale":"2024-09-17T21:29:35.078303+0000","last_undegraded":"2024-09-17T21:29:35.078303+0000","last_fullsized":"2024-09-17T21:29:35.078303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_clean_scrub_stamp":"2024-09-17T21:29:26.925225+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:36:42.585477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":669565,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":79216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":79216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":663733,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:29:37.526 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:29:37.805 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:29:37.805 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-17T21:29:37.817 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-17T21:29:37.391631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218208,"kb_used_data":1932,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756512,"statfs":{"total":773094113280,"available":772870668288,"internally_reserved":0,"allocated":1978368,"data_stored":1808518,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.531803"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:35.002656+0000","last_change":"2024-09-17T21:29:35.002656+0000","last_active":"2024-09-17T21:29:35.002656+0000","last_peered":"2024-09-17T21:29:35.002656+0000","last_clean":"2024-09-17T21:29:35.002656+0000","last_became_active":"2024-09-17T21:29:32.971677+0000","last_became_peered":"2024-09-17T21:29:32.971677+0000","last_unstale":"2024-09-17T21:29:35.002656+0000","last_undegraded":"2024-09-17T21:29:35.002656+0000","last_fullsized":"2024-09-17T21:29:35.002656+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:53.781657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041121300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.980789+0000","last_change":"2024-09-17T21:29:34.980789+0000","last_active":"2024-09-17T21:29:34.980789+0000","last_peered":"2024-09-17T21:29:34.980789+0000","last_clean":"2024-09-17T21:29:34.980789+0000","last_became_active":"2024-09-17T21:29:32.970760+0000","last_became_peered":"2024-09-17T21:29:32.970760+0000","last_unstale":"2024-09-17T21:29:34.980789+0000","last_undegraded":"2024-09-17T21:29:34.980789+0000","last_fullsized":"2024-09-17T21:29:34.980789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:06:15.964358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049926799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:35.078854+0000","last_change":"2024-09-17T21:29:35.078854+0000","last_active":"2024-09-17T21:29:35.078854+0000","last_peered":"2024-09-17T21:29:35.078854+0000","last_clean":"2024-09-17T21:29:35.078854+0000","last_became_active":"2024-09-17T21:29:33.071665+0000","last_became_peered":"2024-09-17T21:29:33.071665+0000","last_unstale":"2024-09-17T21:29:35.078854+0000","last_undegraded":"2024-09-17T21:29:35.078854+0000","last_fullsized":"2024-09-17T21:29:35.078854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.415687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047969899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.980718+0000","last_change":"2024-09-17T21:29:34.980718+0000","last_active":"2024-09-17T21:29:34.980718+0000","last_peered":"2024-09-17T21:29:34.980718+0000","last_clean":"2024-09-17T21:29:34.980718+0000","last_became_active":"2024-09-17T21:29:33.071515+0000","last_became_peered":"2024-09-17T21:29:33.071515+0000","last_unstale":"2024-09-17T21:29:34.980718+0000","last_undegraded":"2024-09-17T21:29:34.980718+0000","last_fullsized":"2024-09-17T21:29:34.980718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:03:09.322560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051127799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.986925+0000","last_change":"2024-09-17T21:29:34.985098+0000","last_active":"2024-09-17T21:29:34.986925+0000","last_peered":"2024-09-17T21:29:34.986925+0000","last_clean":"2024-09-17T21:29:34.986925+0000","last_became_active":"2024-09-17T21:29:32.971754+0000","last_became_peered":"2024-09-17T21:29:32.971754+0000","last_unstale":"2024-09-17T21:29:34.986925+0000","last_undegraded":"2024-09-17T21:29:34.986925+0000","last_fullsized":"2024-09-17T21:29:34.986925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:55.964827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001139555,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.981930+0000","last_change":"2024-09-17T21:29:34.981930+0000","last_active":"2024-09-17T21:29:34.981930+0000","last_peered":"2024-09-17T21:29:34.981930+0000","last_clean":"2024-09-17T21:29:34.981930+0000","last_became_active":"2024-09-17T21:29:33.071313+0000","last_became_peered":"2024-09-17T21:29:33.071313+0000","last_unstale":"2024-09-17T21:29:34.981930+0000","last_undegraded":"2024-09-17T21:29:34.981930+0000","last_fullsized":"2024-09-17T21:29:34.981930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:11:51.460374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000527295,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:35.078843+0000","last_change":"2024-09-17T21:29:35.078843+0000","last_active":"2024-09-17T21:29:35.078843+0000","last_peered":"2024-09-17T21:29:35.078843+0000","last_clean":"2024-09-17T21:29:35.078843+0000","last_became_active":"2024-09-17T21:29:32.971797+0000","last_became_peered":"2024-09-17T21:29:32.971797+0000","last_unstale":"2024-09-17T21:29:35.078843+0000","last_undegraded":"2024-09-17T21:29:35.078843+0000","last_fullsized":"2024-09-17T21:29:35.078843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:05:48.959445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044115199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.984547+0000","last_change":"2024-09-17T21:29:34.984547+0000","last_active":"2024-09-17T21:29:34.984547+0000","last_peered":"2024-09-17T21:29:34.984547+0000","last_clean":"2024-09-17T21:29:34.984547+0000","last_became_active":"2024-09-17T21:29:32.971816+0000","last_became_peered":"2024-09-17T21:29:32.971816+0000","last_unstale":"2024-09-17T21:29:34.984547+0000","last_undegraded":"2024-09-17T21:29:34.984547+0000","last_fullsized":"2024-09-17T21:29:34.984547+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:16.578606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052285799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:35.078303+0000","last_change":"2024-09-17T21:29:28.554672+0000","last_active":"2024-09-17T21:29:35.078303+0000","last_peered":"2024-09-17T21:29:35.078303+0000","last_clean":"2024-09-17T21:29:35.078303+0000","last_became_active":"2024-09-17T21:29:28.553733+0000","last_became_peered":"2024-09-17T21:29:28.553733+0000","last_unstale":"2024-09-17T21:29:35.078303+0000","last_undegraded":"2024-09-17T21:29:35.078303+0000","last_fullsized":"2024-09-17T21:29:35.078303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_clean_scrub_stamp":"2024-09-17T21:29:26.925225+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:36:42.585477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":669565,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":79216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":79216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":663733,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:29:37.818 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T21:29:37.818 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T21:29:37.818 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-17T21:29:37.818 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T21:29:38.136 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:29:38.137 INFO:teuthology.orchestra.run.smithi129.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T21:29:38.149 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-17T21:29:38.150 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-17T21:29:38.163 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-17T21:29:38.163 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '08cec6974d86f81dc304360b31113f30b4085bf8'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '08cec6974d86f81dc304360b31113f30b4085bf8'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '5ecc7400dbbb63ce990b663d2a92e87dbc17f30d'}} 2024-09-17T21:29:38.163 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-17T21:29:38.163 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-17T21:29:38.471 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-17T21:29:38.471 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T21:29:38.471 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:29:38.753 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:29:38.754 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-17T21:29:38.766 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-17T21:29:37.391631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218208,"kb_used_data":1932,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756512,"statfs":{"total":773094113280,"available":772870668288,"internally_reserved":0,"allocated":1978368,"data_stored":1808518,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.531803"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:35.002656+0000","last_change":"2024-09-17T21:29:35.002656+0000","last_active":"2024-09-17T21:29:35.002656+0000","last_peered":"2024-09-17T21:29:35.002656+0000","last_clean":"2024-09-17T21:29:35.002656+0000","last_became_active":"2024-09-17T21:29:32.971677+0000","last_became_peered":"2024-09-17T21:29:32.971677+0000","last_unstale":"2024-09-17T21:29:35.002656+0000","last_undegraded":"2024-09-17T21:29:35.002656+0000","last_fullsized":"2024-09-17T21:29:35.002656+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:53.781657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041121300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.980789+0000","last_change":"2024-09-17T21:29:34.980789+0000","last_active":"2024-09-17T21:29:34.980789+0000","last_peered":"2024-09-17T21:29:34.980789+0000","last_clean":"2024-09-17T21:29:34.980789+0000","last_became_active":"2024-09-17T21:29:32.970760+0000","last_became_peered":"2024-09-17T21:29:32.970760+0000","last_unstale":"2024-09-17T21:29:34.980789+0000","last_undegraded":"2024-09-17T21:29:34.980789+0000","last_fullsized":"2024-09-17T21:29:34.980789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:06:15.964358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049926799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:35.078854+0000","last_change":"2024-09-17T21:29:35.078854+0000","last_active":"2024-09-17T21:29:35.078854+0000","last_peered":"2024-09-17T21:29:35.078854+0000","last_clean":"2024-09-17T21:29:35.078854+0000","last_became_active":"2024-09-17T21:29:33.071665+0000","last_became_peered":"2024-09-17T21:29:33.071665+0000","last_unstale":"2024-09-17T21:29:35.078854+0000","last_undegraded":"2024-09-17T21:29:35.078854+0000","last_fullsized":"2024-09-17T21:29:35.078854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.415687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047969899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.980718+0000","last_change":"2024-09-17T21:29:34.980718+0000","last_active":"2024-09-17T21:29:34.980718+0000","last_peered":"2024-09-17T21:29:34.980718+0000","last_clean":"2024-09-17T21:29:34.980718+0000","last_became_active":"2024-09-17T21:29:33.071515+0000","last_became_peered":"2024-09-17T21:29:33.071515+0000","last_unstale":"2024-09-17T21:29:34.980718+0000","last_undegraded":"2024-09-17T21:29:34.980718+0000","last_fullsized":"2024-09-17T21:29:34.980718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:03:09.322560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051127799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.986925+0000","last_change":"2024-09-17T21:29:34.985098+0000","last_active":"2024-09-17T21:29:34.986925+0000","last_peered":"2024-09-17T21:29:34.986925+0000","last_clean":"2024-09-17T21:29:34.986925+0000","last_became_active":"2024-09-17T21:29:32.971754+0000","last_became_peered":"2024-09-17T21:29:32.971754+0000","last_unstale":"2024-09-17T21:29:34.986925+0000","last_undegraded":"2024-09-17T21:29:34.986925+0000","last_fullsized":"2024-09-17T21:29:34.986925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:55.964827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001139555,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.981930+0000","last_change":"2024-09-17T21:29:34.981930+0000","last_active":"2024-09-17T21:29:34.981930+0000","last_peered":"2024-09-17T21:29:34.981930+0000","last_clean":"2024-09-17T21:29:34.981930+0000","last_became_active":"2024-09-17T21:29:33.071313+0000","last_became_peered":"2024-09-17T21:29:33.071313+0000","last_unstale":"2024-09-17T21:29:34.981930+0000","last_undegraded":"2024-09-17T21:29:34.981930+0000","last_fullsized":"2024-09-17T21:29:34.981930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:11:51.460374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000527295,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:35.078843+0000","last_change":"2024-09-17T21:29:35.078843+0000","last_active":"2024-09-17T21:29:35.078843+0000","last_peered":"2024-09-17T21:29:35.078843+0000","last_clean":"2024-09-17T21:29:35.078843+0000","last_became_active":"2024-09-17T21:29:32.971797+0000","last_became_peered":"2024-09-17T21:29:32.971797+0000","last_unstale":"2024-09-17T21:29:35.078843+0000","last_undegraded":"2024-09-17T21:29:35.078843+0000","last_fullsized":"2024-09-17T21:29:35.078843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:05:48.959445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044115199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.984547+0000","last_change":"2024-09-17T21:29:34.984547+0000","last_active":"2024-09-17T21:29:34.984547+0000","last_peered":"2024-09-17T21:29:34.984547+0000","last_clean":"2024-09-17T21:29:34.984547+0000","last_became_active":"2024-09-17T21:29:32.971816+0000","last_became_peered":"2024-09-17T21:29:32.971816+0000","last_unstale":"2024-09-17T21:29:34.984547+0000","last_undegraded":"2024-09-17T21:29:34.984547+0000","last_fullsized":"2024-09-17T21:29:34.984547+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:16.578606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052285799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:35.078303+0000","last_change":"2024-09-17T21:29:28.554672+0000","last_active":"2024-09-17T21:29:35.078303+0000","last_peered":"2024-09-17T21:29:35.078303+0000","last_clean":"2024-09-17T21:29:35.078303+0000","last_became_active":"2024-09-17T21:29:28.553733+0000","last_became_peered":"2024-09-17T21:29:28.553733+0000","last_unstale":"2024-09-17T21:29:35.078303+0000","last_undegraded":"2024-09-17T21:29:35.078303+0000","last_fullsized":"2024-09-17T21:29:35.078303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_clean_scrub_stamp":"2024-09-17T21:29:26.925225+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:36:42.585477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":669565,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":79216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":79216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":663733,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:29:38.767 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:29:39.050 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:29:39.051 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-17T21:29:39.063 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-17T21:29:38.861090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218208,"kb_used_data":1932,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756512,"statfs":{"total":773094113280,"available":772870668288,"internally_reserved":0,"allocated":1978368,"data_stored":1808518,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.898546"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:35.002656+0000","last_change":"2024-09-17T21:29:35.002656+0000","last_active":"2024-09-17T21:29:35.002656+0000","last_peered":"2024-09-17T21:29:35.002656+0000","last_clean":"2024-09-17T21:29:35.002656+0000","last_became_active":"2024-09-17T21:29:32.971677+0000","last_became_peered":"2024-09-17T21:29:32.971677+0000","last_unstale":"2024-09-17T21:29:35.002656+0000","last_undegraded":"2024-09-17T21:29:35.002656+0000","last_fullsized":"2024-09-17T21:29:35.002656+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:53.781657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041121300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.980789+0000","last_change":"2024-09-17T21:29:34.980789+0000","last_active":"2024-09-17T21:29:34.980789+0000","last_peered":"2024-09-17T21:29:34.980789+0000","last_clean":"2024-09-17T21:29:34.980789+0000","last_became_active":"2024-09-17T21:29:32.970760+0000","last_became_peered":"2024-09-17T21:29:32.970760+0000","last_unstale":"2024-09-17T21:29:34.980789+0000","last_undegraded":"2024-09-17T21:29:34.980789+0000","last_fullsized":"2024-09-17T21:29:34.980789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:06:15.964358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049926799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:35.078854+0000","last_change":"2024-09-17T21:29:35.078854+0000","last_active":"2024-09-17T21:29:35.078854+0000","last_peered":"2024-09-17T21:29:35.078854+0000","last_clean":"2024-09-17T21:29:35.078854+0000","last_became_active":"2024-09-17T21:29:33.071665+0000","last_became_peered":"2024-09-17T21:29:33.071665+0000","last_unstale":"2024-09-17T21:29:35.078854+0000","last_undegraded":"2024-09-17T21:29:35.078854+0000","last_fullsized":"2024-09-17T21:29:35.078854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.415687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047969899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.980718+0000","last_change":"2024-09-17T21:29:34.980718+0000","last_active":"2024-09-17T21:29:34.980718+0000","last_peered":"2024-09-17T21:29:34.980718+0000","last_clean":"2024-09-17T21:29:34.980718+0000","last_became_active":"2024-09-17T21:29:33.071515+0000","last_became_peered":"2024-09-17T21:29:33.071515+0000","last_unstale":"2024-09-17T21:29:34.980718+0000","last_undegraded":"2024-09-17T21:29:34.980718+0000","last_fullsized":"2024-09-17T21:29:34.980718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:03:09.322560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051127799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.986925+0000","last_change":"2024-09-17T21:29:34.985098+0000","last_active":"2024-09-17T21:29:34.986925+0000","last_peered":"2024-09-17T21:29:34.986925+0000","last_clean":"2024-09-17T21:29:34.986925+0000","last_became_active":"2024-09-17T21:29:32.971754+0000","last_became_peered":"2024-09-17T21:29:32.971754+0000","last_unstale":"2024-09-17T21:29:34.986925+0000","last_undegraded":"2024-09-17T21:29:34.986925+0000","last_fullsized":"2024-09-17T21:29:34.986925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:55.964827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001139555,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.981930+0000","last_change":"2024-09-17T21:29:34.981930+0000","last_active":"2024-09-17T21:29:34.981930+0000","last_peered":"2024-09-17T21:29:34.981930+0000","last_clean":"2024-09-17T21:29:34.981930+0000","last_became_active":"2024-09-17T21:29:33.071313+0000","last_became_peered":"2024-09-17T21:29:33.071313+0000","last_unstale":"2024-09-17T21:29:34.981930+0000","last_undegraded":"2024-09-17T21:29:34.981930+0000","last_fullsized":"2024-09-17T21:29:34.981930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:11:51.460374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000527295,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:35.078843+0000","last_change":"2024-09-17T21:29:35.078843+0000","last_active":"2024-09-17T21:29:35.078843+0000","last_peered":"2024-09-17T21:29:35.078843+0000","last_clean":"2024-09-17T21:29:35.078843+0000","last_became_active":"2024-09-17T21:29:32.971797+0000","last_became_peered":"2024-09-17T21:29:32.971797+0000","last_unstale":"2024-09-17T21:29:35.078843+0000","last_undegraded":"2024-09-17T21:29:35.078843+0000","last_fullsized":"2024-09-17T21:29:35.078843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:05:48.959445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044115199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:34.984547+0000","last_change":"2024-09-17T21:29:34.984547+0000","last_active":"2024-09-17T21:29:34.984547+0000","last_peered":"2024-09-17T21:29:34.984547+0000","last_clean":"2024-09-17T21:29:34.984547+0000","last_became_active":"2024-09-17T21:29:32.971816+0000","last_became_peered":"2024-09-17T21:29:32.971816+0000","last_unstale":"2024-09-17T21:29:34.984547+0000","last_undegraded":"2024-09-17T21:29:34.984547+0000","last_fullsized":"2024-09-17T21:29:34.984547+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:16.578606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052285799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T21:29:35.078303+0000","last_change":"2024-09-17T21:29:28.554672+0000","last_active":"2024-09-17T21:29:35.078303+0000","last_peered":"2024-09-17T21:29:35.078303+0000","last_clean":"2024-09-17T21:29:35.078303+0000","last_became_active":"2024-09-17T21:29:28.553733+0000","last_became_peered":"2024-09-17T21:29:28.553733+0000","last_unstale":"2024-09-17T21:29:35.078303+0000","last_undegraded":"2024-09-17T21:29:35.078303+0000","last_fullsized":"2024-09-17T21:29:35.078303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_clean_scrub_stamp":"2024-09-17T21:29:26.925225+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:36:42.585477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":669565,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":79216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":79216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":663733,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:29:39.064 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T21:29:39.064 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-17T21:29:39.350 INFO:teuthology.orchestra.run.smithi129.stdout:epoch 24 2024-09-17T21:29:39.350 INFO:teuthology.orchestra.run.smithi129.stdout:fsid f6d3a80c-8a1f-4716-b25d-e97b28890c0d 2024-09-17T21:29:39.350 INFO:teuthology.orchestra.run.smithi129.stdout:created 2024-09-17T21:29:12.606637+0000 2024-09-17T21:29:39.350 INFO:teuthology.orchestra.run.smithi129.stdout:modified 2024-09-17T21:29:37.389840+0000 2024-09-17T21:29:39.350 INFO:teuthology.orchestra.run.smithi129.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-17T21:29:39.350 INFO:teuthology.orchestra.run.smithi129.stdout:crush_version 5 2024-09-17T21:29:39.350 INFO:teuthology.orchestra.run.smithi129.stdout:full_ratio 0.95 2024-09-17T21:29:39.351 INFO:teuthology.orchestra.run.smithi129.stdout:backfillfull_ratio 0.9 2024-09-17T21:29:39.351 INFO:teuthology.orchestra.run.smithi129.stdout:nearfull_ratio 0.85 2024-09-17T21:29:39.351 INFO:teuthology.orchestra.run.smithi129.stdout:require_min_compat_client luminous 2024-09-17T21:29:39.351 INFO:teuthology.orchestra.run.smithi129.stdout:min_compat_client jewel 2024-09-17T21:29:39.351 INFO:teuthology.orchestra.run.smithi129.stdout:require_osd_release squid 2024-09-17T21:29:39.351 INFO:teuthology.orchestra.run.smithi129.stdout:stretch_mode_enabled false 2024-09-17T21:29:39.351 INFO:teuthology.orchestra.run.smithi129.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 19 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-17T21:29:39.351 INFO:teuthology.orchestra.run.smithi129.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 23 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-17T21:29:39.351 INFO:teuthology.orchestra.run.smithi129.stdout:max_osd 8 2024-09-17T21:29:39.351 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6800/3635367421,v1:172.21.15.129:6801/3635367421] [v2:172.21.15.129:6802/3635367421,v1:172.21.15.129:6803/3635367421] exists,up ad067b51-9402-4f69-a0f0-08fbf068dcfc 2024-09-17T21:29:39.351 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6818/176704438,v1:172.21.15.129:6819/176704438] [v2:172.21.15.129:6820/176704438,v1:172.21.15.129:6821/176704438] exists,up 30385fc0-ef5f-45f5-a3ec-37e890d4826e 2024-09-17T21:29:39.351 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6808/1790706405,v1:172.21.15.129:6809/1790706405] [v2:172.21.15.129:6810/1790706405,v1:172.21.15.129:6811/1790706405] exists,up 76be9b2b-b63f-402a-9ab5-2564bd671ac3 2024-09-17T21:29:39.351 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6826/393361453,v1:172.21.15.129:6827/393361453] [v2:172.21.15.129:6828/393361453,v1:172.21.15.129:6829/393361453] exists,up d6c34e3b-1ac6-415c-9c56-38ae77a5c9b0 2024-09-17T21:29:39.352 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6808/2337546183,v1:172.21.15.171:6809/2337546183] [v2:172.21.15.171:6810/2337546183,v1:172.21.15.171:6811/2337546183] exists,up 7d722e8c-c497-4f0c-9f61-d9826f8c93d8 2024-09-17T21:29:39.352 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6816/1510320351,v1:172.21.15.171:6817/1510320351] [v2:172.21.15.171:6818/1510320351,v1:172.21.15.171:6819/1510320351] exists,up 954f801b-3ac9-410a-a375-eb1c533744de 2024-09-17T21:29:39.352 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6824/471043930,v1:172.21.15.171:6825/471043930] [v2:172.21.15.171:6826/471043930,v1:172.21.15.171:6827/471043930] exists,up 21837515-65a2-4f21-96d6-67cfc4589358 2024-09-17T21:29:39.352 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6800/1770619177,v1:172.21.15.171:6801/1770619177] [v2:172.21.15.171:6802/1770619177,v1:172.21.15.171:6803/1770619177] exists,up 67af7eea-8803-425d-ae50-30a6ab91a711 2024-09-17T21:29:39.363 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6800/3635367421,v1:172.21.15.129:6801/3635367421] [v2:172.21.15.129:6802/3635367421,v1:172.21.15.129:6803/3635367421] exists,up ad067b51-9402-4f69-a0f0-08fbf068dcfc', 'osd.1 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6818/176704438,v1:172.21.15.129:6819/176704438] [v2:172.21.15.129:6820/176704438,v1:172.21.15.129:6821/176704438] exists,up 30385fc0-ef5f-45f5-a3ec-37e890d4826e', 'osd.2 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6808/1790706405,v1:172.21.15.129:6809/1790706405] [v2:172.21.15.129:6810/1790706405,v1:172.21.15.129:6811/1790706405] exists,up 76be9b2b-b63f-402a-9ab5-2564bd671ac3', 'osd.3 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6826/393361453,v1:172.21.15.129:6827/393361453] [v2:172.21.15.129:6828/393361453,v1:172.21.15.129:6829/393361453] exists,up d6c34e3b-1ac6-415c-9c56-38ae77a5c9b0', 'osd.4 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6808/2337546183,v1:172.21.15.171:6809/2337546183] [v2:172.21.15.171:6810/2337546183,v1:172.21.15.171:6811/2337546183] exists,up 7d722e8c-c497-4f0c-9f61-d9826f8c93d8', 'osd.5 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6816/1510320351,v1:172.21.15.171:6817/1510320351] [v2:172.21.15.171:6818/1510320351,v1:172.21.15.171:6819/1510320351] exists,up 954f801b-3ac9-410a-a375-eb1c533744de', 'osd.6 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6824/471043930,v1:172.21.15.171:6825/471043930] [v2:172.21.15.171:6826/471043930,v1:172.21.15.171:6827/471043930] exists,up 21837515-65a2-4f21-96d6-67cfc4589358', 'osd.7 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6800/1770619177,v1:172.21.15.171:6801/1770619177] [v2:172.21.15.171:6802/1770619177,v1:172.21.15.171:6803/1770619177] exists,up 67af7eea-8803-425d-ae50-30a6ab91a711'] 2024-09-17T21:29:39.364 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-17T21:29:39.462 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mon.a", 2024-09-17T21:29:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster": "ceph", 2024-09-17T21:29:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-17T21:29:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: "admin_socket_mode": "", 2024-09-17T21:29:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-17T21:29:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: "auth_client_required": "cephx, none", 2024-09-17T21:29:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: "auth_cluster_required": "cephx", 2024-09-17T21:29:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: "auth_debug": "true", 2024-09-17T21:29:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-17T21:29:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-17T21:29:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "auth_service_required": "cephx", 2024-09-17T21:29:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-17T21:29:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "auth_supported": "cephx", 2024-09-17T21:29:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_aio": "true", 2024-09-17T21:29:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-17T21:29:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_aio_poll_ms": "250", 2024-09-17T21:29:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_aio_reap_max": "16", 2024-09-17T21:29:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_async_discard_threads": "0", 2024-09-17T21:29:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_block_size": "4096", 2024-09-17T21:29:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_debug_aio": "false", 2024-09-17T21:29:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-17T21:29:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-17T21:29:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_debug_inflight_ios": "false", 2024-09-17T21:29:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_enable_discard": "false", 2024-09-17T21:29:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_flock_retry": "3", 2024-09-17T21:29:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-17T21:29:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_inject_crash": "0", 2024-09-17T21:29:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-17T21:29:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_ioring": "false", 2024-09-17T21:29:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_ioring_hipri": "false", 2024-09-17T21:29:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-17T21:29:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-17T21:29:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-17T21:29:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-17T21:29:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-17T21:29:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-17T21:29:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_type": "", 2024-09-17T21:29:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_alloc_size": "1048576", 2024-09-17T21:29:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_allocator": "hybrid", 2024-09-17T21:29:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_buffered_io": "true", 2024-09-17T21:29:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_check_for_zeros": "false", 2024-09-17T21:29:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-17T21:29:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-17T21:29:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_compact_log_sync": "false", 2024-09-17T21:29:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-17T21:29:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-17T21:29:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-17T21:29:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-17T21:29:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_max_log_runway": "4194304", 2024-09-17T21:29:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_max_prefetch": "1048576", 2024-09-17T21:29:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_min_flush_size": "524288", 2024-09-17T21:29:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_min_log_runway": "1048576", 2024-09-17T21:29:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_replay_recovery": "false", 2024-09-17T21:29:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-17T21:29:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-17T21:29:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_sync_write": "false", 2024-09-17T21:29:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-17T21:29:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-17T21:29:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-17T21:29:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_allocation_from_file": "true", 2024-09-17T21:29:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_allocator": "hybrid", 2024-09-17T21:29:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_async_db_compaction": "true", 2024-09-17T21:29:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-17T21:29:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-17T21:29:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-17T21:29:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-17T21:29:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_bdev_label_multi": "true", 2024-09-17T21:29:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-17T21:29:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-17T21:29:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-17T21:29:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-17T21:29:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-17T21:29:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_create": "true", 2024-09-17T21:29:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_db_create": "false", 2024-09-17T21:29:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_db_path": "", 2024-09-17T21:29:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_db_size": "0", 2024-09-17T21:29:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_path": "", 2024-09-17T21:29:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_preallocate_file": "false", 2024-09-17T21:29:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_size": "107374182400", 2024-09-17T21:29:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_wal_create": "false", 2024-09-17T21:29:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_wal_path": "", 2024-09-17T21:29:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_wal_size": "100663296", 2024-09-17T21:29:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_bluefs": "true", 2024-09-17T21:29:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-17T21:29:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-17T21:29:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-17T21:29:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-17T21:29:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-17T21:29:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T21:29:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T21:29:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-17T21:29:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T21:29:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_autotune": "true", 2024-09-17T21:29:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-17T21:29:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-17T21:29:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-17T21:29:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-17T21:29:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_size": "0", 2024-09-17T21:29:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-17T21:29:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-17T21:29:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-17T21:29:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-17T21:29:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_type": "2q", 2024-09-17T21:29:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-17T21:29:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_clone_cow": "true", 2024-09-17T21:29:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-17T21:29:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-17T21:29:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-17T21:29:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-17T21:29:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-17T21:29:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-17T21:29:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-17T21:29:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_compression_mode": "none", 2024-09-17T21:29:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-17T21:29:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_csum_type": "crc32c", 2024-09-17T21:29:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-17T21:29:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-17T21:29:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_freelist": "false", 2024-09-17T21:29:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-17T21:29:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-17T21:29:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-17T21:29:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-17T21:29:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-17T21:29:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-17T21:29:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-17T21:29:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-17T21:29:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-17T21:29:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-17T21:29:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-17T21:29:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-17T21:29:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_small_allocations": "0", 2024-09-17T21:29:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-17T21:29:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_default_buffered_read": "true", 2024-09-17T21:29:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_default_buffered_write": "false", 2024-09-17T21:29:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-17T21:29:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-17T21:29:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-17T21:29:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-17T21:29:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-17T21:29:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-17T21:29:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-17T21:29:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-17T21:29:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-17T21:29:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fail_eio": "false", 2024-09-17T21:29:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-17T21:29:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-17T21:29:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-17T21:29:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-17T21:29:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-17T21:29:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-17T21:29:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_on_mount": "false", 2024-09-17T21:29:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-17T21:29:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_on_umount": "false", 2024-09-17T21:29:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-17T21:29:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-17T21:29:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-17T21:29:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-17T21:29:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-17T21:29:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-17T21:29:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-17T21:29:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-17T21:29:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_ignore_data_csum": "false", 2024-09-17T21:29:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-17T21:29:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-17T21:29:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-17T21:29:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-17T21:29:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_log_op_age": "5.000000", 2024-09-17T21:29:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_max_alloc_size": "0", 2024-09-17T21:29:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_max_blob_size": "0", 2024-09-17T21:29:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-17T21:29:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-17T21:29:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-17T21:29:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_max_deferred_txc": "32", 2024-09-17T21:29:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_min_alloc_size": "0", 2024-09-17T21:29:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-17T21:29:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-17T21:29:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_nid_prealloc": "1024", 2024-09-17T21:29:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-17T21:29:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-17T21:29:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-17T21:29:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-17T21:29:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_retry_disk_reads": "3", 2024-09-17T21:29:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_rocksdb_cf": "true", 2024-09-17T21:29:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-17T21:29:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-17T21:29:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-17T21:29:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-17T21:29:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-17T21:29:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-17T21:29:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-17T21:29:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-17T21:29:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_spdk_mem": "512", 2024-09-17T21:29:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-17T21:29:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-17T21:29:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-17T21:29:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-17T21:29:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-17T21:29:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-17T21:29:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-17T21:29:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_tracing": "false", 2024-09-17T21:29:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-17T21:29:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-17T21:29:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-17T21:29:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-17T21:29:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-17T21:29:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-17T21:29:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-17T21:29:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-17T21:29:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-17T21:29:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_write_v2": "false", 2024-09-17T21:29:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_write_v2_random": "false", 2024-09-17T21:29:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_zero_block_detection": "false", 2024-09-17T21:29:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-17T21:29:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-17T21:29:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-17T21:29:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-17T21:29:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-17T21:29:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-17T21:29:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-17T21:29:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-17T21:29:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-17T21:29:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-17T21:29:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-17T21:29:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-17T21:29:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-17T21:29:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-17T21:29:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "cephx_cluster_require_signatures": "false", 2024-09-17T21:29:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "cephx_cluster_require_version": "2", 2024-09-17T21:29:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "cephx_require_signatures": "false", 2024-09-17T21:29:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "cephx_require_version": "2", 2024-09-17T21:29:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "cephx_service_require_signatures": "false", 2024-09-17T21:29:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "cephx_service_require_version": "2", 2024-09-17T21:29:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "cephx_sign_messages": "true", 2024-09-17T21:29:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "chdir": "", 2024-09-17T21:29:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_acl_type": "", 2024-09-17T21:29:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_asio_thread_count": "2", 2024-09-17T21:29:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_cache_mid": "0.750000", 2024-09-17T21:29:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_cache_size": "16384", 2024-09-17T21:29:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_caps_release_delay": "5", 2024-09-17T21:29:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_check_pool_perm": "true", 2024-09-17T21:29:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-17T21:29:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_debug_force_sync_read": "false", 2024-09-17T21:29:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_debug_getattr_caps": "false", 2024-09-17T21:29:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_debug_inject_features": "", 2024-09-17T21:29:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_debug_inject_tick_delay": "0", 2024-09-17T21:29:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-17T21:29:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_die_on_failed_remount": "false", 2024-09-17T21:29:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_dirsize_rbytes": "true", 2024-09-17T21:29:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_force_lazyio": "false", 2024-09-17T21:29:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_fs": "", 2024-09-17T21:29:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-17T21:29:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_inject_release_failure": "false", 2024-09-17T21:29:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_max_inline_size": "4096", 2024-09-17T21:29:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-17T21:29:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_mds_namespace": "", 2024-09-17T21:29:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_metadata": "", 2024-09-17T21:29:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_mount_gid": "-1", 2024-09-17T21:29:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_mount_timeout": "300", 2024-09-17T21:29:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_mount_uid": "-1", 2024-09-17T21:29:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_mountpoint": "/", 2024-09-17T21:29:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_notify_timeout": "10", 2024-09-17T21:29:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_oc": "true", 2024-09-17T21:29:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "client_oc_max_dirty": "104857600", 2024-09-17T21:29:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-17T21:29:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "client_oc_max_objects": "1000", 2024-09-17T21:29:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "client_oc_size": "209715200", 2024-09-17T21:29:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "client_oc_target_dirty": "8388608", 2024-09-17T21:29:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "client_permissions": "true", 2024-09-17T21:29:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "client_quota": "true", 2024-09-17T21:29:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "client_quota_df": "true", 2024-09-17T21:29:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "client_readahead_max_bytes": "0", 2024-09-17T21:29:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "client_readahead_max_periods": "4", 2024-09-17T21:29:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "client_readahead_min": "131072", 2024-09-17T21:29:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "client_reconnect_stale": "false", 2024-09-17T21:29:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "client_shutdown_timeout": "30", 2024-09-17T21:29:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "client_snapdir": ".snap", 2024-09-17T21:29:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "client_tick_interval": "1", 2024-09-17T21:29:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "client_trace": "", 2024-09-17T21:29:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "client_try_dentry_invalidate": "false", 2024-09-17T21:29:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "client_use_faked_inos": "false", 2024-09-17T21:29:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "client_use_random_mds": "false", 2024-09-17T21:29:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "clog_to_graylog": "false", 2024-09-17T21:29:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-17T21:29:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "clog_to_graylog_port": "12201", 2024-09-17T21:29:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "clog_to_monitors": "default=true", 2024-09-17T21:29:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "clog_to_syslog": "false", 2024-09-17T21:29:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-17T21:29:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "clog_to_syslog_level": "info", 2024-09-17T21:29:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "-", 2024-09-17T21:29:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_network": "", 2024-09-17T21:29:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_network_interface": "", 2024-09-17T21:29:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "compressor_zlib_isal": "false", 2024-09-17T21:29:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "compressor_zlib_level": "5", 2024-09-17T21:29:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "compressor_zlib_winsize": "-15", 2024-09-17T21:29:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "compressor_zstd_level": "1", 2024-09-17T21:29:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-17T21:29:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-17T21:29:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "crimson_alien_op_num_threads": "6", 2024-09-17T21:29:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-17T21:29:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-17T21:29:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-17T21:29:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "crimson_osd_stat_interval": "0", 2024-09-17T21:29:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "crimson_seastar_cpu_cores": "", 2024-09-17T21:29:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "crimson_seastar_num_threads": "0", 2024-09-17T21:29:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_location": "", 2024-09-17T21:29:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_location_hook": "", 2024-09-17T21:29:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_location_hook_timeout": "10", 2024-09-17T21:29:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "daemonize": "false", 2024-09-17T21:29:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "daos_pool": "tank", 2024-09-17T21:29:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-17T21:29:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-17T21:29:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-17T21:29:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_alienstore": "0/5", 2024-09-17T21:29:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_allow_any_pool_priority": "false", 2024-09-17T21:29:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_asok": "1/5", 2024-09-17T21:29:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_asok_assert_abort": "false", 2024-09-17T21:29:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_asserts_on_shutdown": "true", 2024-09-17T21:29:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_auth": "20/20", 2024-09-17T21:29:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_bdev": "1/3", 2024-09-17T21:29:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_bluefs": "1/5", 2024-09-17T21:29:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_bluestore": "1/5", 2024-09-17T21:29:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_buffer": "0/1", 2024-09-17T21:29:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_ceph_dedup": "0/5", 2024-09-17T21:29:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_ceph_exporter": "1/5", 2024-09-17T21:29:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_cephfs_mirror": "0/5", 2024-09-17T21:29:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_cephsqlite": "0/5", 2024-09-17T21:29:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_client": "0/5", 2024-09-17T21:29:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_compressor": "1/5", 2024-09-17T21:29:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_context": "0/1", 2024-09-17T21:29:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_crimson_interrupt": "0/5", 2024-09-17T21:29:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_crush": "1/1", 2024-09-17T21:29:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_crypto": "1/5", 2024-09-17T21:29:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_cyanstore": "0/5", 2024-09-17T21:29:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_deliberately_leak_memory": "false", 2024-09-17T21:29:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_disable_randomized_ping": "false", 2024-09-17T21:29:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_dpdk": "1/5", 2024-09-17T21:29:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_eventtrace": "1/5", 2024-09-17T21:29:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_filer": "0/1", 2024-09-17T21:29:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_filestore": "1/3", 2024-09-17T21:29:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_finisher": "1/1", 2024-09-17T21:29:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_fuse": "1/5", 2024-09-17T21:29:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_heartbeat_testing_span": "0", 2024-09-17T21:29:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_heartbeatmap": "1/5", 2024-09-17T21:29:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-17T21:29:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_javaclient": "1/5", 2024-09-17T21:29:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_journal": "1/3", 2024-09-17T21:29:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_journaler": "0/5", 2024-09-17T21:29:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_kstore": "1/5", 2024-09-17T21:29:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_lockdep": "0/1", 2024-09-17T21:29:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mclock": "1/5", 2024-09-17T21:29:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mds": "1/5", 2024-09-17T21:29:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mds_balancer": "1/5", 2024-09-17T21:29:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mds_locker": "1/5", 2024-09-17T21:29:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mds_log": "1/5", 2024-09-17T21:29:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mds_log_expire": "1/5", 2024-09-17T21:29:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mds_migrator": "1/5", 2024-09-17T21:29:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mds_quiesce": "3/5", 2024-09-17T21:29:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_memstore": "1/5", 2024-09-17T21:29:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mgr": "2/5", 2024-09-17T21:29:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mgrc": "1/5", 2024-09-17T21:29:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mon": "20/20", 2024-09-17T21:29:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_monc": "0/10", 2024-09-17T21:29:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_ms": "1/1", 2024-09-17T21:29:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_none": "0/5", 2024-09-17T21:29:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_objclass": "0/5", 2024-09-17T21:29:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_objectcacher": "0/5", 2024-09-17T21:29:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_objecter": "0/1", 2024-09-17T21:29:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_optracker": "0/5", 2024-09-17T21:29:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_osd": "1/5", 2024-09-17T21:29:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_paxos": "20/20", 2024-09-17T21:29:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_perfcounter": "1/5", 2024-09-17T21:29:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_prioritycache": "1/5", 2024-09-17T21:29:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rados": "0/5", 2024-09-17T21:29:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rbd": "0/5", 2024-09-17T21:29:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rbd_mirror": "0/5", 2024-09-17T21:29:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rbd_pwl": "0/5", 2024-09-17T21:29:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rbd_replay": "0/5", 2024-09-17T21:29:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_refs": "0/0", 2024-09-17T21:29:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_reserver": "1/1", 2024-09-17T21:29:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rgw": "1/5", 2024-09-17T21:29:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rgw_access": "1/5", 2024-09-17T21:29:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rgw_datacache": "1/5", 2024-09-17T21:29:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rgw_dbstore": "1/5", 2024-09-17T21:29:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rgw_flight": "1/5", 2024-09-17T21:29:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-17T21:29:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rgw_notification": "1/5", 2024-09-17T21:29:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rgw_sync": "1/5", 2024-09-17T21:29:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rocksdb": "4/5", 2024-09-17T21:29:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore": "0/5", 2024-09-17T21:29:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_backref": "0/5", 2024-09-17T21:29:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_cache": "0/5", 2024-09-17T21:29:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_cleaner": "0/5", 2024-09-17T21:29:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_device": "0/5", 2024-09-17T21:29:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_epm": "0/5", 2024-09-17T21:29:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-17T21:29:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_journal": "0/5", 2024-09-17T21:29:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_lba": "0/5", 2024-09-17T21:29:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_odata": "0/5", 2024-09-17T21:29:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_omap": "0/5", 2024-09-17T21:29:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_onode": "0/5", 2024-09-17T21:29:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_t": "0/5", 2024-09-17T21:29:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_tm": "0/5", 2024-09-17T21:29:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_striper": "0/1", 2024-09-17T21:29:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_test": "0/5", 2024-09-17T21:29:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_throttle": "1/1", 2024-09-17T21:29:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_timer": "0/1", 2024-09-17T21:29:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_tp": "0/5", 2024-09-17T21:29:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_trace": "1/5", 2024-09-17T21:29:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-17T21:29:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: "device_failure_prediction_mode": "none", 2024-09-17T21:29:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-17T21:29:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-17T21:29:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: "err_to_graylog": "false", 2024-09-17T21:29:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: "err_to_journald": "false", 2024-09-17T21:29:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: "err_to_stderr": "true", 2024-09-17T21:29:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: "err_to_syslog": "false", 2024-09-17T21:29:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event_tracing": "false", 2024-09-17T21:29:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: "exporter_addr": "0.0.0.0", 2024-09-17T21:29:39.494 INFO:teuthology.orchestra.run.smithi129.stdout: "exporter_cert_file": "", 2024-09-17T21:29:39.494 INFO:teuthology.orchestra.run.smithi129.stdout: "exporter_http_port": "9926", 2024-09-17T21:29:39.494 INFO:teuthology.orchestra.run.smithi129.stdout: "exporter_key_file": "", 2024-09-17T21:29:39.494 INFO:teuthology.orchestra.run.smithi129.stdout: "exporter_prio_limit": "5", 2024-09-17T21:29:39.494 INFO:teuthology.orchestra.run.smithi129.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-17T21:29:39.494 INFO:teuthology.orchestra.run.smithi129.stdout: "exporter_sort_metrics": "true", 2024-09-17T21:29:39.494 INFO:teuthology.orchestra.run.smithi129.stdout: "exporter_stats_period": "5", 2024-09-17T21:29:39.494 INFO:teuthology.orchestra.run.smithi129.stdout: "fake_statfs_for_testing": "0", 2024-09-17T21:29:39.494 INFO:teuthology.orchestra.run.smithi129.stdout: "fatal_signal_handlers": "true", 2024-09-17T21:29:39.494 INFO:teuthology.orchestra.run.smithi129.stdout: "filer_max_purge_ops": "10", 2024-09-17T21:29:39.494 INFO:teuthology.orchestra.run.smithi129.stdout: "filer_max_truncate_ops": "128", 2024-09-17T21:29:39.494 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_apply_finisher_threads": "1", 2024-09-17T21:29:39.494 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_blackhole": "false", 2024-09-17T21:29:39.494 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_btrfs_clone_range": "true", 2024-09-17T21:29:39.495 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_btrfs_snap": "true", 2024-09-17T21:29:39.495 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_caller_concurrency": "10", 2024-09-17T21:29:39.495 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_collect_device_partition_information": "true", 2024-09-17T21:29:39.495 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_commit_timeout": "600.000000", 2024-09-17T21:29:39.495 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_debug_inject_read_err": "false", 2024-09-17T21:29:39.495 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_debug_omap_check": "false", 2024-09-17T21:29:39.495 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_debug_verify_split": "false", 2024-09-17T21:29:39.495 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_dump_file": "", 2024-09-17T21:29:39.495 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-17T21:29:39.495 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-17T21:29:39.495 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_fadvise": "true", 2024-09-17T21:29:39.495 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_fail_eio": "true", 2024-09-17T21:29:39.495 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_fd_cache_shards": "16", 2024-09-17T21:29:39.495 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_fd_cache_size": "128", 2024-09-17T21:29:39.496 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_fiemap": "false", 2024-09-17T21:29:39.496 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_fiemap_threshold": "4096", 2024-09-17T21:29:39.496 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-17T21:29:39.496 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-17T21:29:39.496 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_inject_stall": "0", 2024-09-17T21:29:39.496 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_journal_parallel": "false", 2024-09-17T21:29:39.496 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_journal_trailing": "false", 2024-09-17T21:29:39.496 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_journal_writeahead": "false", 2024-09-17T21:29:39.496 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_kill_at": "0", 2024-09-17T21:29:39.496 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-17T21:29:39.496 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-17T21:29:39.497 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-17T21:29:39.497 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-17T21:29:39.497 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-17T21:29:39.497 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_inline_xattrs": "0", 2024-09-17T21:29:39.497 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-17T21:29:39.497 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-17T21:29:39.497 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-17T21:29:39.497 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-17T21:29:39.497 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_xattr_value_size": "0", 2024-09-17T21:29:39.497 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-17T21:29:39.497 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-17T21:29:39.497 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-17T21:29:39.498 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_merge_threshold": "-10", 2024-09-17T21:29:39.498 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-17T21:29:39.498 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_odsync_write": "false", 2024-09-17T21:29:39.498 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_omap_backend": "rocksdb", 2024-09-17T21:29:39.498 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_omap_backend_path": "", 2024-09-17T21:29:39.498 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-17T21:29:39.498 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-17T21:29:39.498 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-17T21:29:39.498 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_op_thread_timeout": "60", 2024-09-17T21:29:39.498 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_op_threads": "2", 2024-09-17T21:29:39.498 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_punch_hole": "false", 2024-09-17T21:29:39.498 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-17T21:29:39.498 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-17T21:29:39.498 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-17T21:29:39.499 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-17T21:29:39.499 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-17T21:29:39.499 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-17T21:29:39.499 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-17T21:29:39.499 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-17T21:29:39.499 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-17T21:29:39.499 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_max_ops": "50", 2024-09-17T21:29:39.499 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-17T21:29:39.499 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_seek_data_hole": "false", 2024-09-17T21:29:39.499 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_sloppy_crc": "false", 2024-09-17T21:29:39.499 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-17T21:29:39.499 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_splice": "false", 2024-09-17T21:29:39.499 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_split_multiple": "2", 2024-09-17T21:29:39.499 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_split_rand_factor": "20", 2024-09-17T21:29:39.500 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_update_to": "1000", 2024-09-17T21:29:39.500 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-17T21:29:39.500 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-17T21:29:39.500 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-17T21:29:39.500 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-17T21:29:39.500 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-17T21:29:39.500 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-17T21:29:39.500 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_enable": "true", 2024-09-17T21:29:39.500 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-17T21:29:39.500 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-17T21:29:39.500 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-17T21:29:39.500 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-17T21:29:39.500 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-17T21:29:39.501 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-17T21:29:39.501 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_xfs_extsize": "false", 2024-09-17T21:29:39.501 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_zfs_snap": "false", 2024-09-17T21:29:39.501 INFO:teuthology.orchestra.run.smithi129.stdout: "fio_dir": "/tmp/fio", 2024-09-17T21:29:39.501 INFO:teuthology.orchestra.run.smithi129.stdout: "fsid": "f6d3a80c-8a1f-4716-b25d-e97b28890c0d", 2024-09-17T21:29:39.501 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_allow_other": "true", 2024-09-17T21:29:39.501 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_atomic_o_trunc": "true", 2024-09-17T21:29:39.501 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_big_writes": "true", 2024-09-17T21:29:39.501 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_debug": "false", 2024-09-17T21:29:39.501 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_default_permissions": "false", 2024-09-17T21:29:39.501 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_disable_pagecache": "false", 2024-09-17T21:29:39.501 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_max_write": "0", 2024-09-17T21:29:39.501 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_multithreaded": "true", 2024-09-17T21:29:39.502 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_require_active_mds": "true", 2024-09-17T21:29:39.502 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_set_user_groups": "true", 2024-09-17T21:29:39.502 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_splice_move": "true", 2024-09-17T21:29:39.502 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_splice_read": "true", 2024-09-17T21:29:39.502 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_splice_write": "true", 2024-09-17T21:29:39.502 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-17T21:29:39.502 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_use_invalidate_cb": "true", 2024-09-17T21:29:39.502 INFO:teuthology.orchestra.run.smithi129.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-17T21:29:39.502 INFO:teuthology.orchestra.run.smithi129.stdout: "gss_target_name": "ceph", 2024-09-17T21:29:39.502 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_file": "", 2024-09-17T21:29:39.502 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_inject_failure": "0", 2024-09-17T21:29:39.502 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_interval": "5", 2024-09-17T21:29:39.502 INFO:teuthology.orchestra.run.smithi129.stdout: "host": "smithi129", 2024-09-17T21:29:39.502 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-17T21:29:39.503 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-17T21:29:39.503 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-17T21:29:39.503 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-17T21:29:39.503 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-17T21:29:39.503 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-17T21:29:39.503 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-17T21:29:39.503 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-17T21:29:39.503 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-17T21:29:39.503 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-17T21:29:39.503 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-17T21:29:39.503 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-17T21:29:39.503 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-17T21:29:39.503 INFO:teuthology.orchestra.run.smithi129.stdout: "inject_early_sigterm": "false", 2024-09-17T21:29:39.504 INFO:teuthology.orchestra.run.smithi129.stdout: "jaeger_agent_port": "6799", 2024-09-17T21:29:39.504 INFO:teuthology.orchestra.run.smithi129.stdout: "jaeger_tracing_enable": "false", 2024-09-17T21:29:39.504 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_aio": "true", 2024-09-17T21:29:39.504 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_align_min_size": "65536", 2024-09-17T21:29:39.504 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_block_align": "true", 2024-09-17T21:29:39.504 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_block_size": "4096", 2024-09-17T21:29:39.504 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_dio": "true", 2024-09-17T21:29:39.504 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_discard": "false", 2024-09-17T21:29:39.504 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_force_aio": "false", 2024-09-17T21:29:39.504 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_ignore_corruption": "false", 2024-09-17T21:29:39.504 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_max_write_bytes": "10485760", 2024-09-17T21:29:39.505 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_max_write_entries": "100", 2024-09-17T21:29:39.505 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_replay_from": "0", 2024-09-17T21:29:39.505 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-17T21:29:39.505 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-17T21:29:39.505 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-17T21:29:39.505 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-17T21:29:39.505 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_write_header_frequency": "0", 2024-09-17T21:29:39.505 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_zero_on_create": "false", 2024-09-17T21:29:39.505 INFO:teuthology.orchestra.run.smithi129.stdout: "journaler_prefetch_periods": "10", 2024-09-17T21:29:39.505 INFO:teuthology.orchestra.run.smithi129.stdout: "journaler_prezero_periods": "5", 2024-09-17T21:29:39.505 INFO:teuthology.orchestra.run.smithi129.stdout: "journaler_write_head_interval": "15", 2024-09-17T21:29:39.505 INFO:teuthology.orchestra.run.smithi129.stdout: "key": "", 2024-09-17T21:29:39.506 INFO:teuthology.orchestra.run.smithi129.stdout: "keyfile": "", 2024-09-17T21:29:39.506 INFO:teuthology.orchestra.run.smithi129.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-17T21:29:39.506 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_backend": "rocksdb", 2024-09-17T21:29:39.506 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_default_stripe_size": "65536", 2024-09-17T21:29:39.506 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_fsck_on_mount": "false", 2024-09-17T21:29:39.506 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-17T21:29:39.506 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_max_bytes": "67108864", 2024-09-17T21:29:39.506 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_max_ops": "512", 2024-09-17T21:29:39.506 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_nid_prealloc": "1024", 2024-09-17T21:29:39.506 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_onode_map_size": "1024", 2024-09-17T21:29:39.506 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-17T21:29:39.506 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_sync_submit_transaction": "false", 2024-09-17T21:29:39.507 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_sync_transaction": "false", 2024-09-17T21:29:39.507 INFO:teuthology.orchestra.run.smithi129.stdout: "librados_thread_count": "2", 2024-09-17T21:29:39.507 INFO:teuthology.orchestra.run.smithi129.stdout: "lockdep": "false", 2024-09-17T21:29:39.507 INFO:teuthology.orchestra.run.smithi129.stdout: "lockdep_force_backtrace": "false", 2024-09-17T21:29:39.507 INFO:teuthology.orchestra.run.smithi129.stdout: "log_coarse_timestamps": "true", 2024-09-17T21:29:39.507 INFO:teuthology.orchestra.run.smithi129.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-17T21:29:39.507 INFO:teuthology.orchestra.run.smithi129.stdout: "log_flush_on_exit": "false", 2024-09-17T21:29:39.507 INFO:teuthology.orchestra.run.smithi129.stdout: "log_graylog_host": "127.0.0.1", 2024-09-17T21:29:39.507 INFO:teuthology.orchestra.run.smithi129.stdout: "log_graylog_port": "12201", 2024-09-17T21:29:39.507 INFO:teuthology.orchestra.run.smithi129.stdout: "log_max_new": "1000", 2024-09-17T21:29:39.507 INFO:teuthology.orchestra.run.smithi129.stdout: "log_max_recent": "10000", 2024-09-17T21:29:39.507 INFO:teuthology.orchestra.run.smithi129.stdout: "log_stderr_prefix": "", 2024-09-17T21:29:39.507 INFO:teuthology.orchestra.run.smithi129.stdout: "log_stop_at_utilization": "0.970000", 2024-09-17T21:29:39.507 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": "true", 2024-09-17T21:29:39.508 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_graylog": "false", 2024-09-17T21:29:39.508 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_journald": "false", 2024-09-17T21:29:39.508 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_stderr": "false", 2024-09-17T21:29:39.508 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_syslog": "false", 2024-09-17T21:29:39.508 INFO:teuthology.orchestra.run.smithi129.stdout: "max_rotating_auth_attempts": "10", 2024-09-17T21:29:39.508 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-17T21:29:39.508 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_action_on_write_error": "1", 2024-09-17T21:29:39.508 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_alternate_name_max": "8192", 2024-09-17T21:29:39.508 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_asio_thread_count": "2", 2024-09-17T21:29:39.508 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_export_pin": "true", 2024-09-17T21:29:39.508 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_fragment_dirs": "true", 2024-09-17T21:29:39.508 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-17T21:29:39.508 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_fragment_interval": "5", 2024-09-17T21:29:39.508 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-17T21:29:39.509 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-17T21:29:39.509 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_interval": "10", 2024-09-17T21:29:39.509 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_max": "-1", 2024-09-17T21:29:39.509 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_max_until": "-1", 2024-09-17T21:29:39.509 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_merge_size": "50", 2024-09-17T21:29:39.509 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_midchunk": "0.300000", 2024-09-17T21:29:39.509 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-17T21:29:39.509 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_min_start": "0.200000", 2024-09-17T21:29:39.509 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_minchunk": "0.001000", 2024-09-17T21:29:39.509 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_mode": "0", 2024-09-17T21:29:39.509 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_need_max": "1.200000", 2024-09-17T21:29:39.509 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_need_min": "0.800000", 2024-09-17T21:29:39.509 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_overload_epochs": "2", 2024-09-17T21:29:39.510 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-17T21:29:39.510 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-17T21:29:39.510 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_split_bits": "3", 2024-09-17T21:29:39.510 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-17T21:29:39.510 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_split_size": "10000", 2024-09-17T21:29:39.510 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-17T21:29:39.510 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_target_decay": "10.000000", 2024-09-17T21:29:39.510 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-17T21:29:39.510 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_beacon_grace": "15.000000", 2024-09-17T21:29:39.510 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_beacon_interval": "4.000000", 2024-09-17T21:29:39.510 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-17T21:29:39.510 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-17T21:29:39.510 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_mid": "0.700000", 2024-09-17T21:29:39.511 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-17T21:29:39.511 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_quiesce_delay": "0", 2024-09-17T21:29:39.511 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-17T21:29:39.511 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-17T21:29:39.511 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-17T21:29:39.511 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_release_free_interval": "10", 2024-09-17T21:29:39.511 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_reservation": "0.050000", 2024-09-17T21:29:39.511 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-17T21:29:39.511 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_trim_interval": "1", 2024-09-17T21:29:39.511 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_trim_threshold": "262144", 2024-09-17T21:29:39.511 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-17T21:29:39.511 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-17T21:29:39.511 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-17T21:29:39.512 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_client_prealloc_inos": "1000", 2024-09-17T21:29:39.512 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-17T21:29:39.512 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_connect_bootstrapping": "false", 2024-09-17T21:29:39.512 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_damage_table_max_entries": "10000", 2024-09-17T21:29:39.512 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-17T21:29:39.512 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_debug_auth_pins": "false", 2024-09-17T21:29:39.512 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_debug_frag": "false", 2024-09-17T21:29:39.512 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_debug_scatterstat": "false", 2024-09-17T21:29:39.512 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_debug_subtrees": "false", 2024-09-17T21:29:39.512 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_decay_halflife": "5.000000", 2024-09-17T21:29:39.512 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_default_dir_hash": "2", 2024-09-17T21:29:39.513 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_defer_session_stale": "true", 2024-09-17T21:29:39.513 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_deny_all_reconnect": "false", 2024-09-17T21:29:39.513 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_dir_keys_per_op": "16384", 2024-09-17T21:29:39.513 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_dir_max_commit_size": "10", 2024-09-17T21:29:39.513 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_dir_max_entries": "0", 2024-09-17T21:29:39.513 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_dir_prefetch": "true", 2024-09-17T21:29:39.513 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-17T21:29:39.513 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-17T21:29:39.513 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_dump_cache_on_map": "false", 2024-09-17T21:29:39.513 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-17T21:29:39.513 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-17T21:29:39.513 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_early_reply": "true", 2024-09-17T21:29:39.513 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_enable_op_tracker": "true", 2024-09-17T21:29:39.514 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_enforce_unique_name": "true", 2024-09-17T21:29:39.514 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-17T21:29:39.514 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-17T21:29:39.514 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_export_ephemeral_random": "true", 2024-09-17T21:29:39.514 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-17T21:29:39.514 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-17T21:29:39.514 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-17T21:29:39.514 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-17T21:29:39.514 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-17T21:29:39.514 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-17T21:29:39.514 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-17T21:29:39.515 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-17T21:29:39.515 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_health_summarize_threshold": "10", 2024-09-17T21:29:39.515 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-17T21:29:39.515 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-17T21:29:39.515 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_inject_health_dummy": "false", 2024-09-17T21:29:39.515 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-17T21:29:39.515 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_inject_migrator_session_race": "false", 2024-09-17T21:29:39.515 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-17T21:29:39.515 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-17T21:29:39.515 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-17T21:29:39.515 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_join_fs": "", 2024-09-17T21:29:39.515 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_journal_format": "1", 2024-09-17T21:29:39.515 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-17T21:29:39.515 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_create_at": "0", 2024-09-17T21:29:39.516 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_dirfrag_at": "0", 2024-09-17T21:29:39.516 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_export_at": "0", 2024-09-17T21:29:39.516 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_import_at": "0", 2024-09-17T21:29:39.516 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_journal_at": "0", 2024-09-17T21:29:39.516 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_journal_expire_at": "0", 2024-09-17T21:29:39.516 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_journal_replay_at": "0", 2024-09-17T21:29:39.516 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_link_at": "0", 2024-09-17T21:29:39.516 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_mdstable_at": "0", 2024-09-17T21:29:39.516 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_openc_at": "0", 2024-09-17T21:29:39.516 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_rename_at": "0", 2024-09-17T21:29:39.516 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_shutdown_at": "0", 2024-09-17T21:29:39.516 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_event_large_threshold": "524288", 2024-09-17T21:29:39.516 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_events_per_segment": "1024", 2024-09-17T21:29:39.516 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-17T21:29:39.517 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_max_events": "-1", 2024-09-17T21:29:39.517 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_max_segments": "128", 2024-09-17T21:29:39.517 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_pause": "false", 2024-09-17T21:29:39.517 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_segment_size": "0", 2024-09-17T21:29:39.517 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-17T21:29:39.517 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-17T21:29:39.517 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-17T21:29:39.517 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_trim_threshold": "128", 2024-09-17T21:29:39.517 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-17T21:29:39.517 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_warn_factor": "2.000000", 2024-09-17T21:29:39.517 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_caps_per_client": "1048576", 2024-09-17T21:29:39.517 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_completed_flushes": "100000", 2024-09-17T21:29:39.517 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_completed_requests": "100000", 2024-09-17T21:29:39.518 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_export_size": "20971520", 2024-09-17T21:29:39.518 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_file_recover": "32", 2024-09-17T21:29:39.518 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_purge_files": "64", 2024-09-17T21:29:39.518 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_purge_ops": "8192", 2024-09-17T21:29:39.518 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-17T21:29:39.518 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-17T21:29:39.518 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_snaps_per_dir": "100", 2024-09-17T21:29:39.518 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_metrics_update_interval": "2", 2024-09-17T21:29:39.518 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_min_caps_per_client": "100", 2024-09-17T21:29:39.518 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_min_caps_working_set": "10000", 2024-09-17T21:29:39.518 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-17T21:29:39.519 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_numa_node": "-1", 2024-09-17T21:29:39.519 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-17T21:29:39.519 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_op_complaint_time": "30.000000", 2024-09-17T21:29:39.519 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_op_history_duration": "600", 2024-09-17T21:29:39.519 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_op_history_size": "20", 2024-09-17T21:29:39.519 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_op_history_slow_op_size": "20", 2024-09-17T21:29:39.519 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-17T21:29:39.519 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_op_log_threshold": "5", 2024-09-17T21:29:39.519 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_ping_grace": "15", 2024-09-17T21:29:39.519 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_ping_interval": "5", 2024-09-17T21:29:39.519 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-17T21:29:39.519 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-17T21:29:39.519 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_recall_max_caps": "30000", 2024-09-17T21:29:39.520 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-17T21:29:39.520 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-17T21:29:39.520 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-17T21:29:39.520 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_recall_warning_threshold": "262144", 2024-09-17T21:29:39.520 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-17T21:29:39.520 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_replay_interval": "1.000000", 2024-09-17T21:29:39.520 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-17T21:29:39.520 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-17T21:29:39.520 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_root_ino_gid": "0", 2024-09-17T21:29:39.520 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_root_ino_uid": "0", 2024-09-17T21:29:39.520 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-17T21:29:39.520 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_scrub_stats_review_period": "1", 2024-09-17T21:29:39.520 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-17T21:29:39.520 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-17T21:29:39.521 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-17T21:29:39.521 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-17T21:29:39.521 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-17T21:29:39.521 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-17T21:29:39.521 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-17T21:29:39.521 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-17T21:29:39.521 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-17T21:29:39.521 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-17T21:29:39.521 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-17T21:29:39.521 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_shutdown_check": "0", 2024-09-17T21:29:39.521 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_skip_ino": "0", 2024-09-17T21:29:39.521 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-17T21:29:39.522 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_snap_max_uid": "4294967294", 2024-09-17T21:29:39.522 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_snap_min_uid": "0", 2024-09-17T21:29:39.522 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_snap_rstat": "false", 2024-09-17T21:29:39.522 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_standby_replay_damaged": "false", 2024-09-17T21:29:39.522 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_symlink_recovery": "true", 2024-09-17T21:29:39.522 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-17T21:29:39.522 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_thrash_exports": "0", 2024-09-17T21:29:39.522 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_thrash_fragments": "0", 2024-09-17T21:29:39.522 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_tick_interval": "5.000000", 2024-09-17T21:29:39.522 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_valgrind_exit": "false", 2024-09-17T21:29:39.522 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_verify_backtrace": "1", 2024-09-17T21:29:39.522 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_verify_scatter": "false", 2024-09-17T21:29:39.522 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-17T21:29:39.522 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_wipe_sessions": "false", 2024-09-17T21:29:39.523 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool_debug": "false", 2024-09-17T21:29:39.523 INFO:teuthology.orchestra.run.smithi129.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-17T21:29:39.523 INFO:teuthology.orchestra.run.smithi129.stdout: "memstore_device_bytes": "1073741824", 2024-09-17T21:29:39.523 INFO:teuthology.orchestra.run.smithi129.stdout: "memstore_page_set": "false", 2024-09-17T21:29:39.523 INFO:teuthology.orchestra.run.smithi129.stdout: "memstore_page_size": "65536", 2024-09-17T21:29:39.523 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_client_bytes": "134217728", 2024-09-17T21:29:39.523 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_client_messages": "512", 2024-09-17T21:29:39.523 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-17T21:29:39.523 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-17T21:29:39.523 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-17T21:29:39.523 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-17T21:29:39.523 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_disabled_modules": "", 2024-09-17T21:29:39.523 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_enable_op_tracker": "true", 2024-09-17T21:29:39.524 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-17T21:29:39.524 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_max_pg_creating": "1024", 2024-09-17T21:29:39.524 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_max_pg_num_change": "128", 2024-09-17T21:29:39.524 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_mds_bytes": "134217728", 2024-09-17T21:29:39.524 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_mds_messages": "128", 2024-09-17T21:29:39.524 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-17T21:29:39.524 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_mon_bytes": "134217728", 2024-09-17T21:29:39.524 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_mon_messages": "128", 2024-09-17T21:29:39.524 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-17T21:29:39.524 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-17T21:29:39.524 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_op_history_duration": "600", 2024-09-17T21:29:39.524 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_op_history_size": "20", 2024-09-17T21:29:39.525 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-17T21:29:39.525 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-17T21:29:39.525 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_op_log_threshold": "5", 2024-09-17T21:29:39.525 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_osd_bytes": "536870912", 2024-09-17T21:29:39.525 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_osd_messages": "8192", 2024-09-17T21:29:39.525 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_pool": "true", 2024-09-17T21:29:39.525 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-17T21:29:39.525 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_standby_modules": "true", 2024-09-17T21:29:39.525 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_stats_period": "5", 2024-09-17T21:29:39.525 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_stats_threshold": "5", 2024-09-17T21:29:39.525 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_tick_period": "2", 2024-09-17T21:29:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-17T21:29:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-17T21:29:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_allow_pool_delete": "true", 2024-09-17T21:29:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_allow_pool_size_one": "true", 2024-09-17T21:29:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_auth_validate_all_caps": "true", 2024-09-17T21:29:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-17T21:29:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-17T21:29:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_bytes": "104857600", 2024-09-17T21:29:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_directed_command_retry": "5", 2024-09-17T21:29:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-17T21:29:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-17T21:29:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-17T21:29:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-17T21:29:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_hunt_parallel": "3", 2024-09-17T21:29:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_log_interval": "1.000000", 2024-09-17T21:29:39.527 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-17T21:29:39.527 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_ping_interval": "10.000000", 2024-09-17T21:29:39.527 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-17T21:29:39.527 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_target_rank": "-1", 2024-09-17T21:29:39.527 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-17T21:29:39.527 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-17T21:29:39.527 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-17T21:29:39.527 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_level": "debug", 2024-09-17T21:29:39.527 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_to_file": "true", 2024-09-17T21:29:39.527 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-17T21:29:39.527 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-17T21:29:39.527 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-17T21:29:39.527 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_to_journald": "false", 2024-09-17T21:29:39.527 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-17T21:29:39.528 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-17T21:29:39.528 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-17T21:29:39.528 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_compact_on_bootstrap": "false", 2024-09-17T21:29:39.528 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_compact_on_start": "false", 2024-09-17T21:29:39.528 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_compact_on_trim": "true", 2024-09-17T21:29:39.528 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-17T21:29:39.528 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-17T21:29:39.528 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-17T21:29:39.528 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cpu_threads": "4", 2024-09-17T21:29:39.528 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_crush_min_required_version": "hammer", 2024-09-17T21:29:39.528 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_daemon_bytes": "419430400", 2024-09-17T21:29:39.528 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-17T21:29:39.528 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_data_avail_crit": "5", 2024-09-17T21:29:39.529 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_data_avail_warn": "5", 2024-09-17T21:29:39.529 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_data_size_warn": "16106127360", 2024-09-17T21:29:39.529 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-17T21:29:39.529 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-17T21:29:39.529 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_dump_json": "false", 2024-09-17T21:29:39.529 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-17T21:29:39.529 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_dump_transactions": "false", 2024-09-17T21:29:39.529 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_extra_checks": "false", 2024-09-17T21:29:39.529 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-17T21:29:39.529 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-17T21:29:39.529 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_no_require_reef": "false", 2024-09-17T21:29:39.529 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_no_require_squid": "false", 2024-09-17T21:29:39.529 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-17T21:29:39.530 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-17T21:29:39.530 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-17T21:29:39.530 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_down_mkfs_grace": "120", 2024-09-17T21:29:39.530 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_down_uptime_grace": "60", 2024-09-17T21:29:39.530 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_election_default_strategy": "1", 2024-09-17T21:29:39.530 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_election_timeout": "5.000000", 2024-09-17T21:29:39.530 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-17T21:29:39.530 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_elector_ping_divisor": "2", 2024-09-17T21:29:39.530 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-17T21:29:39.530 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_enable_op_tracker": "true", 2024-09-17T21:29:39.530 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_fake_pool_delete": "false", 2024-09-17T21:29:39.530 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_force_quorum_join": "false", 2024-09-17T21:29:39.531 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-17T21:29:39.531 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_globalid_prealloc": "10000", 2024-09-17T21:29:39.531 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_health_detail_to_clog": "false", 2024-09-17T21:29:39.531 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_health_log_update_period": "5", 2024-09-17T21:29:39.531 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_health_max_detail": "50", 2024-09-17T21:29:39.531 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_health_to_clog": "true", 2024-09-17T21:29:39.531 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_health_to_clog_interval": "600", 2024-09-17T21:29:39.531 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-17T21:29:39.531 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_host": "172.21.15.129,[v2:172.21.15.129:3301,v1:172.21.15.129:6790],172.21.15.171", 2024-09-17T21:29:39.532 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_host_override": "", 2024-09-17T21:29:39.532 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_initial_members": "", 2024-09-17T21:29:39.532 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-17T21:29:39.532 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-17T21:29:39.532 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-17T21:29:39.532 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-17T21:29:39.532 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-17T21:29:39.532 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_lease": "5.000000", 2024-09-17T21:29:39.532 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-17T21:29:39.533 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-17T21:29:39.533 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_log_full_interval": "50", 2024-09-17T21:29:39.533 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_log_max": "10000", 2024-09-17T21:29:39.533 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_log_max_summary": "50", 2024-09-17T21:29:39.533 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-17T21:29:39.533 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_max_log_epochs": "500", 2024-09-17T21:29:39.533 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-17T21:29:39.533 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-17T21:29:39.533 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-17T21:29:39.534 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_max_osd": "10000", 2024-09-17T21:29:39.534 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_max_pg_per_osd": "10000", 2024-09-17T21:29:39.534 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_max_pool_pg_num": "65536", 2024-09-17T21:29:39.534 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-17T21:29:39.534 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-17T21:29:39.534 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_mds_force_trim_to": "0", 2024-09-17T21:29:39.534 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_mds_skip_sanity": "false", 2024-09-17T21:29:39.534 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_memory_autotune": "true", 2024-09-17T21:29:39.534 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_memory_target": "2147483648", 2024-09-17T21:29:39.535 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_mgr_beacon_grace": "30", 2024-09-17T21:29:39.535 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-17T21:29:39.535 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_mgr_digest_period": "5", 2024-09-17T21:29:39.535 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_mgr_inactive_grace": "60", 2024-09-17T21:29:39.535 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-17T21:29:39.535 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-17T21:29:39.535 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_min_osdmap_epochs": "500", 2024-09-17T21:29:39.535 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-17T21:29:39.535 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-17T21:29:39.536 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_op_complaint_time": "30", 2024-09-17T21:29:39.536 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_op_history_duration": "600", 2024-09-17T21:29:39.536 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_op_history_size": "20", 2024-09-17T21:29:39.536 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_op_history_slow_op_size": "20", 2024-09-17T21:29:39.536 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-17T21:29:39.536 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_op_log_threshold": "5", 2024-09-17T21:29:39.536 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-17T21:29:39.536 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-17T21:29:39.537 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-17T21:29:39.537 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_auto_mark_in": "false", 2024-09-17T21:29:39.537 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-17T21:29:39.537 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-17T21:29:39.537 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-17T21:29:39.537 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_cache_size": "500", 2024-09-17T21:29:39.537 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-17T21:29:39.537 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-17T21:29:39.537 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-17T21:29:39.538 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_down_out_interval": "600", 2024-09-17T21:29:39.538 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-17T21:29:39.538 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-17T21:29:39.538 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_force_trim_to": "0", 2024-09-17T21:29:39.538 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-17T21:29:39.538 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-17T21:29:39.538 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-17T21:29:39.538 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-17T21:29:39.538 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-17T21:29:39.539 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-17T21:29:39.539 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-17T21:29:39.539 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_min_down_reporters": "2", 2024-09-17T21:29:39.539 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-17T21:29:39.539 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-17T21:29:39.539 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-17T21:29:39.539 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-17T21:29:39.539 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-17T21:29:39.539 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-17T21:29:39.539 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_report_timeout": "900", 2024-09-17T21:29:39.540 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-17T21:29:39.540 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-17T21:29:39.540 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-17T21:29:39.540 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-17T21:29:39.540 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-17T21:29:39.540 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-17T21:29:39.540 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-17T21:29:39.540 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-17T21:29:39.540 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-17T21:29:39.541 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_pg_stuck_threshold": "60", 2024-09-17T21:29:39.541 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-17T21:29:39.541 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-17T21:29:39.541 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-17T21:29:39.541 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-17T21:29:39.541 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-17T21:29:39.541 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-17T21:29:39.542 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_probe_timeout": "2.000000", 2024-09-17T21:29:39.542 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_reweight_max_change": "0.050000", 2024-09-17T21:29:39.542 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_reweight_max_osds": "4", 2024-09-17T21:29:39.542 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-17T21:29:39.542 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-17T21:29:39.542 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-17T21:29:39.542 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-17T21:29:39.542 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-17T21:29:39.542 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_scrub_interval": "86400", 2024-09-17T21:29:39.543 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_scrub_max_keys": "100", 2024-09-17T21:29:39.543 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_scrub_timeout": "300", 2024-09-17T21:29:39.543 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_session_timeout": "300", 2024-09-17T21:29:39.543 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_smart_report_timeout": "5", 2024-09-17T21:29:39.543 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_stat_smooth_intervals": "6", 2024-09-17T21:29:39.543 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-17T21:29:39.543 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_stretch_pool_min_size": "2", 2024-09-17T21:29:39.543 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_stretch_pool_size": "4", 2024-09-17T21:29:39.543 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-17T21:29:39.543 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-17T21:29:39.544 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_sync_debug": "false", 2024-09-17T21:29:39.544 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-17T21:29:39.544 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-17T21:29:39.544 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_sync_provider_kill_at": "0", 2024-09-17T21:29:39.544 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_sync_requester_kill_at": "0", 2024-09-17T21:29:39.544 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_sync_timeout": "60.000000", 2024-09-17T21:29:39.544 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_target_pg_per_osd": "100", 2024-09-17T21:29:39.544 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_tick_interval": "5", 2024-09-17T21:29:39.544 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_timecheck_interval": "300.000000", 2024-09-17T21:29:39.544 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-17T21:29:39.544 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_use_min_delay_socket": "false", 2024-09-17T21:29:39.544 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_older_version_delay": "604800", 2024-09-17T21:29:39.544 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-17T21:29:39.544 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-17T21:29:39.545 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-17T21:29:39.545 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-17T21:29:39.545 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-17T21:29:39.545 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-17T21:29:39.545 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-17T21:29:39.545 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_misplaced": "false", 2024-09-17T21:29:39.545 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-17T21:29:39.545 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_older_version": "true", 2024-09-17T21:29:39.545 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-17T21:29:39.545 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-17T21:29:39.545 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-17T21:29:39.545 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-17T21:29:39.545 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-17T21:29:39.545 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-17T21:29:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-17T21:29:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-17T21:29:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-17T21:29:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: "monmap": "", 2024-09-17T21:29:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-17T21:29:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-17T21:29:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-17T21:29:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-17T21:29:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-17T21:29:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-17T21:29:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: "motr_tracing_enabled": "false", 2024-09-17T21:29:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_op_threads": "3", 2024-09-17T21:29:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-17T21:29:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_cm": "false", 2024-09-17T21:29:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_device_name": "", 2024-09-17T21:29:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_dscp": "96", 2024-09-17T21:29:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-17T21:29:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-17T21:29:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_local_gid": "", 2024-09-17T21:29:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-17T21:29:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_port_num": "1", 2024-09-17T21:29:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-17T21:29:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-17T21:29:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-17T21:29:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-17T21:29:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_sl": "3", 2024-09-17T21:29:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_support_srq": "true", 2024-09-17T21:29:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_type": "ib", 2024-09-17T21:29:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_reap_threshold": "5", 2024-09-17T21:29:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_before_connect": "false", 2024-09-17T21:29:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_ipv4": "true", 2024-09-17T21:29:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_ipv6": "false", 2024-09-17T21:29:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_msgr1": "true", 2024-09-17T21:29:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_msgr2": "true", 2024-09-17T21:29:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_port_max": "7568", 2024-09-17T21:29:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_port_min": "6800", 2024-09-17T21:29:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-17T21:29:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_retry_count": "3", 2024-09-17T21:29:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_retry_delay": "5", 2024-09-17T21:29:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_blackhole_client": "false", 2024-09-17T21:29:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_blackhole_mds": "false", 2024-09-17T21:29:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_blackhole_mgr": "false", 2024-09-17T21:29:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_blackhole_mon": "false", 2024-09-17T21:29:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_blackhole_osd": "false", 2024-09-17T21:29:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_client_mode": "crc secure", 2024-09-17T21:29:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-17T21:29:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_cluster_mode": "crc secure", 2024-09-17T21:29:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_cluster_type": "", 2024-09-17T21:29:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_compress_secure": "false", 2024-09-17T21:29:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_connection_idle_timeout": "900", 2024-09-17T21:29:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_connection_ready_timeout": "10", 2024-09-17T21:29:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_crc_data": "true", 2024-09-17T21:29:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_crc_header": "true", 2024-09-17T21:29:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_die_on_bad_msg": "false", 2024-09-17T21:29:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_die_on_bug": "true", 2024-09-17T21:29:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_die_on_old_message": "true", 2024-09-17T21:29:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_die_on_skipped_message": "false", 2024-09-17T21:29:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-17T21:29:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-17T21:29:39.550 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_coremask": "0xF", 2024-09-17T21:29:39.550 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-17T21:29:39.550 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-17T21:29:39.550 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_enable_tso": "true", 2024-09-17T21:29:39.550 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-17T21:29:39.550 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-17T21:29:39.550 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_hugepages": "", 2024-09-17T21:29:39.550 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-17T21:29:39.550 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-17T21:29:39.550 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_lro": "true", 2024-09-17T21:29:39.550 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_memory_channel": "4", 2024-09-17T21:29:39.550 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-17T21:29:39.550 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_pmd": "", 2024-09-17T21:29:39.550 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_port_id": "0", 2024-09-17T21:29:39.550 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-17T21:29:39.551 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-17T21:29:39.551 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dump_on_send": "false", 2024-09-17T21:29:39.551 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_initial_backoff": "0.200000", 2024-09-17T21:29:39.551 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_inject_delay_max": "1.000000", 2024-09-17T21:29:39.551 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-17T21:29:39.551 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_inject_delay_type": "", 2024-09-17T21:29:39.551 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-17T21:29:39.551 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_inject_network_congestion": "0", 2024-09-17T21:29:39.551 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_inject_socket_failures": "5000", 2024-09-17T21:29:39.551 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_learn_addr_from_peer": "true", 2024-09-17T21:29:39.551 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_max_accept_failures": "4", 2024-09-17T21:29:39.551 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_max_backoff": "15.000000", 2024-09-17T21:29:39.551 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_mon_client_mode": "secure crc", 2024-09-17T21:29:39.551 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-17T21:29:39.552 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_mon_service_mode": "secure crc", 2024-09-17T21:29:39.552 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_osd_compress_min_size": "1024", 2024-09-17T21:29:39.552 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_osd_compress_mode": "none", 2024-09-17T21:29:39.552 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-17T21:29:39.552 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-17T21:29:39.552 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_pq_min_cost": "65536", 2024-09-17T21:29:39.552 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_public_type": "", 2024-09-17T21:29:39.552 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_service_mode": "crc secure", 2024-09-17T21:29:39.552 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_tcp_listen_backlog": "512", 2024-09-17T21:29:39.552 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_tcp_nodelay": "true", 2024-09-17T21:29:39.552 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-17T21:29:39.552 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_tcp_rcvbuf": "0", 2024-09-17T21:29:39.552 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-17T21:29:39.552 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_type": "async+posix", 2024-09-17T21:29:39.552 INFO:teuthology.orchestra.run.smithi129.stdout: "no_config_file": "false", 2024-09-17T21:29:39.553 INFO:teuthology.orchestra.run.smithi129.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-17T21:29:39.553 INFO:teuthology.orchestra.run.smithi129.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-17T21:29:39.553 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter_completion_locks_per_session": "32", 2024-09-17T21:29:39.553 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-17T21:29:39.553 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-17T21:29:39.553 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter_inflight_ops": "1024", 2024-09-17T21:29:39.553 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-17T21:29:39.553 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-17T21:29:39.553 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter_tick_interval": "5.000000", 2024-09-17T21:29:39.553 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter_timeout": "10.000000", 2024-09-17T21:29:39.553 INFO:teuthology.orchestra.run.smithi129.stdout: "objectstore_blackhole": "false", 2024-09-17T21:29:39.553 INFO:teuthology.orchestra.run.smithi129.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-17T21:29:39.553 INFO:teuthology.orchestra.run.smithi129.stdout: "openssl_engine_opts": "", 2024-09-17T21:29:39.553 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_agent_delay_time": "5.000000", 2024-09-17T21:29:39.553 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_agent_hist_halflife": "1000", 2024-09-17T21:29:39.554 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_agent_max_low_ops": "2", 2024-09-17T21:29:39.554 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_agent_max_ops": "4", 2024-09-17T21:29:39.554 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-17T21:29:39.554 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-17T21:29:39.554 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_agent_slop": "0.020000", 2024-09-17T21:29:39.554 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-17T21:29:39.554 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-17T21:29:39.554 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_asio_thread_count": "2", 2024-09-17T21:29:39.554 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_async_recovery_min_cost": "100", 2024-09-17T21:29:39.554 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-17T21:29:39.554 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-17T21:29:39.554 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_backfill_scan_max": "512", 2024-09-17T21:29:39.554 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_backfill_scan_min": "64", 2024-09-17T21:29:39.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_backoff_on_degraded": "false", 2024-09-17T21:29:39.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_backoff_on_peering": "false", 2024-09-17T21:29:39.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_backoff_on_unfound": "true", 2024-09-17T21:29:39.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_beacon_report_interval": "300", 2024-09-17T21:29:39.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bench_duration": "30", 2024-09-17T21:29:39.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-17T21:29:39.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bench_max_block_size": "67108864", 2024-09-17T21:29:39.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-17T21:29:39.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_blkin_trace_all": "false", 2024-09-17T21:29:39.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-17T21:29:39.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-17T21:29:39.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-17T21:29:39.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-17T21:29:39.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_check_for_log_corruption": "false", 2024-09-17T21:29:39.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-17T21:29:39.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-17T21:29:39.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-17T21:29:39.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-17T21:29:39.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_class_update_on_start": "true", 2024-09-17T21:29:39.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_client_message_cap": "256", 2024-09-17T21:29:39.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_client_message_size_cap": "524288000", 2024-09-17T21:29:39.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_client_op_priority": "63", 2024-09-17T21:29:39.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_client_watch_timeout": "30", 2024-09-17T21:29:39.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_command_max_records": "256", 2024-09-17T21:29:39.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_compact_on_start": "false", 2024-09-17T21:29:39.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-17T21:29:39.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-17T21:29:39.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-17T21:29:39.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_crush_update_on_start": "true", 2024-09-17T21:29:39.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_crush_update_weight_set": "true", 2024-09-17T21:29:39.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-17T21:29:39.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-17T21:29:39.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-17T21:29:39.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-17T21:29:39.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-17T21:29:39.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_feed_pullee": "-1", 2024-09-17T21:29:39.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-17T21:29:39.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-17T21:29:39.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-17T21:29:39.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_misdirected_ops": "false", 2024-09-17T21:29:39.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_no_acting_change": "false", 2024-09-17T21:29:39.558 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_no_purge_strays": "false", 2024-09-17T21:29:39.558 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_op_order": "false", 2024-09-17T21:29:39.558 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-17T21:29:39.558 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-17T21:29:39.558 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-17T21:29:39.558 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-17T21:29:39.558 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_shutdown": "false", 2024-09-17T21:29:39.558 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-17T21:29:39.558 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-17T21:29:39.558 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_trim_objects": "false", 2024-09-17T21:29:39.558 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-17T21:29:39.558 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-17T21:29:39.558 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_verify_snaps": "false", 2024-09-17T21:29:39.558 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-17T21:29:39.558 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-17T21:29:39.559 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_deep_scrub_keys": "1024", 2024-09-17T21:29:39.559 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-17T21:29:39.559 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-17T21:29:39.559 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-17T21:29:39.559 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_deep_scrub_stride": "524288", 2024-09-17T21:29:39.559 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-17T21:29:39.559 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-17T21:29:39.559 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_default_notify_timeout": "30", 2024-09-17T21:29:39.559 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_delete_sleep": "0.000000", 2024-09-17T21:29:39.559 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-17T21:29:39.559 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-17T21:29:39.559 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-17T21:29:39.559 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_discard_disconnected_ops": "true", 2024-09-17T21:29:39.559 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_ec_partial_reads": "true", 2024-09-17T21:29:39.560 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_enable_op_tracker": "true", 2024-09-17T21:29:39.560 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-17T21:29:39.560 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-17T21:29:39.560 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-17T21:29:39.560 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-17T21:29:39.560 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_fast_info": "true", 2024-09-17T21:29:39.560 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_fast_shutdown": "true", 2024-09-17T21:29:39.560 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-17T21:29:39.560 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-17T21:29:39.560 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-17T21:29:39.560 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-17T21:29:39.560 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-17T21:29:39.560 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_function_tracing": "false", 2024-09-17T21:29:39.560 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_heartbeat_grace": "20", 2024-09-17T21:29:39.561 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_heartbeat_interval": "6", 2024-09-17T21:29:39.561 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-17T21:29:39.561 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_heartbeat_min_peers": "10", 2024-09-17T21:29:39.561 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_heartbeat_min_size": "2000", 2024-09-17T21:29:39.561 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_heartbeat_stale": "600", 2024-09-17T21:29:39.561 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-17T21:29:39.561 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_hit_set_max_size": "100000", 2024-09-17T21:29:39.561 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_hit_set_min_size": "1000", 2024-09-17T21:29:39.561 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-17T21:29:39.561 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-17T21:29:39.561 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-17T21:29:39.561 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-17T21:29:39.561 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-17T21:29:39.561 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-17T21:29:39.561 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_journal_size": "5120", 2024-09-17T21:29:39.562 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_kill_backfill_at": "0", 2024-09-17T21:29:39.562 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-17T21:29:39.562 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_map_cache_size": "50", 2024-09-17T21:29:39.562 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_map_dedup": "true", 2024-09-17T21:29:39.562 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_map_message_max": "40", 2024-09-17T21:29:39.562 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-17T21:29:39.562 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_map_share_max_epochs": "40", 2024-09-17T21:29:39.562 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_attr_name_len": "100", 2024-09-17T21:29:39.562 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_attr_size": "0", 2024-09-17T21:29:39.562 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_backfills": "1", 2024-09-17T21:29:39.562 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_markdown_count": "5", 2024-09-17T21:29:39.562 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_markdown_period": "600", 2024-09-17T21:29:39.562 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_object_name_len": "2048", 2024-09-17T21:29:39.562 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_object_namespace_len": "256", 2024-09-17T21:29:39.562 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_object_size": "134217728", 2024-09-17T21:29:39.563 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-17T21:29:39.563 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-17T21:29:39.563 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_pg_blocked_by": "16", 2024-09-17T21:29:39.563 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_pg_log_entries": "10000", 2024-09-17T21:29:39.563 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-17T21:29:39.563 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_pgls": "1024", 2024-09-17T21:29:39.563 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_push_cost": "8388608", 2024-09-17T21:29:39.563 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_push_objects": "10", 2024-09-17T21:29:39.563 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_scrubs": "3", 2024-09-17T21:29:39.563 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-17T21:29:39.563 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_trimming_pgs": "2", 2024-09-17T21:29:39.563 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_write_op_reply_len": "64", 2024-09-17T21:29:39.563 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_write_size": "90", 2024-09-17T21:29:39.563 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-17T21:29:39.563 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-17T21:29:39.563 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-17T21:29:39.564 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-17T21:29:39.564 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-17T21:29:39.564 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-17T21:29:39.564 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-17T21:29:39.564 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-17T21:29:39.564 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_profile": "balanced", 2024-09-17T21:29:39.564 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-17T21:29:39.564 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-17T21:29:39.564 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-17T21:29:39.564 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-17T21:29:39.564 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-17T21:29:39.564 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-17T21:29:39.564 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-17T21:29:39.564 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-17T21:29:39.564 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-17T21:29:39.565 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-17T21:29:39.565 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-17T21:29:39.565 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_memory_base": "805306368", 2024-09-17T21:29:39.565 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_memory_cache_min": "134217728", 2024-09-17T21:29:39.565 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-17T21:29:39.565 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-17T21:29:39.565 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_memory_target": "4294967296", 2024-09-17T21:29:39.565 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_memory_target_autotune": "false", 2024-09-17T21:29:39.565 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-17T21:29:39.565 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_min_pg_log_entries": "250", 2024-09-17T21:29:39.565 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_min_recovery_priority": "0", 2024-09-17T21:29:39.565 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-17T21:29:39.565 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-17T21:29:39.565 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mon_report_interval": "5", 2024-09-17T21:29:39.565 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-17T21:29:39.566 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-17T21:29:39.566 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_num_cache_shards": "32", 2024-09-17T21:29:39.566 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_num_op_tracker_shard": "32", 2024-09-17T21:29:39.566 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_numa_auto_affinity": "true", 2024-09-17T21:29:39.566 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_numa_node": "-1", 2024-09-17T21:29:39.566 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_numa_prefer_iface": "true", 2024-09-17T21:29:39.566 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-17T21:29:39.566 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_objecter_finishers": "1", 2024-09-17T21:29:39.566 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_objectstore": "bluestore", 2024-09-17T21:29:39.566 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_objectstore_fuse": "false", 2024-09-17T21:29:39.566 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-17T21:29:39.566 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_objectstore_tracing": "false", 2024-09-17T21:29:39.566 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_complaint_time": "30.000000", 2024-09-17T21:29:39.566 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_history_duration": "600", 2024-09-17T21:29:39.566 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_history_size": "20", 2024-09-17T21:29:39.567 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_history_slow_op_size": "20", 2024-09-17T21:29:39.567 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-17T21:29:39.567 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_log_threshold": "5", 2024-09-17T21:29:39.567 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_num_shards": "0", 2024-09-17T21:29:39.567 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_num_shards_hdd": "5", 2024-09-17T21:29:39.567 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_num_shards_ssd": "8", 2024-09-17T21:29:39.567 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-17T21:29:39.567 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-17T21:29:39.567 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-17T21:29:39.567 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-17T21:29:39.567 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_pq_min_cost": "65536", 2024-09-17T21:29:39.567 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-17T21:29:39.567 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_queue_cut_off": "high", 2024-09-17T21:29:39.568 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-17T21:29:39.568 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_thread_timeout": "15", 2024-09-17T21:29:39.568 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_open_classes_on_start": "true", 2024-09-17T21:29:39.568 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_os_flags": "0", 2024-09-17T21:29:39.568 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_peering_op_priority": "255", 2024-09-17T21:29:39.568 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_delete_cost": "1048576", 2024-09-17T21:29:39.568 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_delete_priority": "5", 2024-09-17T21:29:39.568 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-17T21:29:39.568 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-17T21:29:39.568 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-17T21:29:39.568 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_log_trim_max": "10000", 2024-09-17T21:29:39.568 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_log_trim_min": "100", 2024-09-17T21:29:39.568 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-17T21:29:39.568 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-17T21:29:39.568 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-17T21:29:39.569 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-17T21:29:39.569 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-17T21:29:39.569 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-17T21:29:39.569 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-17T21:29:39.569 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-17T21:29:39.569 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-17T21:29:39.569 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-17T21:29:39.569 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_crimson": "false", 2024-09-17T21:29:39.569 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-17T21:29:39.569 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-17T21:29:39.569 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-17T21:29:39.569 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-17T21:29:39.569 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-17T21:29:39.569 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-17T21:29:39.569 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-17T21:29:39.570 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-17T21:29:39.570 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_flags": "0", 2024-09-17T21:29:39.570 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-17T21:29:39.570 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_min_size": "0", 2024-09-17T21:29:39.570 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-17T21:29:39.570 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_pg_num": "32", 2024-09-17T21:29:39.570 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_pgp_num": "0", 2024-09-17T21:29:39.570 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-17T21:29:39.570 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_read_ratio": "70", 2024-09-17T21:29:39.570 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_size": "2", 2024-09-17T21:29:39.570 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_type": "replicated", 2024-09-17T21:29:39.570 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-17T21:29:39.570 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-17T21:29:39.570 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_push_per_object_cost": "1000", 2024-09-17T21:29:39.570 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-17T21:29:39.570 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recover_clone_overlap": "true", 2024-09-17T21:29:39.571 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-17T21:29:39.571 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_cost": "20971520", 2024-09-17T21:29:39.571 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-17T21:29:39.571 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_max_active": "0", 2024-09-17T21:29:39.571 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-17T21:29:39.571 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-17T21:29:39.571 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-17T21:29:39.571 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-17T21:29:39.571 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_max_single_start": "1", 2024-09-17T21:29:39.571 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_op_priority": "3", 2024-09-17T21:29:39.571 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-17T21:29:39.571 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_priority": "5", 2024-09-17T21:29:39.571 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-17T21:29:39.571 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_sleep": "0.000000", 2024-09-17T21:29:39.571 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-17T21:29:39.572 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-17T21:29:39.572 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-17T21:29:39.572 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_repair_during_recovery": "false", 2024-09-17T21:29:39.572 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_requested_scrub_priority": "120", 2024-09-17T21:29:39.572 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-17T21:29:39.572 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-17T21:29:39.572 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_auto_repair": "false", 2024-09-17T21:29:39.572 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-17T21:29:39.572 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-17T21:29:39.572 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_begin_hour": "0", 2024-09-17T21:29:39.572 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_begin_week_day": "0", 2024-09-17T21:29:39.572 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_chunk_max": "15", 2024-09-17T21:29:39.572 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_chunk_min": "5", 2024-09-17T21:29:39.572 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_cost": "52428800", 2024-09-17T21:29:39.573 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-17T21:29:39.573 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_during_recovery": "false", 2024-09-17T21:29:39.573 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_end_hour": "0", 2024-09-17T21:29:39.573 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_end_week_day": "0", 2024-09-17T21:29:39.573 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_event_cost": "4096", 2024-09-17T21:29:39.573 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-17T21:29:39.573 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-17T21:29:39.573 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_invalid_stats": "true", 2024-09-17T21:29:39.573 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-17T21:29:39.573 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-17T21:29:39.573 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_max_preemptions": "5", 2024-09-17T21:29:39.573 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-17T21:29:39.574 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_priority": "5", 2024-09-17T21:29:39.574 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_retry_after_noscrub": "60", 2024-09-17T21:29:39.574 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_retry_delay": "30", 2024-09-17T21:29:39.574 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_retry_new_interval": "10", 2024-09-17T21:29:39.574 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_retry_pg_state": "60", 2024-09-17T21:29:39.574 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_retry_trimming": "10", 2024-09-17T21:29:39.574 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_sleep": "0.000000", 2024-09-17T21:29:39.574 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-17T21:29:39.574 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-17T21:29:39.574 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-17T21:29:39.574 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-17T21:29:39.574 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_skip_data_digest": "false", 2024-09-17T21:29:39.574 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_smart_report_timeout": "5", 2024-09-17T21:29:39.574 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_snap_trim_cost": "1048576", 2024-09-17T21:29:39.575 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_snap_trim_priority": "5", 2024-09-17T21:29:39.575 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-17T21:29:39.575 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-17T21:29:39.575 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-17T21:29:39.575 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-17T21:29:39.575 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-17T21:29:39.575 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-17T21:29:39.575 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-17T21:29:39.575 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_target_transaction_size": "30", 2024-09-17T21:29:39.575 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-17T21:29:39.575 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-17T21:29:39.575 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-17T21:29:39.575 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-17T21:29:39.575 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-17T21:29:39.576 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-17T21:29:39.576 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-17T21:29:39.576 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-17T21:29:39.576 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-17T21:29:39.576 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-17T21:29:39.576 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tracing": "false", 2024-09-17T21:29:39.576 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_use_stale_snap": "false", 2024-09-17T21:29:39.576 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-17T21:29:39.576 INFO:teuthology.orchestra.run.smithi129.stdout: "osdc_blkin_trace_all": "false", 2024-09-17T21:29:39.576 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_kill_at": "0", 2024-09-17T21:29:39.576 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_max_join_drift": "10", 2024-09-17T21:29:39.576 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_min": "500", 2024-09-17T21:29:39.576 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_min_wait": "0.050000", 2024-09-17T21:29:39.577 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_propose_interval": "1.000000", 2024-09-17T21:29:39.577 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_service_trim_max": "500", 2024-09-17T21:29:39.577 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-17T21:29:39.577 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_service_trim_min": "250", 2024-09-17T21:29:39.577 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_stash_full_interval": "25", 2024-09-17T21:29:39.577 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_trim_max": "500", 2024-09-17T21:29:39.577 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_trim_min": "250", 2024-09-17T21:29:39.577 INFO:teuthology.orchestra.run.smithi129.stdout: "perf": "true", 2024-09-17T21:29:39.577 INFO:teuthology.orchestra.run.smithi129.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-17T21:29:39.577 INFO:teuthology.orchestra.run.smithi129.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-17T21:29:39.577 INFO:teuthology.orchestra.run.smithi129.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-17T21:29:39.578 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "-", 2024-09-17T21:29:39.578 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrv": "", 2024-09-17T21:29:39.578 INFO:teuthology.orchestra.run.smithi129.stdout: "public_bind_addr": "-", 2024-09-17T21:29:39.578 INFO:teuthology.orchestra.run.smithi129.stdout: "public_network": "", 2024-09-17T21:29:39.578 INFO:teuthology.orchestra.run.smithi129.stdout: "public_network_interface": "", 2024-09-17T21:29:39.578 INFO:teuthology.orchestra.run.smithi129.stdout: "qat_compressor_busy_polling": "false", 2024-09-17T21:29:39.578 INFO:teuthology.orchestra.run.smithi129.stdout: "qat_compressor_enabled": "false", 2024-09-17T21:29:39.578 INFO:teuthology.orchestra.run.smithi129.stdout: "qat_compressor_session_max_number": "256", 2024-09-17T21:29:39.578 INFO:teuthology.orchestra.run.smithi129.stdout: "rados_mon_op_timeout": "0", 2024-09-17T21:29:39.578 INFO:teuthology.orchestra.run.smithi129.stdout: "rados_osd_op_timeout": "0", 2024-09-17T21:29:39.578 INFO:teuthology.orchestra.run.smithi129.stdout: "rados_tracing": "false", 2024-09-17T21:29:39.578 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_atime_update_interval": "60", 2024-09-17T21:29:39.578 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-17T21:29:39.578 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_balance_parent_reads": "false", 2024-09-17T21:29:39.578 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_balance_snap_reads": "false", 2024-09-17T21:29:39.579 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_blkin_trace_all": "false", 2024-09-17T21:29:39.579 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-17T21:29:39.579 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-17T21:29:39.579 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_cache": "true", 2024-09-17T21:29:39.579 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-17T21:29:39.579 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-17T21:29:39.579 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-17T21:29:39.579 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-17T21:29:39.579 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_cache_policy": "writearound", 2024-09-17T21:29:39.579 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_cache_size": "33554432", 2024-09-17T21:29:39.579 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-17T21:29:39.579 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-17T21:29:39.579 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_clone_copy_on_read": "false", 2024-09-17T21:29:39.579 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_compression_hint": "none", 2024-09-17T21:29:39.579 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_concurrent_management_ops": "10", 2024-09-17T21:29:39.580 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-17T21:29:39.580 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_clone_format": "auto", 2024-09-17T21:29:39.580 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_data_pool": "", 2024-09-17T21:29:39.580 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_features": "61", 2024-09-17T21:29:39.580 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_format": "2", 2024-09-17T21:29:39.580 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_map_options": "", 2024-09-17T21:29:39.580 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_order": "22", 2024-09-17T21:29:39.580 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_pool": "rbd", 2024-09-17T21:29:39.580 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-17T21:29:39.580 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_stripe_count": "0", 2024-09-17T21:29:39.580 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_stripe_unit": "0", 2024-09-17T21:29:39.580 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-17T21:29:39.580 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-17T21:29:39.580 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-17T21:29:39.581 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_enable_alloc_hint": "true", 2024-09-17T21:29:39.581 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-17T21:29:39.581 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_io_scheduler": "simple", 2024-09-17T21:29:39.581 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-17T21:29:39.581 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-17T21:29:39.581 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-17T21:29:39.581 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-17T21:29:39.581 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-17T21:29:39.581 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-17T21:29:39.581 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-17T21:29:39.581 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-17T21:29:39.581 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-17T21:29:39.581 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_order": "24", 2024-09-17T21:29:39.581 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_pool": "", 2024-09-17T21:29:39.582 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_splay_width": "4", 2024-09-17T21:29:39.582 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_localize_parent_reads": "false", 2024-09-17T21:29:39.582 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_localize_snap_reads": "false", 2024-09-17T21:29:39.582 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-17T21:29:39.582 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-17T21:29:39.582 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-17T21:29:39.582 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-17T21:29:39.582 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-17T21:29:39.582 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-17T21:29:39.582 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-17T21:29:39.582 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-17T21:29:39.582 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-17T21:29:39.582 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-17T21:29:39.583 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-17T21:29:39.583 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-17T21:29:39.583 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-17T21:29:39.583 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-17T21:29:39.583 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-17T21:29:39.583 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-17T21:29:39.583 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-17T21:29:39.583 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-17T21:29:39.583 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-17T21:29:39.583 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-17T21:29:39.583 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-17T21:29:39.583 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-17T21:29:39.584 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-17T21:29:39.584 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-17T21:29:39.584 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-17T21:29:39.584 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-17T21:29:39.584 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-17T21:29:39.584 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-17T21:29:39.584 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-17T21:29:39.584 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-17T21:29:39.584 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-17T21:29:39.584 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mtime_update_interval": "60", 2024-09-17T21:29:39.584 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_non_blocking_aio": "true", 2024-09-17T21:29:39.584 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_op_thread_timeout": "60", 2024-09-17T21:29:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_op_threads": "1", 2024-09-17T21:29:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_parent_cache_enabled": "false", 2024-09-17T21:29:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-17T21:29:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-17T21:29:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-17T21:29:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_plugins": "", 2024-09-17T21:29:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_bps_burst": "0", 2024-09-17T21:29:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-17T21:29:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_bps_limit": "0", 2024-09-17T21:29:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_exclude_ops": "0", 2024-09-17T21:29:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_iops_burst": "0", 2024-09-17T21:29:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-17T21:29:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_iops_limit": "0", 2024-09-17T21:29:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-17T21:29:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-17T21:29:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-17T21:29:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-17T21:29:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-17T21:29:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-17T21:29:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-17T21:29:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-17T21:29:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-17T21:29:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-17T21:29:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-17T21:29:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-17T21:29:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-17T21:29:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-17T21:29:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_read_from_replica_policy": "default", 2024-09-17T21:29:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-17T21:29:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-17T21:29:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-17T21:29:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-17T21:29:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_skip_partial_discard": "true", 2024-09-17T21:29:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-17T21:29:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_tracing": "false", 2024-09-17T21:29:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_validate_names": "true", 2024-09-17T21:29:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_validate_pool": "true", 2024-09-17T21:29:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "restapi_base_url": "", 2024-09-17T21:29:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "restapi_log_level": "", 2024-09-17T21:29:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-17T21:29:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-17T21:29:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_acl_grants_max_num": "100", 2024-09-17T21:29:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_admin_entry": "admin", 2024-09-17T21:29:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-17T21:29:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_asio_assert_yielding": "false", 2024-09-17T21:29:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_backend_store": "rados", 2024-09-17T21:29:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_barbican_url": "", 2024-09-17T21:29:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_beast_enable_async": "true", 2024-09-17T21:29:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_bucket_counters_cache": "false", 2024-09-17T21:29:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-17T21:29:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-17T21:29:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-17T21:29:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-17T21:29:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-17T21:29:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-17T21:29:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-17T21:29:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-17T21:29:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_cache_enabled": "true", 2024-09-17T21:29:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_cache_expiry_interval": "900", 2024-09-17T21:29:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_cache_lru_size": "25000", 2024-09-17T21:29:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_config_store": "rados", 2024-09-17T21:29:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_content_length_compat": "false", 2024-09-17T21:29:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_copy_obj_progress": "true", 2024-09-17T21:29:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-17T21:29:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_cors_rules_max_num": "100", 2024-09-17T21:29:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_cross_domain_policy": "", 2024-09-17T21:29:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-17T21:29:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_kmip_addr": "", 2024-09-17T21:29:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-17T21:29:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-17T21:29:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-17T21:29:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-17T21:29:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_kmip_password": "", 2024-09-17T21:29:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-17T21:29:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_kmip_username": "", 2024-09-17T21:29:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_require_ssl": "true", 2024-09-17T21:29:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-17T21:29:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-17T21:29:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-17T21:29:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-17T21:29:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-17T21:29:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-17T21:29:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-17T21:29:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-17T21:29:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-17T21:29:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-17T21:29:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-17T21:29:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-17T21:29:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-17T21:29:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-17T21:29:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-17T21:29:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_addr": "", 2024-09-17T21:29:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_auth": "token", 2024-09-17T21:29:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_namespace": "", 2024-09-17T21:29:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_prefix": "", 2024-09-17T21:29:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-17T21:29:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-17T21:29:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-17T21:29:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-17T21:29:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_token_file": "", 2024-09-17T21:29:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-17T21:29:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_curl_buffersize": "524288", 2024-09-17T21:29:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-17T21:29:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_curl_low_speed_time": "300", 2024-09-17T21:29:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-17T21:29:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-17T21:29:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-17T21:29:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-17T21:29:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-17T21:29:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-17T21:29:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-17T21:29:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-17T21:29:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-17T21:29:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-17T21:29:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-17T21:29:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-17T21:29:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-17T21:29:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-17T21:29:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-17T21:29:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-17T21:29:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-17T21:29:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-17T21:29:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-17T21:29:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_data_log_changes_size": "1000", 2024-09-17T21:29:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_data_log_num_shards": "128", 2024-09-17T21:29:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-17T21:29:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_data_log_window": "30", 2024-09-17T21:29:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-17T21:29:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-17T21:29:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-17T21:29:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-17T21:29:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-17T21:29:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-17T21:29:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-17T21:29:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-17T21:29:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-17T21:29:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-17T21:29:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-17T21:29:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-17T21:29:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-17T21:29:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_disable_s3select": "false", 2024-09-17T21:29:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-17T21:29:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-17T21:29:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-17T21:29:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-17T21:29:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-17T21:29:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-17T21:29:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-17T21:29:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-17T21:29:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-17T21:29:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-17T21:29:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-17T21:29:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-17T21:29:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dns_name": "", 2024-09-17T21:29:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dns_s3website_name": "", 2024-09-17T21:29:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dynamic_resharding": "true", 2024-09-17T21:29:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-17T21:29:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-17T21:29:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-17T21:29:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_enable_gc_threads": "true", 2024-09-17T21:29:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_enable_lc_threads": "true", 2024-09-17T21:29:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_enable_mdsearch": "true", 2024-09-17T21:29:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_enable_ops_log": "false", 2024-09-17T21:29:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_enable_quota_threads": "true", 2024-09-17T21:29:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_enable_static_website": "false", 2024-09-17T21:29:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_enable_usage_log": "false", 2024-09-17T21:29:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_enforce_swift_acls": "true", 2024-09-17T21:29:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_exit_timeout_secs": "120", 2024-09-17T21:29:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_expose_bucket": "false", 2024-09-17T21:29:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_extended_http_attrs": "", 2024-09-17T21:29:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_filter": "none", 2024-09-17T21:29:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-17T21:29:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_frontends": "beast port=7480", 2024-09-17T21:29:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-17T21:29:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_gc_max_deferred": "50", 2024-09-17T21:29:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-17T21:29:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_gc_max_objs": "32", 2024-09-17T21:29:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-17T21:29:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-17T21:29:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-17T21:29:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-17T21:29:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_gc_processor_period": "3600", 2024-09-17T21:29:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-17T21:29:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-17T21:29:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-17T21:29:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-17T21:29:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_init_timeout": "300", 2024-09-17T21:29:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-17T21:29:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-17T21:29:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_kafka_connection_idle": "300", 2024-09-17T21:29:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-17T21:29:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-17T21:29:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-17T21:29:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-17T21:29:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-17T21:29:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_admin_domain": "", 2024-09-17T21:29:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_admin_password": "", 2024-09-17T21:29:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_admin_password_path": "", 2024-09-17T21:29:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_admin_project": "", 2024-09-17T21:29:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_admin_tenant": "", 2024-09-17T21:29:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_admin_token": "", 2024-09-17T21:29:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_admin_token_path": "", 2024-09-17T21:29:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_admin_user": "", 2024-09-17T21:29:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_barbican_domain": "", 2024-09-17T21:29:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_barbican_password": "", 2024-09-17T21:29:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_barbican_project": "", 2024-09-17T21:29:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-17T21:29:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_barbican_user": "", 2024-09-17T21:29:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-17T21:29:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-17T21:29:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-17T21:29:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-17T21:29:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-17T21:29:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_url": "", 2024-09-17T21:29:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-17T21:29:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lc_debug_interval": "-1", 2024-09-17T21:29:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lc_lock_max_time": "90", 2024-09-17T21:29:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lc_max_objs": "32", 2024-09-17T21:29:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lc_max_rules": "1000", 2024-09-17T21:29:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lc_max_worker": "3", 2024-09-17T21:29:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-17T21:29:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lc_thread_delay": "0", 2024-09-17T21:29:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-17T21:29:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ldap_dnattr": "uid", 2024-09-17T21:29:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-17T21:29:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ldap_searchfilter": "", 2024-09-17T21:29:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-17T21:29:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ldap_uri": "", 2024-09-17T21:29:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-17T21:29:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-17T21:29:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-17T21:29:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-17T21:29:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_log_http_headers": "", 2024-09-17T21:29:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-17T21:29:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-17T21:29:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_log_object_name_utc": "false", 2024-09-17T21:29:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-17T21:29:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-17T21:29:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_attr_name_len": "0", 2024-09-17T21:29:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_attr_size": "0", 2024-09-17T21:29:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-17T21:29:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_chunk_size": "4194304", 2024-09-17T21:29:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-17T21:29:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-17T21:29:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-17T21:29:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_listing_results": "5000", 2024-09-17T21:29:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_notify_retries": "10", 2024-09-17T21:29:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-17T21:29:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_put_param_size": "1048576", 2024-09-17T21:29:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_put_size": "5368709120", 2024-09-17T21:29:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_slo_entries": "1000", 2024-09-17T21:29:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_md_log_max_shards": "64", 2024-09-17T21:29:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-17T21:29:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-17T21:29:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-17T21:29:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-17T21:29:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_mp_lock_max_time": "600", 2024-09-17T21:29:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-17T21:29:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-17T21:29:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-17T21:29:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-17T21:29:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-17T21:29:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-17T21:29:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-17T21:29:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-17T21:29:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_max_gc": "300", 2024-09-17T21:29:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-17T21:29:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-17T21:29:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-17T21:29:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-17T21:29:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-17T21:29:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-17T21:29:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-17T21:29:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_num_async_rados_threads": "32", 2024-09-17T21:29:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_num_control_oids": "8", 2024-09-17T21:29:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_numa_node": "-1", 2024-09-17T21:29:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-17T21:29:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-17T21:29:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_objexp_chunk_size": "100", 2024-09-17T21:29:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_objexp_gc_interval": "600", 2024-09-17T21:29:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-17T21:29:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-17T21:29:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-17T21:29:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_op_thread_timeout": "600", 2024-09-17T21:29:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_op_tracing": "false", 2024-09-17T21:29:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_opa_token": "", 2024-09-17T21:29:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_opa_url": "", 2024-09-17T21:29:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_opa_verify_ssl": "true", 2024-09-17T21:29:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-17T21:29:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-17T21:29:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ops_log_rados": "false", 2024-09-17T21:29:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ops_log_socket_path": "", 2024-09-17T21:29:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-17T21:29:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-17T21:29:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-17T21:29:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_period_push_interval": "2.000000", 2024-09-17T21:29:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-17T21:29:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-17T21:29:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-17T21:29:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-17T21:29:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_posix_cache_lanes": "3", 2024-09-17T21:29:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-17T21:29:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-17T21:29:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_posix_cache_partitions": "3", 2024-09-17T21:29:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-17T21:29:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_print_continue": "true", 2024-09-17T21:29:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-17T21:29:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-17T21:29:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-17T21:29:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-17T21:29:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-17T21:29:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_rados_tracing": "false", 2024-09-17T21:29:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_realm": "", 2024-09-17T21:29:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_realm_id": "", 2024-09-17T21:29:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-17T21:29:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_region": "", 2024-09-17T21:29:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-17T21:29:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-17T21:29:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-17T21:29:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-17T21:29:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_request_uri": "", 2024-09-17T21:29:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_reshard_batch_size": "64", 2024-09-17T21:29:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-17T21:29:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-17T21:29:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_reshard_max_aio": "128", 2024-09-17T21:29:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_reshard_num_logs": "16", 2024-09-17T21:29:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-17T21:29:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-17T21:29:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_reshard_thread_interval": "600", 2024-09-17T21:29:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-17T21:29:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_resolve_cname": "false", 2024-09-17T21:29:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-17T21:29:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_run_sync_thread": "true", 2024-09-17T21:29:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-17T21:29:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-17T21:29:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-17T21:29:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-17T21:29:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-17T21:29:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-17T21:29:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-17T21:29:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-17T21:29:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-17T21:29:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_scheduler_type": "throttler", 2024-09-17T21:29:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_script_uri": "", 2024-09-17T21:29:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_service_provider_name": "", 2024-09-17T21:29:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-17T21:29:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sts_client_id": "", 2024-09-17T21:29:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sts_client_secret": "", 2024-09-17T21:29:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sts_entry": "sts", 2024-09-17T21:29:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sts_key": "sts", 2024-09-17T21:29:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-17T21:29:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sts_min_session_duration": "900", 2024-09-17T21:29:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sts_token_introspection_url": "", 2024-09-17T21:29:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_account_in_url": "false", 2024-09-17T21:29:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_auth_entry": "auth", 2024-09-17T21:29:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_auth_url": "", 2024-09-17T21:29:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_custom_header": "", 2024-09-17T21:29:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-17T21:29:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_need_stats": "true", 2024-09-17T21:29:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_tenant_name": "", 2024-09-17T21:29:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_token_expiration": "86400", 2024-09-17T21:29:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_url": "", 2024-09-17T21:29:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_url_prefix": "swift", 2024-09-17T21:29:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-17T21:29:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-17T21:29:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-17T21:29:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_lease_period": "120", 2024-09-17T21:29:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-17T21:29:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-17T21:29:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-17T21:29:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-17T21:29:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-17T21:29:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-17T21:29:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-17T21:29:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-17T21:29:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-17T21:29:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_thread_pool_size": "512", 2024-09-17T21:29:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-17T21:29:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-17T21:29:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-17T21:29:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-17T21:29:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_torrent_comment": "", 2024-09-17T21:29:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_torrent_createby": "", 2024-09-17T21:29:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_torrent_encoding": "", 2024-09-17T21:29:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_torrent_flag": "false", 2024-09-17T21:29:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-17T21:29:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_torrent_origin": "", 2024-09-17T21:29:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-17T21:29:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_torrent_tracker": "", 2024-09-17T21:29:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_trust_forwarded_https": "false", 2024-09-17T21:29:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-17T21:29:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-17T21:29:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_usage_max_shards": "32", 2024-09-17T21:29:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_usage_max_user_shards": "1", 2024-09-17T21:29:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_use_opa_authz": "false", 2024-09-17T21:29:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_counters_cache": "false", 2024-09-17T21:29:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-17T21:29:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-17T21:29:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-17T21:29:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_max_buckets": "1000", 2024-09-17T21:29:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_policies_max_num": "100", 2024-09-17T21:29:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-17T21:29:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-17T21:29:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-17T21:29:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-17T21:29:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_unique_email": "true", 2024-09-17T21:29:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_verify_ssl": "true", 2024-09-17T21:29:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-17T21:29:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_zone": "", 2024-09-17T21:29:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_zone_id": "", 2024-09-17T21:29:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-17T21:29:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_zonegroup": "", 2024-09-17T21:29:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_zonegroup_id": "", 2024-09-17T21:29:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-17T21:29:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "rgwlc_auto_session_clear": "true", 2024-09-17T21:29:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-17T21:29:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_block_size": "4096", 2024-09-17T21:29:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-17T21:29:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-17T21:29:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-17T21:29:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-17T21:29:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-17T21:29:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_cache_size": "536870912", 2024-09-17T21:29:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-17T21:29:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-17T21:29:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-17T21:29:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-17T21:29:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-17T21:29:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-17T21:29:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-17T21:29:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-17T21:29:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_index_type": "binary_search", 2024-09-17T21:29:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-17T21:29:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-17T21:29:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_partition_filters": "false", 2024-09-17T21:29:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_perf": "false", 2024-09-17T21:29:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-17T21:29:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-17T21:29:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-17T21:29:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "run_dir": "/var/run/ceph", 2024-09-17T21:29:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_block_create": "true", 2024-09-17T21:29:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_cache_lru_size": "67108864", 2024-09-17T21:29:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-17T21:29:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-17T21:29:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_default_max_object_size": "16777216", 2024-09-17T21:29:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-17T21:29:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_device_size": "53687091200", 2024-09-17T21:29:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-17T21:29:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_full_integrity_check": "false", 2024-09-17T21:29:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_journal_batch_capacity": "16", 2024-09-17T21:29:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-17T21:29:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-17T21:29:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-17T21:29:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_main_device_type": "SSD", 2024-09-17T21:29:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-17T21:29:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-17T21:29:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-17T21:29:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-17T21:29:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-17T21:29:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-17T21:29:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_segment_size": "67108864", 2024-09-17T21:29:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "set_keepcaps": "false", 2024-09-17T21:29:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "setgroup": "", 2024-09-17T21:29:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "setuser": "", 2024-09-17T21:29:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "setuser_match_path": "", 2024-09-17T21:29:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-17T21:29:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "thp": "false", 2024-09-17T21:29:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "threadpool_default_timeout": "60", 2024-09-17T21:29:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-17T21:29:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "throttler_perf_counter": "true", 2024-09-17T21:29:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "tmp_dir": "/tmp", 2024-09-17T21:29:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-17T21:29:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "uadk_compressor_enabled": "false", 2024-09-17T21:29:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-17T21:29:39.619 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:39.621 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-17T21:29:39.732 INFO:teuthology.orchestra.run.smithi129.stdout:mon.a: {} 2024-09-17T21:29:39.733 INFO:teuthology.orchestra.run.smithi129.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T21:29:39.742 INFO:teuthology.orchestra.run.smithi129.stdout:mon.b: {} 2024-09-17T21:29:39.742 INFO:teuthology.orchestra.run.smithi129.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T21:29:39.750 INFO:teuthology.orchestra.run.smithi129.stdout:mon.c: {} 2024-09-17T21:29:39.751 INFO:teuthology.orchestra.run.smithi129.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T21:29:39.761 DEBUG:teuthology.orchestra.run.smithi171:> type ceph-objectstore-tool 2024-09-17T21:29:39.780 INFO:teuthology.orchestra.run.smithi171.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T21:29:39.780 DEBUG:teuthology.orchestra.run.smithi129:> type ceph-objectstore-tool 2024-09-17T21:29:39.795 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T21:29:39.795 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-17T21:29:39.804 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-17T21:29:39.805 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-17T21:29:39.805 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T21:29:39.805 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:29:39.805 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T21:29:39.805 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T21:29:39.805 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-17T21:29:39.806 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-17T21:29:39.806 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:29:39.806 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-17T21:29:39.806 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:29:39.806 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-17T21:29:39.807 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:29:39.807 DEBUG:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-17T21:29:39.821 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:39.818+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:39.875 INFO:teuthology.orchestra.run.smithi129.stdout:mapped writes DISABLED 2024-09-17T21:29:39.876 INFO:teuthology.orchestra.run.smithi129.stdout:mapped reads DISABLED 2024-09-17T21:29:39.876 INFO:teuthology.orchestra.run.smithi129.stdout:Seed set to 1760 2024-09-17T21:29:39.917 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:39.916+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:39.984 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:29:39.984 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:29:39.985 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:39.985 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:39.998 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:29:40.009 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:29:40.009 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:29:40.010 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:40.008+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:40.028 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:29:40.028 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:29:40.061 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:29:40.061 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:29:40.082 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:29:40.082 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:29:40.099 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:29:40.100 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:29:40.109 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:40.106+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:40.131 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:40.131 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:40.134 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:29:40.134 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:29:40.143 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:29:40.154 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:29:40.154 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:29:40.165 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:29:40.165 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:40.165 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 24, 2024-09-17T21:29:40.165 INFO:teuthology.orchestra.run.smithi129.stdout: "fsid": "f6d3a80c-8a1f-4716-b25d-e97b28890c0d", 2024-09-17T21:29:40.166 INFO:teuthology.orchestra.run.smithi129.stdout: "created": "2024-09-17T21:29:12.606637+0000", 2024-09-17T21:29:40.166 INFO:teuthology.orchestra.run.smithi129.stdout: "modified": "2024-09-17T21:29:37.389840+0000", 2024-09-17T21:29:40.166 INFO:teuthology.orchestra.run.smithi129.stdout: "last_up_change": "2024-09-17T21:29:24.913767+0000", 2024-09-17T21:29:40.166 INFO:teuthology.orchestra.run.smithi129.stdout: "last_in_change": "2024-09-17T21:29:15.180249+0000", 2024-09-17T21:29:40.166 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T21:29:40.166 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_num": 5799936, 2024-09-17T21:29:40.166 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_set": [ 2024-09-17T21:29:40.166 INFO:teuthology.orchestra.run.smithi129.stdout: "pglog_hardlimit", 2024-09-17T21:29:40.166 INFO:teuthology.orchestra.run.smithi129.stdout: "purged_snapdirs", 2024-09-17T21:29:40.166 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_deletes", 2024-09-17T21:29:40.166 INFO:teuthology.orchestra.run.smithi129.stdout: "sortbitwise" 2024-09-17T21:29:40.166 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:29:40.166 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_version": 5, 2024-09-17T21:29:40.167 INFO:teuthology.orchestra.run.smithi129.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T21:29:40.167 INFO:teuthology.orchestra.run.smithi129.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T21:29:40.167 INFO:teuthology.orchestra.run.smithi129.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T21:29:40.167 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_snapshot": "", 2024-09-17T21:29:40.167 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_max": 2, 2024-09-17T21:29:40.167 INFO:teuthology.orchestra.run.smithi129.stdout: "max_osd": 8, 2024-09-17T21:29:40.167 INFO:teuthology.orchestra.run.smithi129.stdout: "require_min_compat_client": "luminous", 2024-09-17T21:29:40.167 INFO:teuthology.orchestra.run.smithi129.stdout: "min_compat_client": "jewel", 2024-09-17T21:29:40.167 INFO:teuthology.orchestra.run.smithi129.stdout: "require_osd_release": "squid", 2024-09-17T21:29:40.167 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_crimson": false, 2024-09-17T21:29:40.167 INFO:teuthology.orchestra.run.smithi129.stdout: "pools": [ 2024-09-17T21:29:40.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.167 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 1, 2024-09-17T21:29:40.167 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": ".mgr", 2024-09-17T21:29:40.168 INFO:teuthology.orchestra.run.smithi129.stdout: "create_time": "2024-09-17T21:29:26.876763+0000", 2024-09-17T21:29:40.168 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-17T21:29:40.168 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_names": "hashpspool", 2024-09-17T21:29:40.168 INFO:teuthology.orchestra.run.smithi129.stdout: "type": 1, 2024-09-17T21:29:40.168 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 2, 2024-09-17T21:29:40.168 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": 1, 2024-09-17T21:29:40.168 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_rule": 0, 2024-09-17T21:29:40.168 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:29:40.168 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:29:40.168 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:29:40.168 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:29:40.168 INFO:teuthology.orchestra.run.smithi129.stdout: "is_stretch_pool": false, 2024-09-17T21:29:40.168 INFO:teuthology.orchestra.run.smithi129.stdout: "object_hash": 2, 2024-09-17T21:29:40.169 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:29:40.169 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": 1, 2024-09-17T21:29:40.169 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num": 1, 2024-09-17T21:29:40.169 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num_target": 1, 2024-09-17T21:29:40.169 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_target": 1, 2024-09-17T21:29:40.169 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_pending": 1, 2024-09-17T21:29:40.169 INFO:teuthology.orchestra.run.smithi129.stdout: "last_pg_merge_meta": { 2024-09-17T21:29:40.169 INFO:teuthology.orchestra.run.smithi129.stdout: "source_pgid": "0.0", 2024-09-17T21:29:40.169 INFO:teuthology.orchestra.run.smithi129.stdout: "ready_epoch": 0, 2024-09-17T21:29:40.169 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_started": 0, 2024-09-17T21:29:40.169 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_clean": 0, 2024-09-17T21:29:40.169 INFO:teuthology.orchestra.run.smithi129.stdout: "source_version": "0'0", 2024-09-17T21:29:40.169 INFO:teuthology.orchestra.run.smithi129.stdout: "target_version": "0'0" 2024-09-17T21:29:40.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.170 INFO:teuthology.orchestra.run.smithi129.stdout: "last_change": "19", 2024-09-17T21:29:40.170 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend": "0", 2024-09-17T21:29:40.170 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:29:40.170 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:29:40.170 INFO:teuthology.orchestra.run.smithi129.stdout: "auid": 0, 2024-09-17T21:29:40.170 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:29:40.170 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_seq": 0, 2024-09-17T21:29:40.170 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_epoch": 0, 2024-09-17T21:29:40.170 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_snaps": [], 2024-09-17T21:29:40.170 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps": "[]", 2024-09-17T21:29:40.170 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_bytes": 0, 2024-09-17T21:29:40.170 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_objects": 0, 2024-09-17T21:29:40.170 INFO:teuthology.orchestra.run.smithi129.stdout: "tiers": [], 2024-09-17T21:29:40.170 INFO:teuthology.orchestra.run.smithi129.stdout: "tier_of": -1, 2024-09-17T21:29:40.171 INFO:teuthology.orchestra.run.smithi129.stdout: "read_tier": -1, 2024-09-17T21:29:40.171 INFO:teuthology.orchestra.run.smithi129.stdout: "write_tier": -1, 2024-09-17T21:29:40.171 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_mode": "none", 2024-09-17T21:29:40.171 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_bytes": 0, 2024-09-17T21:29:40.171 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_objects": 0, 2024-09-17T21:29:40.171 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:29:40.171 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:29:40.171 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:29:40.171 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_flush_age": 0, 2024-09-17T21:29:40.171 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_evict_age": 0, 2024-09-17T21:29:40.171 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profile": "", 2024-09-17T21:29:40.171 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_params": { 2024-09-17T21:29:40.171 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "none" 2024-09-17T21:29:40.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.172 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_period": 0, 2024-09-17T21:29:40.172 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_count": 0, 2024-09-17T21:29:40.172 INFO:teuthology.orchestra.run.smithi129.stdout: "use_gmt_hitset": true, 2024-09-17T21:29:40.172 INFO:teuthology.orchestra.run.smithi129.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:29:40.172 INFO:teuthology.orchestra.run.smithi129.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:29:40.172 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:29:40.172 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:29:40.172 INFO:teuthology.orchestra.run.smithi129.stdout: "grade_table": [], 2024-09-17T21:29:40.172 INFO:teuthology.orchestra.run.smithi129.stdout: "stripe_width": 0, 2024-09-17T21:29:40.172 INFO:teuthology.orchestra.run.smithi129.stdout: "expected_num_objects": 0, 2024-09-17T21:29:40.172 INFO:teuthology.orchestra.run.smithi129.stdout: "fast_read": false, 2024-09-17T21:29:40.172 INFO:teuthology.orchestra.run.smithi129.stdout: "options": { 2024-09-17T21:29:40.172 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_max": 32, 2024-09-17T21:29:40.172 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_min": 1 2024-09-17T21:29:40.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.173 INFO:teuthology.orchestra.run.smithi129.stdout: "application_metadata": { 2024-09-17T21:29:40.173 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr": {} 2024-09-17T21:29:40.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.173 INFO:teuthology.orchestra.run.smithi129.stdout: "read_balance": { 2024-09-17T21:29:40.173 INFO:teuthology.orchestra.run.smithi129.stdout: "score_type": "Fair distribution", 2024-09-17T21:29:40.173 INFO:teuthology.orchestra.run.smithi129.stdout: "score_acting": 8, 2024-09-17T21:29:40.173 INFO:teuthology.orchestra.run.smithi129.stdout: "score_stable": 8, 2024-09-17T21:29:40.173 INFO:teuthology.orchestra.run.smithi129.stdout: "optimal_score": 0.25, 2024-09-17T21:29:40.173 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_acting": 2, 2024-09-17T21:29:40.173 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_stable": 2, 2024-09-17T21:29:40.173 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:29:40.173 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity": 1, 2024-09-17T21:29:40.174 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:29:40.174 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.174 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 2, 2024-09-17T21:29:40.174 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": "rbd", 2024-09-17T21:29:40.174 INFO:teuthology.orchestra.run.smithi129.stdout: "create_time": "2024-09-17T21:29:31.078466+0000", 2024-09-17T21:29:40.174 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 8193, 2024-09-17T21:29:40.174 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:29:40.174 INFO:teuthology.orchestra.run.smithi129.stdout: "type": 1, 2024-09-17T21:29:40.174 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 2, 2024-09-17T21:29:40.174 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": 1, 2024-09-17T21:29:40.174 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_rule": 0, 2024-09-17T21:29:40.174 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:29:40.175 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:29:40.175 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:29:40.175 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:29:40.175 INFO:teuthology.orchestra.run.smithi129.stdout: "is_stretch_pool": false, 2024-09-17T21:29:40.175 INFO:teuthology.orchestra.run.smithi129.stdout: "object_hash": 2, 2024-09-17T21:29:40.175 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:29:40.175 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": 8, 2024-09-17T21:29:40.175 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num": 8, 2024-09-17T21:29:40.175 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num_target": 8, 2024-09-17T21:29:40.175 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_target": 8, 2024-09-17T21:29:40.175 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_pending": 8, 2024-09-17T21:29:40.175 INFO:teuthology.orchestra.run.smithi129.stdout: "last_pg_merge_meta": { 2024-09-17T21:29:40.175 INFO:teuthology.orchestra.run.smithi129.stdout: "source_pgid": "0.0", 2024-09-17T21:29:40.176 INFO:teuthology.orchestra.run.smithi129.stdout: "ready_epoch": 0, 2024-09-17T21:29:40.176 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_started": 0, 2024-09-17T21:29:40.176 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_clean": 0, 2024-09-17T21:29:40.176 INFO:teuthology.orchestra.run.smithi129.stdout: "source_version": "0'0", 2024-09-17T21:29:40.176 INFO:teuthology.orchestra.run.smithi129.stdout: "target_version": "0'0" 2024-09-17T21:29:40.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.176 INFO:teuthology.orchestra.run.smithi129.stdout: "last_change": "23", 2024-09-17T21:29:40.176 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend": "0", 2024-09-17T21:29:40.176 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:29:40.176 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:29:40.176 INFO:teuthology.orchestra.run.smithi129.stdout: "auid": 0, 2024-09-17T21:29:40.176 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:29:40.176 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_seq": 2, 2024-09-17T21:29:40.176 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_epoch": 23, 2024-09-17T21:29:40.177 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_snaps": [], 2024-09-17T21:29:40.177 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps": "[]", 2024-09-17T21:29:40.177 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_bytes": 0, 2024-09-17T21:29:40.177 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_objects": 0, 2024-09-17T21:29:40.177 INFO:teuthology.orchestra.run.smithi129.stdout: "tiers": [], 2024-09-17T21:29:40.177 INFO:teuthology.orchestra.run.smithi129.stdout: "tier_of": -1, 2024-09-17T21:29:40.177 INFO:teuthology.orchestra.run.smithi129.stdout: "read_tier": -1, 2024-09-17T21:29:40.177 INFO:teuthology.orchestra.run.smithi129.stdout: "write_tier": -1, 2024-09-17T21:29:40.177 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_mode": "none", 2024-09-17T21:29:40.177 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_bytes": 0, 2024-09-17T21:29:40.177 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_objects": 0, 2024-09-17T21:29:40.177 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:29:40.177 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:29:40.178 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:29:40.178 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_flush_age": 0, 2024-09-17T21:29:40.178 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_evict_age": 0, 2024-09-17T21:29:40.178 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profile": "", 2024-09-17T21:29:40.178 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_params": { 2024-09-17T21:29:40.178 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "none" 2024-09-17T21:29:40.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.178 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_period": 0, 2024-09-17T21:29:40.178 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_count": 0, 2024-09-17T21:29:40.178 INFO:teuthology.orchestra.run.smithi129.stdout: "use_gmt_hitset": true, 2024-09-17T21:29:40.178 INFO:teuthology.orchestra.run.smithi129.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:29:40.178 INFO:teuthology.orchestra.run.smithi129.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:29:40.178 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:29:40.179 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:29:40.179 INFO:teuthology.orchestra.run.smithi129.stdout: "grade_table": [], 2024-09-17T21:29:40.179 INFO:teuthology.orchestra.run.smithi129.stdout: "stripe_width": 0, 2024-09-17T21:29:40.179 INFO:teuthology.orchestra.run.smithi129.stdout: "expected_num_objects": 0, 2024-09-17T21:29:40.179 INFO:teuthology.orchestra.run.smithi129.stdout: "fast_read": false, 2024-09-17T21:29:40.179 INFO:teuthology.orchestra.run.smithi129.stdout: "options": {}, 2024-09-17T21:29:40.179 INFO:teuthology.orchestra.run.smithi129.stdout: "application_metadata": { 2024-09-17T21:29:40.179 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd": {} 2024-09-17T21:29:40.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.179 INFO:teuthology.orchestra.run.smithi129.stdout: "read_balance": { 2024-09-17T21:29:40.179 INFO:teuthology.orchestra.run.smithi129.stdout: "score_type": "Fair distribution", 2024-09-17T21:29:40.179 INFO:teuthology.orchestra.run.smithi129.stdout: "score_acting": 1.9900000095367432, 2024-09-17T21:29:40.179 INFO:teuthology.orchestra.run.smithi129.stdout: "score_stable": 1.9900000095367432, 2024-09-17T21:29:40.179 INFO:teuthology.orchestra.run.smithi129.stdout: "optimal_score": 0.87999999523162842, 2024-09-17T21:29:40.180 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_acting": 1.75, 2024-09-17T21:29:40.180 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_stable": 1.75, 2024-09-17T21:29:40.180 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:29:40.180 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity": 1, 2024-09-17T21:29:40.180 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:29:40.180 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.180 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.180 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:29:40.180 INFO:teuthology.orchestra.run.smithi129.stdout: "osds": [ 2024-09-17T21:29:40.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.180 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 0, 2024-09-17T21:29:40.180 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "ad067b51-9402-4f69-a0f0-08fbf068dcfc", 2024-09-17T21:29:40.180 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:29:40.181 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:29:40.181 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:29:40.181 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:29:40.181 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:29:40.181 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:29:40.181 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 13, 2024-09-17T21:29:40.181 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 0, 2024-09-17T21:29:40.181 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:29:40.181 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:29:40.181 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:29:40.181 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.181 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.181 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6800", 2024-09-17T21:29:40.181 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:29:40.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.182 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.182 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6801", 2024-09-17T21:29:40.182 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:29:40.182 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.182 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.182 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:29:40.182 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.182 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.182 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6802", 2024-09-17T21:29:40.183 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:29:40.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.183 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.183 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6803", 2024-09-17T21:29:40.183 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:29:40.183 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.183 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.183 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:29:40.183 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.183 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.183 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6806", 2024-09-17T21:29:40.184 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:29:40.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.184 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.184 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6807", 2024-09-17T21:29:40.184 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:29:40.184 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.184 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.184 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:29:40.184 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.184 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.184 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6804", 2024-09-17T21:29:40.184 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:29:40.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.185 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.185 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6805", 2024-09-17T21:29:40.185 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:29:40.185 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.185 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.185 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6801/3635367421", 2024-09-17T21:29:40.185 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6803/3635367421", 2024-09-17T21:29:40.185 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6807/3635367421", 2024-09-17T21:29:40.185 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6805/3635367421", 2024-09-17T21:29:40.185 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:29:40.186 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:29:40.186 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:29:40.186 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.186 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 1, 2024-09-17T21:29:40.186 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "30385fc0-ef5f-45f5-a3ec-37e890d4826e", 2024-09-17T21:29:40.186 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:29:40.186 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:29:40.186 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:29:40.186 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:29:40.186 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:29:40.186 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:29:40.186 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:29:40.186 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 20, 2024-09-17T21:29:40.187 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:29:40.187 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:29:40.187 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:29:40.187 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.187 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.187 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6818", 2024-09-17T21:29:40.187 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 176704438 2024-09-17T21:29:40.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.187 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.187 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6819", 2024-09-17T21:29:40.187 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 176704438 2024-09-17T21:29:40.188 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.188 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.188 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:29:40.188 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.188 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.188 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6820", 2024-09-17T21:29:40.188 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 176704438 2024-09-17T21:29:40.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.188 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.188 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6821", 2024-09-17T21:29:40.188 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 176704438 2024-09-17T21:29:40.188 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.189 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.189 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:29:40.189 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.189 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.189 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6824", 2024-09-17T21:29:40.189 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 176704438 2024-09-17T21:29:40.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.189 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.189 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6825", 2024-09-17T21:29:40.189 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 176704438 2024-09-17T21:29:40.189 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.190 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.190 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:29:40.190 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.190 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.190 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6822", 2024-09-17T21:29:40.190 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 176704438 2024-09-17T21:29:40.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.190 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.190 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6823", 2024-09-17T21:29:40.190 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 176704438 2024-09-17T21:29:40.190 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.190 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.191 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6819/176704438", 2024-09-17T21:29:40.191 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6821/176704438", 2024-09-17T21:29:40.191 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6825/176704438", 2024-09-17T21:29:40.191 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6823/176704438", 2024-09-17T21:29:40.191 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:29:40.191 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:29:40.191 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:29:40.191 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.192 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 2, 2024-09-17T21:29:40.192 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "76be9b2b-b63f-402a-9ab5-2564bd671ac3", 2024-09-17T21:29:40.192 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:29:40.192 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:29:40.192 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:29:40.192 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:29:40.192 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:29:40.192 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:29:40.192 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:29:40.192 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 20, 2024-09-17T21:29:40.192 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:29:40.193 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:29:40.193 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:29:40.193 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.193 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.193 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6808", 2024-09-17T21:29:40.193 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:29:40.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.193 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.193 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6809", 2024-09-17T21:29:40.193 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:29:40.193 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.193 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.194 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:29:40.194 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.194 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.194 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6810", 2024-09-17T21:29:40.194 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:29:40.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.194 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.194 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6811", 2024-09-17T21:29:40.194 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:29:40.194 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.194 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.195 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:29:40.195 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.195 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.195 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6814", 2024-09-17T21:29:40.195 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:29:40.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.195 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.195 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6815", 2024-09-17T21:29:40.195 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:29:40.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.195 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.196 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:29:40.196 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.196 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.196 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6812", 2024-09-17T21:29:40.196 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:29:40.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.196 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.196 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6813", 2024-09-17T21:29:40.196 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:29:40.196 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.197 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.197 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6809/1790706405", 2024-09-17T21:29:40.197 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6811/1790706405", 2024-09-17T21:29:40.197 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6815/1790706405", 2024-09-17T21:29:40.197 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6813/1790706405", 2024-09-17T21:29:40.197 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:29:40.197 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:29:40.197 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:29:40.197 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.197 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 3, 2024-09-17T21:29:40.197 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "d6c34e3b-1ac6-415c-9c56-38ae77a5c9b0", 2024-09-17T21:29:40.198 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:29:40.198 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:29:40.198 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:29:40.198 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:29:40.198 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:29:40.198 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:29:40.198 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:29:40.198 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 0, 2024-09-17T21:29:40.198 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:29:40.198 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:29:40.198 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:29:40.198 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.198 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.199 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6826", 2024-09-17T21:29:40.199 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:29:40.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.199 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.199 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6827", 2024-09-17T21:29:40.199 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:29:40.199 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.199 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.199 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:29:40.199 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.199 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.200 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6828", 2024-09-17T21:29:40.200 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:29:40.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.200 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.200 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6829", 2024-09-17T21:29:40.200 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:29:40.200 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.200 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.200 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:29:40.200 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.200 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.201 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6832", 2024-09-17T21:29:40.201 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:29:40.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.201 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.201 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6833", 2024-09-17T21:29:40.201 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:29:40.201 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.201 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.201 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:29:40.201 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.202 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.202 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6830", 2024-09-17T21:29:40.202 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:29:40.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.202 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.202 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6831", 2024-09-17T21:29:40.202 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:29:40.202 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.202 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.202 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6827/393361453", 2024-09-17T21:29:40.202 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6829/393361453", 2024-09-17T21:29:40.202 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6833/393361453", 2024-09-17T21:29:40.203 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6831/393361453", 2024-09-17T21:29:40.203 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:29:40.203 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:29:40.203 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:29:40.203 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.203 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 4, 2024-09-17T21:29:40.203 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "7d722e8c-c497-4f0c-9f61-d9826f8c93d8", 2024-09-17T21:29:40.203 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:29:40.203 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:29:40.203 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:29:40.203 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:29:40.204 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:29:40.204 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:29:40.204 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:29:40.204 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 0, 2024-09-17T21:29:40.204 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:29:40.204 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:29:40.204 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:29:40.204 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.204 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.204 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6808", 2024-09-17T21:29:40.204 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:29:40.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.205 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.205 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6809", 2024-09-17T21:29:40.205 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:29:40.205 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.205 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.205 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:29:40.205 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.205 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.205 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6810", 2024-09-17T21:29:40.205 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:29:40.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.206 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.206 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6811", 2024-09-17T21:29:40.206 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:29:40.206 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.206 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.206 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:29:40.206 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.206 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.206 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6814", 2024-09-17T21:29:40.206 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:29:40.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.207 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.207 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6815", 2024-09-17T21:29:40.207 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:29:40.207 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.207 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.207 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:29:40.207 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.207 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.207 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6812", 2024-09-17T21:29:40.207 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:29:40.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.208 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.208 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6813", 2024-09-17T21:29:40.208 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:29:40.208 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.208 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.208 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6809/2337546183", 2024-09-17T21:29:40.208 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6811/2337546183", 2024-09-17T21:29:40.208 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6815/2337546183", 2024-09-17T21:29:40.208 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6813/2337546183", 2024-09-17T21:29:40.208 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:29:40.208 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:29:40.208 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:29:40.209 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.209 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 5, 2024-09-17T21:29:40.209 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "954f801b-3ac9-410a-a375-eb1c533744de", 2024-09-17T21:29:40.209 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:29:40.209 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:29:40.209 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:29:40.209 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:29:40.209 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:29:40.209 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:29:40.209 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:29:40.209 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 20, 2024-09-17T21:29:40.209 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:29:40.210 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:29:40.210 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:29:40.210 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.210 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.210 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6816", 2024-09-17T21:29:40.210 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:29:40.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.210 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.210 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6817", 2024-09-17T21:29:40.210 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:29:40.210 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.211 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.211 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:29:40.211 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.211 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.211 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6818", 2024-09-17T21:29:40.211 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:29:40.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.211 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.211 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6819", 2024-09-17T21:29:40.211 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:29:40.211 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.212 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.212 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:29:40.212 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.212 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.212 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6822", 2024-09-17T21:29:40.212 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:29:40.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.212 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.212 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6823", 2024-09-17T21:29:40.212 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:29:40.213 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.213 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.213 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:29:40.213 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.213 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.213 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6820", 2024-09-17T21:29:40.213 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:29:40.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.213 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.214 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6821", 2024-09-17T21:29:40.214 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:29:40.214 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.214 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.214 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6817/1510320351", 2024-09-17T21:29:40.214 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6819/1510320351", 2024-09-17T21:29:40.214 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6823/1510320351", 2024-09-17T21:29:40.214 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6821/1510320351", 2024-09-17T21:29:40.214 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:29:40.214 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:29:40.215 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:29:40.215 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.215 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 6, 2024-09-17T21:29:40.215 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "21837515-65a2-4f21-96d6-67cfc4589358", 2024-09-17T21:29:40.215 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:29:40.215 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:29:40.215 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:29:40.215 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:29:40.215 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:29:40.215 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:29:40.215 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:29:40.215 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 20, 2024-09-17T21:29:40.216 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:29:40.216 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:29:40.216 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:29:40.216 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.216 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.216 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6824", 2024-09-17T21:29:40.216 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:29:40.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.216 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.216 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6825", 2024-09-17T21:29:40.216 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:29:40.217 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.217 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.217 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:29:40.217 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.217 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.217 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6826", 2024-09-17T21:29:40.217 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:29:40.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.217 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.217 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6827", 2024-09-17T21:29:40.217 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:29:40.217 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.218 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.218 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:29:40.218 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.218 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.218 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6830", 2024-09-17T21:29:40.218 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:29:40.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.218 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.218 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6831", 2024-09-17T21:29:40.218 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:29:40.218 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.219 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.219 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:29:40.219 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.219 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.219 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6828", 2024-09-17T21:29:40.219 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:29:40.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.219 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.219 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6829", 2024-09-17T21:29:40.219 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:29:40.219 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.220 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.220 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6825/471043930", 2024-09-17T21:29:40.220 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6827/471043930", 2024-09-17T21:29:40.220 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6831/471043930", 2024-09-17T21:29:40.220 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6829/471043930", 2024-09-17T21:29:40.220 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:29:40.220 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:29:40.220 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:29:40.220 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.220 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 7, 2024-09-17T21:29:40.221 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "67af7eea-8803-425d-ae50-30a6ab91a711", 2024-09-17T21:29:40.221 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:29:40.221 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:29:40.221 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:29:40.221 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:29:40.221 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:29:40.221 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:29:40.221 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:29:40.221 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 20, 2024-09-17T21:29:40.221 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:29:40.221 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:29:40.221 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:29:40.221 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.222 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.222 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6800", 2024-09-17T21:29:40.222 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:29:40.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.222 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.222 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6801", 2024-09-17T21:29:40.222 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:29:40.222 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.222 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.222 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:29:40.222 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.223 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.223 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6802", 2024-09-17T21:29:40.223 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:29:40.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.223 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.223 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6803", 2024-09-17T21:29:40.223 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:29:40.223 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.223 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.223 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:29:40.223 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.224 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.224 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6806", 2024-09-17T21:29:40.224 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:29:40.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.224 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.224 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6807", 2024-09-17T21:29:40.224 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:29:40.224 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.224 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.224 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:29:40.225 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:40.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.225 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:40.225 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6804", 2024-09-17T21:29:40.225 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:29:40.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.225 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:40.225 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6805", 2024-09-17T21:29:40.225 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:29:40.225 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.225 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.225 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6801/1770619177", 2024-09-17T21:29:40.226 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6803/1770619177", 2024-09-17T21:29:40.226 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6807/1770619177", 2024-09-17T21:29:40.226 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6805/1770619177", 2024-09-17T21:29:40.226 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:29:40.226 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:29:40.226 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:29:40.226 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.226 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.226 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:29:40.226 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_xinfo": [ 2024-09-17T21:29:40.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.226 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 0, 2024-09-17T21:29:40.226 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:29:40.226 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:29:40.226 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:29:40.227 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:29:40.227 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:29:40.227 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:21.445879+0000", 2024-09-17T21:29:40.227 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:29:40.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.227 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 1, 2024-09-17T21:29:40.227 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:29:40.227 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:29:40.227 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:29:40.227 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:29:40.227 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:29:40.227 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.240967+0000", 2024-09-17T21:29:40.227 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:29:40.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.228 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 2, 2024-09-17T21:29:40.228 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:29:40.228 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:29:40.228 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:29:40.228 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:29:40.228 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:29:40.228 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.039097+0000", 2024-09-17T21:29:40.228 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:29:40.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.228 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 3, 2024-09-17T21:29:40.229 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:29:40.229 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:29:40.229 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:29:40.229 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:29:40.229 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:29:40.229 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.372014+0000", 2024-09-17T21:29:40.229 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:29:40.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.229 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 4, 2024-09-17T21:29:40.229 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:29:40.229 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:29:40.229 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:29:40.229 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:29:40.230 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:29:40.230 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.049543+0000", 2024-09-17T21:29:40.230 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:29:40.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.230 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 5, 2024-09-17T21:29:40.230 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:29:40.230 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:29:40.230 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:29:40.230 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:29:40.230 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:29:40.230 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.009384+0000", 2024-09-17T21:29:40.230 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:29:40.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.231 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 6, 2024-09-17T21:29:40.231 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:29:40.231 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:29:40.231 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:29:40.231 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:29:40.231 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:29:40.231 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.048598+0000", 2024-09-17T21:29:40.231 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:29:40.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.231 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 7, 2024-09-17T21:29:40.231 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:29:40.231 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:29:40.232 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:29:40.232 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:29:40.232 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:29:40.232 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.046345+0000", 2024-09-17T21:29:40.232 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:29:40.232 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.232 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:29:40.232 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_upmap": [], 2024-09-17T21:29:40.232 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_upmap_items": [], 2024-09-17T21:29:40.232 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_upmap_primaries": [], 2024-09-17T21:29:40.232 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_temp": [], 2024-09-17T21:29:40.232 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_temp": [], 2024-09-17T21:29:40.232 INFO:teuthology.orchestra.run.smithi129.stdout: "blocklist": {}, 2024-09-17T21:29:40.233 INFO:teuthology.orchestra.run.smithi129.stdout: "range_blocklist": {}, 2024-09-17T21:29:40.233 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profiles": { 2024-09-17T21:29:40.233 INFO:teuthology.orchestra.run.smithi129.stdout: "default": { 2024-09-17T21:29:40.233 INFO:teuthology.orchestra.run.smithi129.stdout: "crush-failure-domain": "osd", 2024-09-17T21:29:40.233 INFO:teuthology.orchestra.run.smithi129.stdout: "k": "2", 2024-09-17T21:29:40.233 INFO:teuthology.orchestra.run.smithi129.stdout: "m": "1", 2024-09-17T21:29:40.233 INFO:teuthology.orchestra.run.smithi129.stdout: "plugin": "jerasure", 2024-09-17T21:29:40.233 INFO:teuthology.orchestra.run.smithi129.stdout: "technique": "reed_sol_van" 2024-09-17T21:29:40.233 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:40.233 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps_queue": [], 2024-09-17T21:29:40.233 INFO:teuthology.orchestra.run.smithi129.stdout: "new_removed_snaps": [], 2024-09-17T21:29:40.233 INFO:teuthology.orchestra.run.smithi129.stdout: "new_purged_snaps": [ 2024-09-17T21:29:40.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.234 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 2, 2024-09-17T21:29:40.234 INFO:teuthology.orchestra.run.smithi129.stdout: "snaps": [ 2024-09-17T21:29:40.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:40.234 INFO:teuthology.orchestra.run.smithi129.stdout: "begin": 2, 2024-09-17T21:29:40.234 INFO:teuthology.orchestra.run.smithi129.stdout: "length": 1 2024-09-17T21:29:40.234 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.234 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:40.234 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.234 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:29:40.234 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_node_flags": {}, 2024-09-17T21:29:40.234 INFO:teuthology.orchestra.run.smithi129.stdout: "device_class_flags": {}, 2024-09-17T21:29:40.234 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_mode": { 2024-09-17T21:29:40.234 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_mode_enabled": false, 2024-09-17T21:29:40.234 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_bucket_count": 0, 2024-09-17T21:29:40.235 INFO:teuthology.orchestra.run.smithi129.stdout: "degraded_stretch_mode": 0, 2024-09-17T21:29:40.235 INFO:teuthology.orchestra.run.smithi129.stdout: "recovering_stretch_mode": 0, 2024-09-17T21:29:40.235 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_mode_bucket": 0 2024-09-17T21:29:40.235 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:40.235 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:40.237 INFO:tasks.thrashosds.thrasher:j is {'epoch': 24, 'fsid': 'f6d3a80c-8a1f-4716-b25d-e97b28890c0d', 'created': '2024-09-17T21:29:12.606637+0000', 'modified': '2024-09-17T21:29:37.389840+0000', 'last_up_change': '2024-09-17T21:29:24.913767+0000', 'last_in_change': '2024-09-17T21:29:15.180249+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T21:29:26.876763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T21:29:31.078466+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'ad067b51-9402-4f69-a0f0-08fbf068dcfc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6800', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6801', 'nonce': 3635367421}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6802', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6803', 'nonce': 3635367421}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6806', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6807', 'nonce': 3635367421}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6804', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6805', 'nonce': 3635367421}]}, 'public_addr': '172.21.15.129:6801/3635367421', 'cluster_addr': '172.21.15.129:6803/3635367421', 'heartbeat_back_addr': '172.21.15.129:6807/3635367421', 'heartbeat_front_addr': '172.21.15.129:6805/3635367421', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '30385fc0-ef5f-45f5-a3ec-37e890d4826e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6818', 'nonce': 176704438}, {'type': 'v1', 'addr': '172.21.15.129:6819', 'nonce': 176704438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6820', 'nonce': 176704438}, {'type': 'v1', 'addr': '172.21.15.129:6821', 'nonce': 176704438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6824', 'nonce': 176704438}, {'type': 'v1', 'addr': '172.21.15.129:6825', 'nonce': 176704438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6822', 'nonce': 176704438}, {'type': 'v1', 'addr': '172.21.15.129:6823', 'nonce': 176704438}]}, 'public_addr': '172.21.15.129:6819/176704438', 'cluster_addr': '172.21.15.129:6821/176704438', 'heartbeat_back_addr': '172.21.15.129:6825/176704438', 'heartbeat_front_addr': '172.21.15.129:6823/176704438', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '76be9b2b-b63f-402a-9ab5-2564bd671ac3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6808', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6809', 'nonce': 1790706405}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6810', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6811', 'nonce': 1790706405}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6814', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6815', 'nonce': 1790706405}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6812', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6813', 'nonce': 1790706405}]}, 'public_addr': '172.21.15.129:6809/1790706405', 'cluster_addr': '172.21.15.129:6811/1790706405', 'heartbeat_back_addr': '172.21.15.129:6815/1790706405', 'heartbeat_front_addr': '172.21.15.129:6813/1790706405', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'd6c34e3b-1ac6-415c-9c56-38ae77a5c9b0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6826', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6827', 'nonce': 393361453}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6828', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6829', 'nonce': 393361453}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6832', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6833', 'nonce': 393361453}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6830', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6831', 'nonce': 393361453}]}, 'public_addr': '172.21.15.129:6827/393361453', 'cluster_addr': '172.21.15.129:6829/393361453', 'heartbeat_back_addr': '172.21.15.129:6833/393361453', 'heartbeat_front_addr': '172.21.15.129:6831/393361453', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '7d722e8c-c497-4f0c-9f61-d9826f8c93d8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6808', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6809', 'nonce': 2337546183}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6810', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6811', 'nonce': 2337546183}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6814', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6815', 'nonce': 2337546183}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6812', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6813', 'nonce': 2337546183}]}, 'public_addr': '172.21.15.171:6809/2337546183', 'cluster_addr': '172.21.15.171:6811/2337546183', 'heartbeat_back_addr': '172.21.15.171:6815/2337546183', 'heartbeat_front_addr': '172.21.15.171:6813/2337546183', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '954f801b-3ac9-410a-a375-eb1c533744de', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6816', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6817', 'nonce': 1510320351}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6818', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6819', 'nonce': 1510320351}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6822', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6823', 'nonce': 1510320351}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6820', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6821', 'nonce': 1510320351}]}, 'public_addr': '172.21.15.171:6817/1510320351', 'cluster_addr': '172.21.15.171:6819/1510320351', 'heartbeat_back_addr': '172.21.15.171:6823/1510320351', 'heartbeat_front_addr': '172.21.15.171:6821/1510320351', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '21837515-65a2-4f21-96d6-67cfc4589358', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6824', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6825', 'nonce': 471043930}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6826', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6827', 'nonce': 471043930}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6830', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6831', 'nonce': 471043930}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6828', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6829', 'nonce': 471043930}]}, 'public_addr': '172.21.15.171:6825/471043930', 'cluster_addr': '172.21.15.171:6827/471043930', 'heartbeat_back_addr': '172.21.15.171:6831/471043930', 'heartbeat_front_addr': '172.21.15.171:6829/471043930', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '67af7eea-8803-425d-ae50-30a6ab91a711', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6800', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6801', 'nonce': 1770619177}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6802', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6803', 'nonce': 1770619177}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6806', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6807', 'nonce': 1770619177}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6804', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6805', 'nonce': 1770619177}]}, 'public_addr': '172.21.15.171:6801/1770619177', 'cluster_addr': '172.21.15.171:6803/1770619177', 'heartbeat_back_addr': '172.21.15.171:6807/1770619177', 'heartbeat_front_addr': '172.21.15.171:6805/1770619177', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:21.445879+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.240967+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.039097+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.372014+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.049543+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.009384+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.048598+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.046345+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [{'pool': 2, 'snaps': [{'begin': 2, 'length': 1}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T21:29:40.237 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-17T21:29:40.238 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:40.236+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:40.257 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:40.257 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:40.266 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:29:40.350 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:40.349+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:29:40.410 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:40.410 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:40.419 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:29:40.459 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:40.458+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:40.461 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is set 2024-09-17T21:29:40.541 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:40.539+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:40.565 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:40.565 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:40.574 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:29:40.642 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:40.639+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:40.676 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:40.677 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:40.685 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:29:40.742 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:40.741+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:40.788 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:40.788 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:40.797 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:29:40.882 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:40.880+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:40.941 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:40.942 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:40.944 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:40.942+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:40.950 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:29:41.045 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:41.043+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:41.093 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:41.094 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:41.102 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:29:41.146 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:41.143+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:41.205 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:41.205 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:41.214 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:29:41.258 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:41.257+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:29:41.317 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:41.318 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:41.326 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:29:41.347 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:41.345+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:41.448 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:41.447+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:41.478 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:41.478 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:41.489 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:29:41.550 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:41.548+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:41.602 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:41.602 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:41.611 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:29:41.650 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:41.649+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:41.714 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:41.714 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:41.723 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:29:41.766 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:41.763+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:41.825 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:41.825 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:41.833 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:29:41.874 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:41.871+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:41.953 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:41.951+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:41.977 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:41.977 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:41.986 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:29:42.054 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:42.052+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:29:42.088 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:42.088 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:42.096 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:29:42.155 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:42.153+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:29:42.198 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:42.198 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:42.207 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:29:42.236 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:29:42.280 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:42.279+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:42.309 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:42.309 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:42.317 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:29:42.357 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:42.355+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:42.364 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:29:42.365 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:29:42.381 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:29:42.381 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:29:42.397 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:29:42.397 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:29:42.413 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:29:42.413 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:29:42.420 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:42.420 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:42.429 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:29:42.429 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:29:42.429 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:29:42.445 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:29:42.446 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:29:42.461 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:29:42.462 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:29:42.473 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:42.470+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:42.478 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:29:42.481 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:29:42.481 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:29:42.533 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:42.534 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:29:42.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:29:42.534 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [] 2024-09-17T21:29:42.534 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:42.542 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:29:42.583 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:42.580+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:42.660 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:42.656+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:42.686 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:42.686 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:29:42.686 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:29:42.686 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:42.694 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:29:42.778 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:42.775+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:42.837 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:42.837 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:29:42.837 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:29:42.837 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:29:42.837 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:42.846 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:29:42.887 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:42.885+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:42.962 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:42.959+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:42.992 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:42.992 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:29:42.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:29:42.992 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:29:42.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:42.992 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:42.993 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:29:42.993 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.049024527, 2024-09-17T21:29:42.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61572000699999996, 2024-09-17T21:29:42.993 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:42.993 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:42.993 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:42.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:42.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:42.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:42.993 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T21:29:42.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:42.993 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:42.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:42.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:42.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:29:42.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:42.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:42.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:42.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:42.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:29:42.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:42.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:42.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:42.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:42.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939745+0000", 2024-09-17T21:29:42.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6899999999999999e-06 2024-09-17T21:29:42.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:42.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:42.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:42.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939753+0000", 2024-09-17T21:29:42.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1950000000000003e-06 2024-09-17T21:29:42.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:42.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:42.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:42.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939767+0000", 2024-09-17T21:29:42.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3429e-05 2024-09-17T21:29:42.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:42.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:42.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:42.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939776+0000", 2024-09-17T21:29:42.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:29:42.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:42.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:42.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:42.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939858+0000", 2024-09-17T21:29:42.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2496000000000004e-05 2024-09-17T21:29:42.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:42.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:42.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T21:29:42.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939868+0000", 2024-09-17T21:29:42.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1910000000000006e-06 2024-09-17T21:29:42.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:42.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:42.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:42.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940612+0000", 2024-09-17T21:29:42.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074429599999999998 2024-09-17T21:29:42.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:42.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:42.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:29:42.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940621+0000", 2024-09-17T21:29:42.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9139999999999997e-06 2024-09-17T21:29:42.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:42.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:42.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:42.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554006+0000", 2024-09-17T21:29:42.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61338505099999996 2024-09-17T21:29:42.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:42.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:42.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:42.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554166+0000", 2024-09-17T21:29:42.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015982600000000001 2024-09-17T21:29:42.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:42.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:42.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:42.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554296+0000", 2024-09-17T21:29:42.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013002399999999999 2024-09-17T21:29:42.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:42.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:42.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:42.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554936+0000", 2024-09-17T21:29:42.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064009400000000001 2024-09-17T21:29:42.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:42.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:42.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:42.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555389+0000", 2024-09-17T21:29:42.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045355200000000002 2024-09-17T21:29:42.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:42.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:42.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:42.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555432+0000", 2024-09-17T21:29:42.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2477e-05 2024-09-17T21:29:42.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:42.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:42.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:42.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555463+0000", 2024-09-17T21:29:42.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1612000000000002e-05 2024-09-17T21:29:42.999 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:42.999 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:42.999 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:42.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:42.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.000 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.000 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.685889+0000", 2024-09-17T21:29:43.000 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.302879187, 2024-09-17T21:29:43.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0095986289999999991, 2024-09-17T21:29:43.000 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.000 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.000 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.000 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.000 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.000 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.000 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 47 2024-09-17T21:29:43.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.000 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.685889+0000", 2024-09-17T21:29:43.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.685889+0000", 2024-09-17T21:29:43.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.685889+0000", 2024-09-17T21:29:43.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1600000000000001e-07 2024-09-17T21:29:43.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.685955+0000", 2024-09-17T21:29:43.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5702000000000002e-05 2024-09-17T21:29:43.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.685966+0000", 2024-09-17T21:29:43.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0497e-05 2024-09-17T21:29:43.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.685973+0000", 2024-09-17T21:29:43.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2710000000000003e-06 2024-09-17T21:29:43.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.690062+0000", 2024-09-17T21:29:43.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040890789999999998 2024-09-17T21:29:43.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.690111+0000", 2024-09-17T21:29:43.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9332999999999997e-05 2024-09-17T21:29:43.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.690205+0000", 2024-09-17T21:29:43.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4257000000000002e-05 2024-09-17T21:29:43.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.693126+0000", 2024-09-17T21:29:43.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029209729999999999 2024-09-17T21:29:43.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.695391+0000", 2024-09-17T21:29:43.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022643820000000001 2024-09-17T21:29:43.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.695448+0000", 2024-09-17T21:29:43.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6916000000000002e-05 2024-09-17T21:29:43.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.695487+0000", 2024-09-17T21:29:43.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9603e-05 2024-09-17T21:29:43.005 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.005 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.005 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.005 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.005 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.685988+0000", 2024-09-17T21:29:43.005 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.302780339, 2024-09-17T21:29:43.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0096066710000000007, 2024-09-17T21:29:43.005 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.006 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.006 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.006 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.006 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.006 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.006 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48 2024-09-17T21:29:43.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.006 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.685988+0000", 2024-09-17T21:29:43.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.685988+0000", 2024-09-17T21:29:43.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.685988+0000", 2024-09-17T21:29:43.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1200000000000002e-07 2024-09-17T21:29:43.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.685993+0000", 2024-09-17T21:29:43.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4299999999999999e-06 2024-09-17T21:29:43.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686000+0000", 2024-09-17T21:29:43.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4030000000000001e-06 2024-09-17T21:29:43.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686007+0000", 2024-09-17T21:29:43.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:29:43.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.690669+0000", 2024-09-17T21:29:43.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0046618299999999996 2024-09-17T21:29:43.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.690719+0000", 2024-09-17T21:29:43.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0389000000000002e-05 2024-09-17T21:29:43.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.690798+0000", 2024-09-17T21:29:43.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9135999999999996e-05 2024-09-17T21:29:43.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.693463+0000", 2024-09-17T21:29:43.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0026646650000000001 2024-09-17T21:29:43.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.695533+0000", 2024-09-17T21:29:43.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020697210000000001 2024-09-17T21:29:43.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.695569+0000", 2024-09-17T21:29:43.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6217000000000002e-05 2024-09-17T21:29:43.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.695594+0000", 2024-09-17T21:29:43.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5442000000000001e-05 2024-09-17T21:29:43.010 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.010 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.010 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.011 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.011 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.686019+0000", 2024-09-17T21:29:43.011 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.302749283000001, 2024-09-17T21:29:43.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0096819560000000002, 2024-09-17T21:29:43.011 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.011 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.011 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.011 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.011 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.011 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.011 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49 2024-09-17T21:29:43.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.011 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686019+0000", 2024-09-17T21:29:43.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686019+0000", 2024-09-17T21:29:43.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686019+0000", 2024-09-17T21:29:43.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6199999999999998e-07 2024-09-17T21:29:43.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686023+0000", 2024-09-17T21:29:43.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8079999999999998e-06 2024-09-17T21:29:43.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686042+0000", 2024-09-17T21:29:43.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8885e-05 2024-09-17T21:29:43.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686048+0000", 2024-09-17T21:29:43.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.195e-06 2024-09-17T21:29:43.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.691114+0000", 2024-09-17T21:29:43.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0050663799999999997 2024-09-17T21:29:43.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.691166+0000", 2024-09-17T21:29:43.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1430000000000001e-05 2024-09-17T21:29:43.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.691247+0000", 2024-09-17T21:29:43.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1225999999999998e-05 2024-09-17T21:29:43.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.693511+0000", 2024-09-17T21:29:43.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022640310000000001 2024-09-17T21:29:43.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.695635+0000", 2024-09-17T21:29:43.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002123888 2024-09-17T21:29:43.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.695664+0000", 2024-09-17T21:29:43.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9280000000000001e-05 2024-09-17T21:29:43.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.695701+0000", 2024-09-17T21:29:43.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6371000000000001e-05 2024-09-17T21:29:43.016 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.016 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.016 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.016 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.016 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.686061+0000", 2024-09-17T21:29:43.016 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.302707051000001, 2024-09-17T21:29:43.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0097187909999999992, 2024-09-17T21:29:43.017 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.017 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.017 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.017 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.017 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.017 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.017 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 50 2024-09-17T21:29:43.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.017 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686061+0000", 2024-09-17T21:29:43.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686061+0000", 2024-09-17T21:29:43.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686061+0000", 2024-09-17T21:29:43.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3099999999999997e-07 2024-09-17T21:29:43.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686115+0000", 2024-09-17T21:29:43.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3539999999999999e-05 2024-09-17T21:29:43.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686130+0000", 2024-09-17T21:29:43.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5339e-05 2024-09-17T21:29:43.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686147+0000", 2024-09-17T21:29:43.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7027000000000001e-05 2024-09-17T21:29:43.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.691526+0000", 2024-09-17T21:29:43.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053781200000000001 2024-09-17T21:29:43.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.691575+0000", 2024-09-17T21:29:43.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9410000000000003e-05 2024-09-17T21:29:43.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.691656+0000", 2024-09-17T21:29:43.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1290000000000003e-05 2024-09-17T21:29:43.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.693524+0000", 2024-09-17T21:29:43.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018678060000000001 2024-09-17T21:29:43.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.695728+0000", 2024-09-17T21:29:43.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002203736 2024-09-17T21:29:43.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.695756+0000", 2024-09-17T21:29:43.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8402e-05 2024-09-17T21:29:43.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.695780+0000", 2024-09-17T21:29:43.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.349e-05 2024-09-17T21:29:43.022 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.022 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.022 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.022 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.022 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.686167+0000", 2024-09-17T21:29:43.022 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.302601165, 2024-09-17T21:29:43.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011824993000000001, 2024-09-17T21:29:43.022 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.023 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.023 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.023 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.023 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.023 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.023 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 51 2024-09-17T21:29:43.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.023 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686167+0000", 2024-09-17T21:29:43.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686167+0000", 2024-09-17T21:29:43.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686167+0000", 2024-09-17T21:29:43.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1099999999999995e-07 2024-09-17T21:29:43.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686172+0000", 2024-09-17T21:29:43.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.34e-06 2024-09-17T21:29:43.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686179+0000", 2024-09-17T21:29:43.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4309999999999997e-06 2024-09-17T21:29:43.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686191+0000", 2024-09-17T21:29:43.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2052000000000001e-05 2024-09-17T21:29:43.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.692018+0000", 2024-09-17T21:29:43.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0058268 2024-09-17T21:29:43.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.692077+0000", 2024-09-17T21:29:43.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9190000000000001e-05 2024-09-17T21:29:43.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.692158+0000", 2024-09-17T21:29:43.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0792999999999997e-05 2024-09-17T21:29:43.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.693536+0000", 2024-09-17T21:29:43.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013777819999999999 2024-09-17T21:29:43.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.697931+0000", 2024-09-17T21:29:43.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004394813 2024-09-17T21:29:43.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.697965+0000", 2024-09-17T21:29:43.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4774000000000003e-05 2024-09-17T21:29:43.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.697992+0000", 2024-09-17T21:29:43.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6307000000000001e-05 2024-09-17T21:29:43.027 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.028 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.028 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.028 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.028 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.686203+0000", 2024-09-17T21:29:43.028 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.302565032, 2024-09-17T21:29:43.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011895525000000001, 2024-09-17T21:29:43.028 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.028 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.028 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.028 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.028 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.028 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.029 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 52 2024-09-17T21:29:43.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.029 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686203+0000", 2024-09-17T21:29:43.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686203+0000", 2024-09-17T21:29:43.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686203+0000", 2024-09-17T21:29:43.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.68e-07 2024-09-17T21:29:43.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686214+0000", 2024-09-17T21:29:43.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0896999999999999e-05 2024-09-17T21:29:43.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686224+0000", 2024-09-17T21:29:43.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5119999999999997e-06 2024-09-17T21:29:43.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.686237+0000", 2024-09-17T21:29:43.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3613e-05 2024-09-17T21:29:43.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.692424+0000", 2024-09-17T21:29:43.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061866999999999998 2024-09-17T21:29:43.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.692474+0000", 2024-09-17T21:29:43.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0077999999999997e-05 2024-09-17T21:29:43.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.692557+0000", 2024-09-17T21:29:43.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2336000000000006e-05 2024-09-17T21:29:43.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.693541+0000", 2024-09-17T21:29:43.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00098429799999999999 2024-09-17T21:29:43.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.698027+0000", 2024-09-17T21:29:43.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044866209999999997 2024-09-17T21:29:43.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.698060+0000", 2024-09-17T21:29:43.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2762999999999998e-05 2024-09-17T21:29:43.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.698098+0000", 2024-09-17T21:29:43.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8139000000000002e-05 2024-09-17T21:29:43.033 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.033 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.033 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.033 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:64 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.034 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.913431+0000", 2024-09-17T21:29:43.034 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.075337282, 2024-09-17T21:29:43.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0086774409999999993, 2024-09-17T21:29:43.034 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.034 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.034 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.034 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.034 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.034 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.034 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 64 2024-09-17T21:29:43.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.034 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913431+0000", 2024-09-17T21:29:43.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913431+0000", 2024-09-17T21:29:43.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913431+0000", 2024-09-17T21:29:43.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5099999999999999e-07 2024-09-17T21:29:43.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913437+0000", 2024-09-17T21:29:43.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5339999999999999e-06 2024-09-17T21:29:43.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913444+0000", 2024-09-17T21:29:43.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3159999999999999e-06 2024-09-17T21:29:43.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913448+0000", 2024-09-17T21:29:43.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.985e-06 2024-09-17T21:29:43.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913482+0000", 2024-09-17T21:29:43.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3796e-05 2024-09-17T21:29:43.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913524+0000", 2024-09-17T21:29:43.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2490999999999997e-05 2024-09-17T21:29:43.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913627+0000", 2024-09-17T21:29:43.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010237499999999999 2024-09-17T21:29:43.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.916093+0000", 2024-09-17T21:29:43.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002465826 2024-09-17T21:29:43.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922052+0000", 2024-09-17T21:29:43.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059588130000000003 2024-09-17T21:29:43.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922083+0000", 2024-09-17T21:29:43.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1819000000000001e-05 2024-09-17T21:29:43.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922108+0000", 2024-09-17T21:29:43.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4735e-05 2024-09-17T21:29:43.039 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.039 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.039 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.039 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:65 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.039 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.913466+0000", 2024-09-17T21:29:43.039 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.075302287, 2024-09-17T21:29:43.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0087604399999999995, 2024-09-17T21:29:43.040 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.040 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.040 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.040 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.040 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.040 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.040 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65 2024-09-17T21:29:43.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.040 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913466+0000", 2024-09-17T21:29:43.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913466+0000", 2024-09-17T21:29:43.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913466+0000", 2024-09-17T21:29:43.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:29:43.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913469+0000", 2024-09-17T21:29:43.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5270000000000001e-06 2024-09-17T21:29:43.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913476+0000", 2024-09-17T21:29:43.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9609999999999999e-06 2024-09-17T21:29:43.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913481+0000", 2024-09-17T21:29:43.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1499999999999998e-06 2024-09-17T21:29:43.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914036+0000", 2024-09-17T21:29:43.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00055487899999999998 2024-09-17T21:29:43.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914126+0000", 2024-09-17T21:29:43.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0481000000000003e-05 2024-09-17T21:29:43.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914227+0000", 2024-09-17T21:29:43.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010136699999999999 2024-09-17T21:29:43.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.916098+0000", 2024-09-17T21:29:43.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018710930000000001 2024-09-17T21:29:43.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922157+0000", 2024-09-17T21:29:43.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060583740000000001 2024-09-17T21:29:43.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922189+0000", 2024-09-17T21:29:43.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1717000000000001e-05 2024-09-17T21:29:43.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922226+0000", 2024-09-17T21:29:43.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7472999999999998e-05 2024-09-17T21:29:43.045 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.045 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.045 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.045 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:66 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.045 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.913556+0000", 2024-09-17T21:29:43.045 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.075212274, 2024-09-17T21:29:43.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0087585830000000003, 2024-09-17T21:29:43.045 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.045 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.046 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.046 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.046 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.046 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.046 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 66 2024-09-17T21:29:43.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.046 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913556+0000", 2024-09-17T21:29:43.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913556+0000", 2024-09-17T21:29:43.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913556+0000", 2024-09-17T21:29:43.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2300000000000001e-07 2024-09-17T21:29:43.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913624+0000", 2024-09-17T21:29:43.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7421000000000004e-05 2024-09-17T21:29:43.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913631+0000", 2024-09-17T21:29:43.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5279999999999998e-06 2024-09-17T21:29:43.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913635+0000", 2024-09-17T21:29:43.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8800000000000001e-06 2024-09-17T21:29:43.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914514+0000", 2024-09-17T21:29:43.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000878676 2024-09-17T21:29:43.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914562+0000", 2024-09-17T21:29:43.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8109000000000003e-05 2024-09-17T21:29:43.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914636+0000", 2024-09-17T21:29:43.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4435000000000001e-05 2024-09-17T21:29:43.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.919084+0000", 2024-09-17T21:29:43.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044479669999999997 2024-09-17T21:29:43.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922265+0000", 2024-09-17T21:29:43.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031811449999999998 2024-09-17T21:29:43.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922293+0000", 2024-09-17T21:29:43.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7838999999999999e-05 2024-09-17T21:29:43.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922314+0000", 2024-09-17T21:29:43.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1060000000000002e-05 2024-09-17T21:29:43.051 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.051 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.051 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.051 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:67 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.051 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.913644+0000", 2024-09-17T21:29:43.051 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.075123729, 2024-09-17T21:29:43.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010343946, 2024-09-17T21:29:43.051 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.051 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.051 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.051 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.051 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.051 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.052 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 67 2024-09-17T21:29:43.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.052 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913644+0000", 2024-09-17T21:29:43.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913644+0000", 2024-09-17T21:29:43.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913645+0000", 2024-09-17T21:29:43.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T21:29:43.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913647+0000", 2024-09-17T21:29:43.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3939999999999999e-06 2024-09-17T21:29:43.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913651+0000", 2024-09-17T21:29:43.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9349999999999996e-06 2024-09-17T21:29:43.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913655+0000", 2024-09-17T21:29:43.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0679999999999998e-06 2024-09-17T21:29:43.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.915321+0000", 2024-09-17T21:29:43.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016660970000000001 2024-09-17T21:29:43.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.915368+0000", 2024-09-17T21:29:43.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6542999999999998e-05 2024-09-17T21:29:43.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.915443+0000", 2024-09-17T21:29:43.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4907000000000006e-05 2024-09-17T21:29:43.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.921543+0000", 2024-09-17T21:29:43.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061008570000000003 2024-09-17T21:29:43.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.923927+0000", 2024-09-17T21:29:43.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002383614 2024-09-17T21:29:43.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.923964+0000", 2024-09-17T21:29:43.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7144000000000002e-05 2024-09-17T21:29:43.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.923988+0000", 2024-09-17T21:29:43.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3938e-05 2024-09-17T21:29:43.056 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.056 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.056 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.057 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.057 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.913717+0000", 2024-09-17T21:29:43.057 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.075051191, 2024-09-17T21:29:43.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010388189000000001, 2024-09-17T21:29:43.057 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.057 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.057 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.057 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.057 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.057 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.057 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 68 2024-09-17T21:29:43.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.057 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913717+0000", 2024-09-17T21:29:43.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913717+0000", 2024-09-17T21:29:43.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913717+0000", 2024-09-17T21:29:43.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1699999999999998e-07 2024-09-17T21:29:43.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913720+0000", 2024-09-17T21:29:43.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.943e-06 2024-09-17T21:29:43.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913726+0000", 2024-09-17T21:29:43.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T21:29:43.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913729+0000", 2024-09-17T21:29:43.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7359999999999999e-06 2024-09-17T21:29:43.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.915710+0000", 2024-09-17T21:29:43.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019803300000000002 2024-09-17T21:29:43.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.915756+0000", 2024-09-17T21:29:43.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6195000000000001e-05 2024-09-17T21:29:43.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.915852+0000", 2024-09-17T21:29:43.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6241000000000002e-05 2024-09-17T21:29:43.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922119+0000", 2024-09-17T21:29:43.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062663989999999998 2024-09-17T21:29:43.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.924033+0000", 2024-09-17T21:29:43.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00191442 2024-09-17T21:29:43.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.924070+0000", 2024-09-17T21:29:43.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6990000000000003e-05 2024-09-17T21:29:43.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.924105+0000", 2024-09-17T21:29:43.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4969999999999999e-05 2024-09-17T21:29:43.062 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.062 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.062 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.062 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.062 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.913762+0000", 2024-09-17T21:29:43.063 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.075005463, 2024-09-17T21:29:43.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012579095, 2024-09-17T21:29:43.063 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.063 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.063 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.063 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.063 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.063 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.063 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 69 2024-09-17T21:29:43.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.063 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913762+0000", 2024-09-17T21:29:43.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913762+0000", 2024-09-17T21:29:43.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913763+0000", 2024-09-17T21:29:43.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:29:43.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913843+0000", 2024-09-17T21:29:43.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9771999999999994e-05 2024-09-17T21:29:43.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913851+0000", 2024-09-17T21:29:43.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3280000000000006e-06 2024-09-17T21:29:43.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913855+0000", 2024-09-17T21:29:43.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7799999999999998e-06 2024-09-17T21:29:43.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.916131+0000", 2024-09-17T21:29:43.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022758880000000002 2024-09-17T21:29:43.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.916179+0000", 2024-09-17T21:29:43.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8288999999999998e-05 2024-09-17T21:29:43.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.916254+0000", 2024-09-17T21:29:43.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4479000000000003e-05 2024-09-17T21:29:43.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922323+0000", 2024-09-17T21:29:43.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060697659999999999 2024-09-17T21:29:43.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.926268+0000", 2024-09-17T21:29:43.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0039449860000000002 2024-09-17T21:29:43.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.926308+0000", 2024-09-17T21:29:43.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0102000000000003e-05 2024-09-17T21:29:43.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.926342+0000", 2024-09-17T21:29:43.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3178e-05 2024-09-17T21:29:43.068 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.068 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.068 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.068 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.068 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.913863+0000", 2024-09-17T21:29:43.068 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.074905133, 2024-09-17T21:29:43.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012581904, 2024-09-17T21:29:43.068 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.068 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.069 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.069 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 70 2024-09-17T21:29:43.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.069 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913863+0000", 2024-09-17T21:29:43.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913863+0000", 2024-09-17T21:29:43.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913863+0000", 2024-09-17T21:29:43.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.34e-07 2024-09-17T21:29:43.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913865+0000", 2024-09-17T21:29:43.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2199999999999999e-06 2024-09-17T21:29:43.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913874+0000", 2024-09-17T21:29:43.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0729999999999994e-06 2024-09-17T21:29:43.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913878+0000", 2024-09-17T21:29:43.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.411e-06 2024-09-17T21:29:43.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.918727+0000", 2024-09-17T21:29:43.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004848832 2024-09-17T21:29:43.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.918776+0000", 2024-09-17T21:29:43.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9743999999999997e-05 2024-09-17T21:29:43.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.918855+0000", 2024-09-17T21:29:43.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8900999999999996e-05 2024-09-17T21:29:43.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922327+0000", 2024-09-17T21:29:43.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034711730000000001 2024-09-17T21:29:43.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.926387+0000", 2024-09-17T21:29:43.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040604589999999998 2024-09-17T21:29:43.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.926419+0000", 2024-09-17T21:29:43.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.222e-05 2024-09-17T21:29:43.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.926445+0000", 2024-09-17T21:29:43.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5437000000000001e-05 2024-09-17T21:29:43.074 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.074 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.074 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.074 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.074 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.913883+0000", 2024-09-17T21:29:43.074 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.074884804, 2024-09-17T21:29:43.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012651625999999999, 2024-09-17T21:29:43.074 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.074 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.074 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.074 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.075 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.075 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.075 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 71 2024-09-17T21:29:43.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.075 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913883+0000", 2024-09-17T21:29:43.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913883+0000", 2024-09-17T21:29:43.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913883+0000", 2024-09-17T21:29:43.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T21:29:43.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913885+0000", 2024-09-17T21:29:43.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.013e-06 2024-09-17T21:29:43.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913891+0000", 2024-09-17T21:29:43.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1780000000000002e-06 2024-09-17T21:29:43.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913897+0000", 2024-09-17T21:29:43.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:29:43.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.919117+0000", 2024-09-17T21:29:43.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052191080000000001 2024-09-17T21:29:43.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.919165+0000", 2024-09-17T21:29:43.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8386000000000001e-05 2024-09-17T21:29:43.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.919242+0000", 2024-09-17T21:29:43.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7541999999999997e-05 2024-09-17T21:29:43.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922330+0000", 2024-09-17T21:29:43.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030875849999999999 2024-09-17T21:29:43.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.926474+0000", 2024-09-17T21:29:43.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004144043 2024-09-17T21:29:43.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.926502+0000", 2024-09-17T21:29:43.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7736999999999999e-05 2024-09-17T21:29:43.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.926535+0000", 2024-09-17T21:29:43.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2911999999999999e-05 2024-09-17T21:29:43.080 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.080 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.080 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.080 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.080 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.913908+0000", 2024-09-17T21:29:43.080 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.074859515, 2024-09-17T21:29:43.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015054164, 2024-09-17T21:29:43.080 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.080 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.080 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.080 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.081 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.081 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.081 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 72 2024-09-17T21:29:43.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.081 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913908+0000", 2024-09-17T21:29:43.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913908+0000", 2024-09-17T21:29:43.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913909+0000", 2024-09-17T21:29:43.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.77e-07 2024-09-17T21:29:43.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913945+0000", 2024-09-17T21:29:43.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5941e-05 2024-09-17T21:29:43.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913951+0000", 2024-09-17T21:29:43.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2319999999999998e-06 2024-09-17T21:29:43.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913954+0000", 2024-09-17T21:29:43.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000001e-06 2024-09-17T21:29:43.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.919503+0000", 2024-09-17T21:29:43.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00554887 2024-09-17T21:29:43.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.919547+0000", 2024-09-17T21:29:43.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3878000000000002e-05 2024-09-17T21:29:43.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.919620+0000", 2024-09-17T21:29:43.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3409000000000001e-05 2024-09-17T21:29:43.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.923380+0000", 2024-09-17T21:29:43.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037595060000000001 2024-09-17T21:29:43.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.928914+0000", 2024-09-17T21:29:43.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055341640000000003 2024-09-17T21:29:43.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.928943+0000", 2024-09-17T21:29:43.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8467999999999999e-05 2024-09-17T21:29:43.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.928963+0000", 2024-09-17T21:29:43.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9981000000000001e-05 2024-09-17T21:29:43.086 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.086 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.086 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.086 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.086 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.913962+0000", 2024-09-17T21:29:43.086 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.074805842, 2024-09-17T21:29:43.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015116418, 2024-09-17T21:29:43.086 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.086 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.086 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.087 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.087 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.087 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.087 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 73 2024-09-17T21:29:43.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.087 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913962+0000", 2024-09-17T21:29:43.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913962+0000", 2024-09-17T21:29:43.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913962+0000", 2024-09-17T21:29:43.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5400000000000002e-07 2024-09-17T21:29:43.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913965+0000", 2024-09-17T21:29:43.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6259999999999999e-06 2024-09-17T21:29:43.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913970+0000", 2024-09-17T21:29:43.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9459999999999997e-06 2024-09-17T21:29:43.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913975+0000", 2024-09-17T21:29:43.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.544e-06 2024-09-17T21:29:43.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.921586+0000", 2024-09-17T21:29:43.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0076114549999999996 2024-09-17T21:29:43.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.921653+0000", 2024-09-17T21:29:43.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7417999999999997e-05 2024-09-17T21:29:43.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.921737+0000", 2024-09-17T21:29:43.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3856000000000005e-05 2024-09-17T21:29:43.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.923386+0000", 2024-09-17T21:29:43.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001648654 2024-09-17T21:29:43.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.929009+0000", 2024-09-17T21:29:43.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056226610000000002 2024-09-17T21:29:43.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.929040+0000", 2024-09-17T21:29:43.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1622999999999999e-05 2024-09-17T21:29:43.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.929078+0000", 2024-09-17T21:29:43.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8281000000000002e-05 2024-09-17T21:29:43.091 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.091 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.092 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.092 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.092 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.913984+0000", 2024-09-17T21:29:43.092 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.074783821, 2024-09-17T21:29:43.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015173944, 2024-09-17T21:29:43.092 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.092 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.092 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.092 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.092 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.092 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.093 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 74 2024-09-17T21:29:43.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.093 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913984+0000", 2024-09-17T21:29:43.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913984+0000", 2024-09-17T21:29:43.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913985+0000", 2024-09-17T21:29:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3099999999999998e-07 2024-09-17T21:29:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913988+0000", 2024-09-17T21:29:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3739999999999998e-06 2024-09-17T21:29:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.913997+0000", 2024-09-17T21:29:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9600000000000006e-06 2024-09-17T21:29:43.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914003+0000", 2024-09-17T21:29:43.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7490000000000004e-06 2024-09-17T21:29:43.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922360+0000", 2024-09-17T21:29:43.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0083569130000000005 2024-09-17T21:29:43.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922409+0000", 2024-09-17T21:29:43.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9724e-05 2024-09-17T21:29:43.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922488+0000", 2024-09-17T21:29:43.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8237000000000003e-05 2024-09-17T21:29:43.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.923389+0000", 2024-09-17T21:29:43.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090156100000000005 2024-09-17T21:29:43.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.929113+0000", 2024-09-17T21:29:43.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057240030000000001 2024-09-17T21:29:43.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.929139+0000", 2024-09-17T21:29:43.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6024999999999999e-05 2024-09-17T21:29:43.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.929158+0000", 2024-09-17T21:29:43.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8867000000000001e-05 2024-09-17T21:29:43.098 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.098 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.098 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.098 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.098 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.914037+0000", 2024-09-17T21:29:43.098 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.074731149, 2024-09-17T21:29:43.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015214011, 2024-09-17T21:29:43.098 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.098 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.098 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.099 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.099 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 75 2024-09-17T21:29:43.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.099 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914037+0000", 2024-09-17T21:29:43.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914037+0000", 2024-09-17T21:29:43.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914037+0000", 2024-09-17T21:29:43.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6100000000000001e-07 2024-09-17T21:29:43.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914087+0000", 2024-09-17T21:29:43.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0083999999999997e-05 2024-09-17T21:29:43.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914097+0000", 2024-09-17T21:29:43.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7550000000000006e-06 2024-09-17T21:29:43.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914107+0000", 2024-09-17T21:29:43.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0006e-05 2024-09-17T21:29:43.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922739+0000", 2024-09-17T21:29:43.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0086314010000000004 2024-09-17T21:29:43.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922786+0000", 2024-09-17T21:29:43.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7392000000000003e-05 2024-09-17T21:29:43.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.922869+0000", 2024-09-17T21:29:43.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2719000000000005e-05 2024-09-17T21:29:43.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.923813+0000", 2024-09-17T21:29:43.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00094400199999999999 2024-09-17T21:29:43.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.929187+0000", 2024-09-17T21:29:43.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053740790000000004 2024-09-17T21:29:43.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.929217+0000", 2024-09-17T21:29:43.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0514e-05 2024-09-17T21:29:43.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.929251+0000", 2024-09-17T21:29:43.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3497999999999997e-05 2024-09-17T21:29:43.104 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.104 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.104 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.104 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:43.104 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:28.914124+0000", 2024-09-17T21:29:43.104 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.074643527999999, 2024-09-17T21:29:43.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017437057999999998, 2024-09-17T21:29:43.104 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:43.104 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:43.104 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.105 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:43.105 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:43.105 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:43.105 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 76 2024-09-17T21:29:43.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.105 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:43.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:43.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914124+0000", 2024-09-17T21:29:43.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:43.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914124+0000", 2024-09-17T21:29:43.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:43.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:43.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914125+0000", 2024-09-17T21:29:43.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1699999999999998e-07 2024-09-17T21:29:43.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:43.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914129+0000", 2024-09-17T21:29:43.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3080000000000001e-06 2024-09-17T21:29:43.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:43.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914143+0000", 2024-09-17T21:29:43.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3998999999999999e-05 2024-09-17T21:29:43.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.914151+0000", 2024-09-17T21:29:43.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7589999999999997e-06 2024-09-17T21:29:43.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:43.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.923419+0000", 2024-09-17T21:29:43.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0092675029999999999 2024-09-17T21:29:43.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:43.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.923465+0000", 2024-09-17T21:29:43.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6660999999999999e-05 2024-09-17T21:29:43.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:43.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.923554+0000", 2024-09-17T21:29:43.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9015000000000001e-05 2024-09-17T21:29:43.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:43.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.924118+0000", 2024-09-17T21:29:43.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00056390399999999995 2024-09-17T21:29:43.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:43.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.931509+0000", 2024-09-17T21:29:43.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073904699999999997 2024-09-17T21:29:43.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:43.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.931539+0000", 2024-09-17T21:29:43.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0663000000000001e-05 2024-09-17T21:29:43.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:43.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:43.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:43.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.931561+0000", 2024-09-17T21:29:43.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2158999999999999e-05 2024-09-17T21:29:43.109 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.109 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.109 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.110 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:43.110 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:43.110 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:43.111 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:29:43.112 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:43.109+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:43.212 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:43.210+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:43.215 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:43.215 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:43.215 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:29:43.215 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:43.224 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:29:43.313 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:43.311+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:29:43.326 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:43.327 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:43.327 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:29:43.327 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:29:43.327 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:43.336 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:29:43.414 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:43.411+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:43.475 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is set 2024-09-17T21:29:43.488 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:29:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:29:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:29:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:47 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'29, mlcod=18'25)", 2024-09-17T21:29:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.690276+0000", 2024-09-17T21:29:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.794053099999999, 2024-09-17T21:29:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0049232540000000002, 2024-09-17T21:29:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.690276+0000", 2024-09-17T21:29:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.690276+0000", 2024-09-17T21:29:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.690276+0000", 2024-09-17T21:29:43.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:29:43.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.690337+0000", 2024-09-17T21:29:43.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0927000000000001e-05 2024-09-17T21:29:43.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.690345+0000", 2024-09-17T21:29:43.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4560000000000003e-06 2024-09-17T21:29:43.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.690355+0000", 2024-09-17T21:29:43.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7270000000000002e-06 2024-09-17T21:29:43.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.690400+0000", 2024-09-17T21:29:43.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5454000000000002e-05 2024-09-17T21:29:43.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.690413+0000", 2024-09-17T21:29:43.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3189e-05 2024-09-17T21:29:43.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.695162+0000", 2024-09-17T21:29:43.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0047492539999999996 2024-09-17T21:29:43.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.695199+0000", 2024-09-17T21:29:43.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6887000000000001e-05 2024-09-17T21:29:43.493 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.493 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.494 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.494 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:48 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'30, mlcod=18'25)", 2024-09-17T21:29:43.494 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.690847+0000", 2024-09-17T21:29:43.494 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.793482662000001, 2024-09-17T21:29:43.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043852800000000001, 2024-09-17T21:29:43.494 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.494 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.494 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.494 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.690847+0000", 2024-09-17T21:29:43.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.690847+0000", 2024-09-17T21:29:43.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.690847+0000", 2024-09-17T21:29:43.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8799999999999998e-07 2024-09-17T21:29:43.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.690850+0000", 2024-09-17T21:29:43.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8679999999999999e-06 2024-09-17T21:29:43.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.690858+0000", 2024-09-17T21:29:43.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1089999999999998e-06 2024-09-17T21:29:43.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.690862+0000", 2024-09-17T21:29:43.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4449999999999998e-06 2024-09-17T21:29:43.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.690903+0000", 2024-09-17T21:29:43.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0336999999999997e-05 2024-09-17T21:29:43.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.690915+0000", 2024-09-17T21:29:43.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2751999999999999e-05 2024-09-17T21:29:43.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.695202+0000", 2024-09-17T21:29:43.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042866720000000001 2024-09-17T21:29:43.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.695232+0000", 2024-09-17T21:29:43.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9808999999999999e-05 2024-09-17T21:29:43.498 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.498 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.498 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.499 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:49 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'31, mlcod=18'26)", 2024-09-17T21:29:43.499 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.691295+0000", 2024-09-17T21:29:43.499 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.793034475000001, 2024-09-17T21:29:43.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039651809999999999, 2024-09-17T21:29:43.499 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.499 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.499 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.499 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.691295+0000", 2024-09-17T21:29:43.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.691295+0000", 2024-09-17T21:29:43.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.691295+0000", 2024-09-17T21:29:43.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8299999999999997e-07 2024-09-17T21:29:43.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.691302+0000", 2024-09-17T21:29:43.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:29:43.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.691310+0000", 2024-09-17T21:29:43.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5059999999999998e-06 2024-09-17T21:29:43.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.691313+0000", 2024-09-17T21:29:43.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5370000000000002e-06 2024-09-17T21:29:43.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.691356+0000", 2024-09-17T21:29:43.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2514e-05 2024-09-17T21:29:43.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.691367+0000", 2024-09-17T21:29:43.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1629e-05 2024-09-17T21:29:43.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.695233+0000", 2024-09-17T21:29:43.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038656860000000001 2024-09-17T21:29:43.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.695260+0000", 2024-09-17T21:29:43.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6837000000000001e-05 2024-09-17T21:29:43.503 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.503 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.503 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.503 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:51 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'33, mlcod=18'26)", 2024-09-17T21:29:43.503 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.692211+0000", 2024-09-17T21:29:43.503 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.792118007999999, 2024-09-17T21:29:43.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0055015469999999999, 2024-09-17T21:29:43.504 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.504 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.504 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.504 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.692211+0000", 2024-09-17T21:29:43.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.692211+0000", 2024-09-17T21:29:43.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.692212+0000", 2024-09-17T21:29:43.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.75e-07 2024-09-17T21:29:43.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.692230+0000", 2024-09-17T21:29:43.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8323999999999999e-05 2024-09-17T21:29:43.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.692237+0000", 2024-09-17T21:29:43.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2150000000000004e-06 2024-09-17T21:29:43.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.692242+0000", 2024-09-17T21:29:43.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2089999999999999e-06 2024-09-17T21:29:43.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.692276+0000", 2024-09-17T21:29:43.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4950999999999998e-05 2024-09-17T21:29:43.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.692289+0000", 2024-09-17T21:29:43.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2998e-05 2024-09-17T21:29:43.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.697678+0000", 2024-09-17T21:29:43.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053885249999999999 2024-09-17T21:29:43.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.697713+0000", 2024-09-17T21:29:43.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4749999999999998e-05 2024-09-17T21:29:43.507 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.507 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.508 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.508 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:52 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'34, mlcod=18'26)", 2024-09-17T21:29:43.508 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.692625+0000", 2024-09-17T21:29:43.508 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.791704533000001, 2024-09-17T21:29:43.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0051204909999999996, 2024-09-17T21:29:43.508 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.508 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.508 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.508 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.692625+0000", 2024-09-17T21:29:43.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.692625+0000", 2024-09-17T21:29:43.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.692625+0000", 2024-09-17T21:29:43.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.13e-07 2024-09-17T21:29:43.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.692629+0000", 2024-09-17T21:29:43.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0230000000000003e-06 2024-09-17T21:29:43.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.692636+0000", 2024-09-17T21:29:43.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.199e-06 2024-09-17T21:29:43.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.692640+0000", 2024-09-17T21:29:43.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8480000000000003e-06 2024-09-17T21:29:43.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.692678+0000", 2024-09-17T21:29:43.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7766999999999999e-05 2024-09-17T21:29:43.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.692690+0000", 2024-09-17T21:29:43.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2034e-05 2024-09-17T21:29:43.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.697715+0000", 2024-09-17T21:29:43.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0050253980000000004 2024-09-17T21:29:43.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.697745+0000", 2024-09-17T21:29:43.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9709e-05 2024-09-17T21:29:43.512 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.512 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.512 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.512 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:54 1.0 e18/17 1:30306672:devicehealth::main.db.0000000000000000:head v 18'36, mlcod=18'33)", 2024-09-17T21:29:43.512 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.699286+0000", 2024-09-17T21:29:43.513 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.785042861999999, 2024-09-17T21:29:43.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0035910270000000001, 2024-09-17T21:29:43.513 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.513 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.513 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.513 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.699286+0000", 2024-09-17T21:29:43.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.699286+0000", 2024-09-17T21:29:43.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.699287+0000", 2024-09-17T21:29:43.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3799999999999997e-07 2024-09-17T21:29:43.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.699364+0000", 2024-09-17T21:29:43.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7405999999999997e-05 2024-09-17T21:29:43.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.699373+0000", 2024-09-17T21:29:43.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3049999999999993e-06 2024-09-17T21:29:43.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.699377+0000", 2024-09-17T21:29:43.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1980000000000003e-06 2024-09-17T21:29:43.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.699412+0000", 2024-09-17T21:29:43.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5676999999999997e-05 2024-09-17T21:29:43.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.699425+0000", 2024-09-17T21:29:43.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2499e-05 2024-09-17T21:29:43.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.702838+0000", 2024-09-17T21:29:43.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0034128100000000001 2024-09-17T21:29:43.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.702877+0000", 2024-09-17T21:29:43.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9594000000000001e-05 2024-09-17T21:29:43.519 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.519 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.519 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.519 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:56 1.0 e18/17 1:30306672:devicehealth::main.db.0000000000000000:head v 18'38, mlcod=18'33)", 2024-09-17T21:29:43.519 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.700307+0000", 2024-09-17T21:29:43.519 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.784022219000001, 2024-09-17T21:29:43.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0050829070000000002, 2024-09-17T21:29:43.519 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.519 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.520 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.520 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.700307+0000", 2024-09-17T21:29:43.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.700307+0000", 2024-09-17T21:29:43.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.700308+0000", 2024-09-17T21:29:43.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:29:43.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.700394+0000", 2024-09-17T21:29:43.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6525000000000003e-05 2024-09-17T21:29:43.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.700403+0000", 2024-09-17T21:29:43.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.371e-06 2024-09-17T21:29:43.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.700408+0000", 2024-09-17T21:29:43.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6720000000000001e-06 2024-09-17T21:29:43.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.700445+0000", 2024-09-17T21:29:43.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6628000000000002e-05 2024-09-17T21:29:43.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.700457+0000", 2024-09-17T21:29:43.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1948e-05 2024-09-17T21:29:43.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.705356+0000", 2024-09-17T21:29:43.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0048990190000000001 2024-09-17T21:29:43.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.705390+0000", 2024-09-17T21:29:43.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4088000000000003e-05 2024-09-17T21:29:43.523 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.524 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.524 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.524 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:64 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'42, mlcod=18'40)", 2024-09-17T21:29:43.524 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.913677+0000", 2024-09-17T21:29:43.524 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.570651979000001, 2024-09-17T21:29:43.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068767359999999996, 2024-09-17T21:29:43.524 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.524 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.524 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.524 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.913677+0000", 2024-09-17T21:29:43.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.913677+0000", 2024-09-17T21:29:43.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.913678+0000", 2024-09-17T21:29:43.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:29:43.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.913683+0000", 2024-09-17T21:29:43.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1059999999999999e-06 2024-09-17T21:29:43.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.913695+0000", 2024-09-17T21:29:43.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2259000000000001e-05 2024-09-17T21:29:43.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.913700+0000", 2024-09-17T21:29:43.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:29:43.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.913745+0000", 2024-09-17T21:29:43.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4257e-05 2024-09-17T21:29:43.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.913765+0000", 2024-09-17T21:29:43.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0514999999999999e-05 2024-09-17T21:29:43.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.920520+0000", 2024-09-17T21:29:43.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067544140000000003 2024-09-17T21:29:43.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.920554+0000", 2024-09-17T21:29:43.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.447e-05 2024-09-17T21:29:43.528 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.528 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.528 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.529 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:65 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'43, mlcod=18'40)", 2024-09-17T21:29:43.529 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.914282+0000", 2024-09-17T21:29:43.529 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.570047662, 2024-09-17T21:29:43.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063005159999999999, 2024-09-17T21:29:43.529 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.529 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.529 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.529 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.914282+0000", 2024-09-17T21:29:43.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.914282+0000", 2024-09-17T21:29:43.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.914282+0000", 2024-09-17T21:29:43.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.39e-07 2024-09-17T21:29:43.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.914288+0000", 2024-09-17T21:29:43.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.57e-06 2024-09-17T21:29:43.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.914293+0000", 2024-09-17T21:29:43.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5650000000000001e-06 2024-09-17T21:29:43.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.914297+0000", 2024-09-17T21:29:43.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1060000000000002e-06 2024-09-17T21:29:43.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.914331+0000", 2024-09-17T21:29:43.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4233000000000002e-05 2024-09-17T21:29:43.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.914344+0000", 2024-09-17T21:29:43.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2748999999999999e-05 2024-09-17T21:29:43.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.920557+0000", 2024-09-17T21:29:43.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0062123320000000001 2024-09-17T21:29:43.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.920582+0000", 2024-09-17T21:29:43.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5522000000000001e-05 2024-09-17T21:29:43.533 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.533 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.533 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.534 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:66 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'44, mlcod=18'40)", 2024-09-17T21:29:43.534 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.914691+0000", 2024-09-17T21:29:43.534 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.569637751, 2024-09-17T21:29:43.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0064362710000000004, 2024-09-17T21:29:43.534 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.534 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.534 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.534 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.914691+0000", 2024-09-17T21:29:43.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.914691+0000", 2024-09-17T21:29:43.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.914692+0000", 2024-09-17T21:29:43.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3500000000000002e-07 2024-09-17T21:29:43.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.914749+0000", 2024-09-17T21:29:43.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7364000000000002e-05 2024-09-17T21:29:43.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.914756+0000", 2024-09-17T21:29:43.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4119999999999998e-06 2024-09-17T21:29:43.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.914761+0000", 2024-09-17T21:29:43.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2220000000000001e-06 2024-09-17T21:29:43.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.914789+0000", 2024-09-17T21:29:43.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8546000000000001e-05 2024-09-17T21:29:43.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.914800+0000", 2024-09-17T21:29:43.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1078e-05 2024-09-17T21:29:43.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.921081+0000", 2024-09-17T21:29:43.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006280224 2024-09-17T21:29:43.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.921128+0000", 2024-09-17T21:29:43.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7089999999999998e-05 2024-09-17T21:29:43.538 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.538 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.538 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.538 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:67 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'45, mlcod=18'40)", 2024-09-17T21:29:43.538 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.915520+0000", 2024-09-17T21:29:43.538 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.568809554, 2024-09-17T21:29:43.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0082865339999999999, 2024-09-17T21:29:43.538 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.539 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.539 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.539 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.915520+0000", 2024-09-17T21:29:43.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.915520+0000", 2024-09-17T21:29:43.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.915520+0000", 2024-09-17T21:29:43.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.68e-07 2024-09-17T21:29:43.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.915525+0000", 2024-09-17T21:29:43.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2849999999999996e-06 2024-09-17T21:29:43.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.915534+0000", 2024-09-17T21:29:43.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8970000000000003e-06 2024-09-17T21:29:43.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.915539+0000", 2024-09-17T21:29:43.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4920000000000004e-06 2024-09-17T21:29:43.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.917349+0000", 2024-09-17T21:29:43.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0018096379999999999 2024-09-17T21:29:43.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.917361+0000", 2024-09-17T21:29:43.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1955e-05 2024-09-17T21:29:43.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.923775+0000", 2024-09-17T21:29:43.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0064148850000000004 2024-09-17T21:29:43.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.923806+0000", 2024-09-17T21:29:43.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0814e-05 2024-09-17T21:29:43.543 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.543 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.543 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.543 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:68 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'46, mlcod=18'40)", 2024-09-17T21:29:43.543 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.915922+0000", 2024-09-17T21:29:43.543 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.568407014, 2024-09-17T21:29:43.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079116810000000003, 2024-09-17T21:29:43.543 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.543 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.543 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.543 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.915922+0000", 2024-09-17T21:29:43.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.915922+0000", 2024-09-17T21:29:43.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.915923+0000", 2024-09-17T21:29:43.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5799999999999999e-07 2024-09-17T21:29:43.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.915928+0000", 2024-09-17T21:29:43.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6909999999999996e-06 2024-09-17T21:29:43.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.915937+0000", 2024-09-17T21:29:43.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9229999999999997e-06 2024-09-17T21:29:43.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.915942+0000", 2024-09-17T21:29:43.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6090000000000003e-06 2024-09-17T21:29:43.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.917581+0000", 2024-09-17T21:29:43.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001638865 2024-09-17T21:29:43.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.917594+0000", 2024-09-17T21:29:43.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3278999999999999e-05 2024-09-17T21:29:43.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.923808+0000", 2024-09-17T21:29:43.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0062141829999999999 2024-09-17T21:29:43.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.923834+0000", 2024-09-17T21:29:43.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5473000000000002e-05 2024-09-17T21:29:43.547 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.547 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.547 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.548 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:69 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'47, mlcod=18'40)", 2024-09-17T21:29:43.548 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.916345+0000", 2024-09-17T21:29:43.548 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.567984211000001, 2024-09-17T21:29:43.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0097631490000000005, 2024-09-17T21:29:43.548 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.548 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.548 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.548 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.916345+0000", 2024-09-17T21:29:43.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.916345+0000", 2024-09-17T21:29:43.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.916346+0000", 2024-09-17T21:29:43.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9100000000000004e-07 2024-09-17T21:29:43.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.916376+0000", 2024-09-17T21:29:43.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0150000000000001e-05 2024-09-17T21:29:43.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.916386+0000", 2024-09-17T21:29:43.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0404999999999999e-05 2024-09-17T21:29:43.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.916392+0000", 2024-09-17T21:29:43.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0129999999999999e-06 2024-09-17T21:29:43.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.917758+0000", 2024-09-17T21:29:43.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001366024 2024-09-17T21:29:43.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.917769+0000", 2024-09-17T21:29:43.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1041e-05 2024-09-17T21:29:43.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.926084+0000", 2024-09-17T21:29:43.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0083143980000000006 2024-09-17T21:29:43.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.926108+0000", 2024-09-17T21:29:43.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4527000000000001e-05 2024-09-17T21:29:43.552 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.552 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.552 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.552 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:70 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'48, mlcod=18'40)", 2024-09-17T21:29:43.552 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.918956+0000", 2024-09-17T21:29:43.553 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.565373343999999, 2024-09-17T21:29:43.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071818250000000002, 2024-09-17T21:29:43.553 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.553 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.553 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.553 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.918956+0000", 2024-09-17T21:29:43.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.918956+0000", 2024-09-17T21:29:43.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.918956+0000", 2024-09-17T21:29:43.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6599999999999996e-07 2024-09-17T21:29:43.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.918961+0000", 2024-09-17T21:29:43.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2869999999999998e-06 2024-09-17T21:29:43.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.918971+0000", 2024-09-17T21:29:43.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7690000000000008e-06 2024-09-17T21:29:43.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.918975+0000", 2024-09-17T21:29:43.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4469999999999999e-06 2024-09-17T21:29:43.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.920617+0000", 2024-09-17T21:29:43.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001642059 2024-09-17T21:29:43.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.920632+0000", 2024-09-17T21:29:43.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.465e-05 2024-09-17T21:29:43.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.926109+0000", 2024-09-17T21:29:43.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0054778149999999996 2024-09-17T21:29:43.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.926138+0000", 2024-09-17T21:29:43.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8232e-05 2024-09-17T21:29:43.557 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.557 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.557 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.557 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:71 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'49, mlcod=18'40)", 2024-09-17T21:29:43.557 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.919352+0000", 2024-09-17T21:29:43.557 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.564977561999999, 2024-09-17T21:29:43.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068135299999999999, 2024-09-17T21:29:43.557 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.557 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.558 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.558 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.919352+0000", 2024-09-17T21:29:43.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.919352+0000", 2024-09-17T21:29:43.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.919352+0000", 2024-09-17T21:29:43.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4300000000000003e-07 2024-09-17T21:29:43.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.919357+0000", 2024-09-17T21:29:43.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7700000000000001e-06 2024-09-17T21:29:43.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.919367+0000", 2024-09-17T21:29:43.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9879999999999999e-06 2024-09-17T21:29:43.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.919372+0000", 2024-09-17T21:29:43.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5120000000000002e-06 2024-09-17T21:29:43.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.920827+0000", 2024-09-17T21:29:43.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0014552230000000001 2024-09-17T21:29:43.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.920845+0000", 2024-09-17T21:29:43.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7971000000000001e-05 2024-09-17T21:29:43.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.926139+0000", 2024-09-17T21:29:43.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0052939959999999996 2024-09-17T21:29:43.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.926165+0000", 2024-09-17T21:29:43.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6427e-05 2024-09-17T21:29:43.561 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.562 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.562 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.562 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:72 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'50, mlcod=18'40)", 2024-09-17T21:29:43.562 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.919727+0000", 2024-09-17T21:29:43.562 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.564602535000001, 2024-09-17T21:29:43.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0090588579999999995, 2024-09-17T21:29:43.562 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.562 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.562 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.562 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.919727+0000", 2024-09-17T21:29:43.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.919727+0000", 2024-09-17T21:29:43.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.919727+0000", 2024-09-17T21:29:43.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6599999999999996e-07 2024-09-17T21:29:43.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.919751+0000", 2024-09-17T21:29:43.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3451e-05 2024-09-17T21:29:43.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.919760+0000", 2024-09-17T21:29:43.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5599999999999999e-06 2024-09-17T21:29:43.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.919765+0000", 2024-09-17T21:29:43.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3229999999999999e-06 2024-09-17T21:29:43.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.921159+0000", 2024-09-17T21:29:43.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0013945279999999999 2024-09-17T21:29:43.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.921172+0000", 2024-09-17T21:29:43.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2815e-05 2024-09-17T21:29:43.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.928742+0000", 2024-09-17T21:29:43.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0075698069999999996 2024-09-17T21:29:43.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.928786+0000", 2024-09-17T21:29:43.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3807999999999998e-05 2024-09-17T21:29:43.566 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.566 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.566 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.567 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:73 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'51, mlcod=18'40)", 2024-09-17T21:29:43.567 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.921834+0000", 2024-09-17T21:29:43.567 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.562495391000001, 2024-09-17T21:29:43.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006981564, 2024-09-17T21:29:43.567 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.567 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.567 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.567 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.921834+0000", 2024-09-17T21:29:43.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.921834+0000", 2024-09-17T21:29:43.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.921835+0000", 2024-09-17T21:29:43.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8100000000000002e-07 2024-09-17T21:29:43.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.921839+0000", 2024-09-17T21:29:43.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4839999999999998e-06 2024-09-17T21:29:43.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.921849+0000", 2024-09-17T21:29:43.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9410000000000002e-06 2024-09-17T21:29:43.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.921853+0000", 2024-09-17T21:29:43.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-17T21:29:43.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.923297+0000", 2024-09-17T21:29:43.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0014435170000000001 2024-09-17T21:29:43.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.923314+0000", 2024-09-17T21:29:43.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7099e-05 2024-09-17T21:29:43.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.928787+0000", 2024-09-17T21:29:43.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0054729540000000004 2024-09-17T21:29:43.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.928815+0000", 2024-09-17T21:29:43.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8449000000000002e-05 2024-09-17T21:29:43.571 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.571 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.571 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.571 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:74 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'52, mlcod=18'43)", 2024-09-17T21:29:43.571 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.922559+0000", 2024-09-17T21:29:43.571 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.561770264, 2024-09-17T21:29:43.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0062774099999999998, 2024-09-17T21:29:43.571 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.571 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.572 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.572 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.922559+0000", 2024-09-17T21:29:43.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.922559+0000", 2024-09-17T21:29:43.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.922560+0000", 2024-09-17T21:29:43.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6000000000000003e-07 2024-09-17T21:29:43.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.922565+0000", 2024-09-17T21:29:43.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7150000000000003e-06 2024-09-17T21:29:43.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.922575+0000", 2024-09-17T21:29:43.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7019999999999996e-06 2024-09-17T21:29:43.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.922579+0000", 2024-09-17T21:29:43.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3159999999999998e-06 2024-09-17T21:29:43.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.923548+0000", 2024-09-17T21:29:43.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00096821400000000001 2024-09-17T21:29:43.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.923564+0000", 2024-09-17T21:29:43.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6922000000000002e-05 2024-09-17T21:29:43.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.928817+0000", 2024-09-17T21:29:43.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0052529100000000004 2024-09-17T21:29:43.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.928836+0000", 2024-09-17T21:29:43.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8970999999999998e-05 2024-09-17T21:29:43.575 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.575 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.576 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.576 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:75 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'53, mlcod=18'43)", 2024-09-17T21:29:43.576 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.922918+0000", 2024-09-17T21:29:43.576 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.561411014000001, 2024-09-17T21:29:43.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059410620000000004, 2024-09-17T21:29:43.576 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.576 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.576 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.576 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.922918+0000", 2024-09-17T21:29:43.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.922918+0000", 2024-09-17T21:29:43.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.922919+0000", 2024-09-17T21:29:43.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:29:43.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.922987+0000", 2024-09-17T21:29:43.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8485999999999994e-05 2024-09-17T21:29:43.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.922993+0000", 2024-09-17T21:29:43.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8479999999999997e-06 2024-09-17T21:29:43.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.922997+0000", 2024-09-17T21:29:43.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.403e-06 2024-09-17T21:29:43.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.923864+0000", 2024-09-17T21:29:43.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00086649999999999997 2024-09-17T21:29:43.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.923876+0000", 2024-09-17T21:29:43.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2035000000000001e-05 2024-09-17T21:29:43.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.928837+0000", 2024-09-17T21:29:43.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0049612479999999997 2024-09-17T21:29:43.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.928859+0000", 2024-09-17T21:29:43.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2121e-05 2024-09-17T21:29:43.580 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.580 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.580 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.580 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4197.0:76 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'54, mlcod=18'43)", 2024-09-17T21:29:43.580 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:28.923598+0000", 2024-09-17T21:29:43.580 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.560731328999999, 2024-09-17T21:29:43.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077439040000000002, 2024-09-17T21:29:43.580 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:43.581 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:43.581 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:43.581 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:43.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:43.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.923598+0000", 2024-09-17T21:29:43.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:43.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.923598+0000", 2024-09-17T21:29:43.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:43.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:43.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.923598+0000", 2024-09-17T21:29:43.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:29:43.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:43.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.923602+0000", 2024-09-17T21:29:43.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5099999999999999e-06 2024-09-17T21:29:43.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:43.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.923607+0000", 2024-09-17T21:29:43.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8790000000000002e-06 2024-09-17T21:29:43.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:43.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.923611+0000", 2024-09-17T21:29:43.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.729e-06 2024-09-17T21:29:43.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:43.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.925895+0000", 2024-09-17T21:29:43.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022831879999999998 2024-09-17T21:29:43.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:43.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.925913+0000", 2024-09-17T21:29:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8287999999999998e-05 2024-09-17T21:29:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.931318+0000", 2024-09-17T21:29:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0054050000000000001 2024-09-17T21:29:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:28.931342+0000", 2024-09-17T21:29:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3881000000000001e-05 2024-09-17T21:29:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.585 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:43.585 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:43.585 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:43.587 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:29:43.587 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:43.584+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:43.687 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:43.686+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:43.731 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:43.731 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:43.731 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:29:43.731 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:43.740 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:29:43.788 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:43.786+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:43.884 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:29:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:29:43.884 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:43.888 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:43.887+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:43.893 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:29:43.989 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:43.986+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:44.037 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:29:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:29:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:29:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4269.0:5 2.2 e22/20 2:5cea7035:::rbd_info:head v 22'1, mlcod=0'0)", 2024-09-17T21:29:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:33.969321+0000", 2024-09-17T21:29:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.066668877, 2024-09-17T21:29:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00090029699999999999, 2024-09-17T21:29:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:33.969321+0000", 2024-09-17T21:29:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:33.969321+0000", 2024-09-17T21:29:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:33.969321+0000", 2024-09-17T21:29:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0400000000000002e-07 2024-09-17T21:29:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:33.969333+0000", 2024-09-17T21:29:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2315e-05 2024-09-17T21:29:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:33.969349+0000", 2024-09-17T21:29:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5353999999999999e-05 2024-09-17T21:29:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:33.969361+0000", 2024-09-17T21:29:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2914e-05 2024-09-17T21:29:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:33.969432+0000", 2024-09-17T21:29:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0749999999999999e-05 2024-09-17T21:29:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:33.969456+0000", 2024-09-17T21:29:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3635e-05 2024-09-17T21:29:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:33.970173+0000", 2024-09-17T21:29:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000717415 2024-09-17T21:29:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:33.970221+0000", 2024-09-17T21:29:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.761e-05 2024-09-17T21:29:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4269.0:7 2.2 e23/20 2:5cea7035:::rbd_info:head v 23'2, mlcod=0'0)", 2024-09-17T21:29:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:34.984686+0000", 2024-09-17T21:29:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.0513037710000006, 2024-09-17T21:29:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0019783800000000001, 2024-09-17T21:29:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:34.984686+0000", 2024-09-17T21:29:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:34.984686+0000", 2024-09-17T21:29:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:34.984686+0000", 2024-09-17T21:29:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.48e-07 2024-09-17T21:29:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:34.984690+0000", 2024-09-17T21:29:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0319999999999997e-06 2024-09-17T21:29:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:34.984701+0000", 2024-09-17T21:29:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0774000000000001e-05 2024-09-17T21:29:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:34.984717+0000", 2024-09-17T21:29:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6427e-05 2024-09-17T21:29:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:34.984771+0000", 2024-09-17T21:29:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4304e-05 2024-09-17T21:29:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:34.984789+0000", 2024-09-17T21:29:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7504999999999999e-05 2024-09-17T21:29:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:34.986626+0000", 2024-09-17T21:29:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0018374909999999999 2024-09-17T21:29:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:34.986664+0000", 2024-09-17T21:29:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7698999999999999e-05 2024-09-17T21:29:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:7 3.1a e27/25 3:5cea7035:::rbd_info:head v 27'1, mlcod=0'0)", 2024-09-17T21:29:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:42.470460+0000", 2024-09-17T21:29:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.565529569, 2024-09-17T21:29:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00086037099999999997, 2024-09-17T21:29:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:42.470460+0000", 2024-09-17T21:29:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:42.470460+0000", 2024-09-17T21:29:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:42.470460+0000", 2024-09-17T21:29:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.99e-07 2024-09-17T21:29:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:42.470464+0000", 2024-09-17T21:29:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0929999999999996e-06 2024-09-17T21:29:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:42.470473+0000", 2024-09-17T21:29:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9579999999999996e-06 2024-09-17T21:29:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:42.470484+0000", 2024-09-17T21:29:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0662000000000001e-05 2024-09-17T21:29:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:42.470537+0000", 2024-09-17T21:29:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3247e-05 2024-09-17T21:29:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:42.470559+0000", 2024-09-17T21:29:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2198999999999999e-05 2024-09-17T21:29:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:42.471288+0000", 2024-09-17T21:29:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00072896500000000004 2024-09-17T21:29:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:42.471320+0000", 2024-09-17T21:29:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2048000000000002e-05 2024-09-17T21:29:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:9 3.1a e28/25 3:5cea7035:::rbd_info:head v 28'2, mlcod=0'0)", 2024-09-17T21:29:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:43.476809+0000", 2024-09-17T21:29:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.55918019399999996, 2024-09-17T21:29:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00076541800000000002, 2024-09-17T21:29:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.476809+0000", 2024-09-17T21:29:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.476809+0000", 2024-09-17T21:29:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.476810+0000", 2024-09-17T21:29:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:29:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.476816+0000", 2024-09-17T21:29:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2689999999999996e-06 2024-09-17T21:29:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.476837+0000", 2024-09-17T21:29:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1617e-05 2024-09-17T21:29:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.476851+0000", 2024-09-17T21:29:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3616e-05 2024-09-17T21:29:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.476922+0000", 2024-09-17T21:29:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1248000000000006e-05 2024-09-17T21:29:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.476948+0000", 2024-09-17T21:29:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5395999999999999e-05 2024-09-17T21:29:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.477538+0000", 2024-09-17T21:29:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00059009999999999998 2024-09-17T21:29:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.477575+0000", 2024-09-17T21:29:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6878999999999997e-05 2024-09-17T21:29:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:13 3.1a e28/25 3:5a9e26c2:::rbd_object_map.1120555f8258:head v 28'3, mlcod=27'1)", 2024-09-17T21:29:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:43.493374+0000", 2024-09-17T21:29:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.54261560900000005, 2024-09-17T21:29:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000645416, 2024-09-17T21:29:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.493374+0000", 2024-09-17T21:29:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.493374+0000", 2024-09-17T21:29:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.493376+0000", 2024-09-17T21:29:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8679999999999999e-06 2024-09-17T21:29:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.493385+0000", 2024-09-17T21:29:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6919999999999997e-06 2024-09-17T21:29:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.493404+0000", 2024-09-17T21:29:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.821e-05 2024-09-17T21:29:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.493412+0000", 2024-09-17T21:29:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5369999999999997e-06 2024-09-17T21:29:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.493464+0000", 2024-09-17T21:29:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1452999999999997e-05 2024-09-17T21:29:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.493480+0000", 2024-09-17T21:29:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5909999999999998e-05 2024-09-17T21:29:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.493979+0000", 2024-09-17T21:29:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00049964499999999995 2024-09-17T21:29:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:43.494019+0000", 2024-09-17T21:29:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0101000000000001e-05 2024-09-17T21:29:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:44.059 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:44.060 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:29:44.104 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:44.102+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:44.164 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:44.164 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:44.164 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:29:44.164 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:44.173 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:29:44.191 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:44.188+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:44.292 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:44.289+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:44.318 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:44.319 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:44.319 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:29:44.319 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:29:44.319 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:44.328 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:29:44.392 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:44.391+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:44.433 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:44.433 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:29:44.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:29:44.434 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:29:44.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.434 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4269.0:1 2.3 e21/20 2:c4c92e5a:::rbd_trash:head v 21'1, mlcod=0'0)", 2024-09-17T21:29:44.434 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:32.973144+0000", 2024-09-17T21:29:44.434 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 11.458912308, 2024-09-17T21:29:44.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00080378800000000005, 2024-09-17T21:29:44.434 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:44.434 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:44.434 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:44.434 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:44.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:44.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973144+0000", 2024-09-17T21:29:44.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:44.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:44.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973144+0000", 2024-09-17T21:29:44.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:44.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:44.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973144+0000", 2024-09-17T21:29:44.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7599999999999998e-07 2024-09-17T21:29:44.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:44.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973159+0000", 2024-09-17T21:29:44.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4997999999999999e-05 2024-09-17T21:29:44.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:44.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973168+0000", 2024-09-17T21:29:44.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0380000000000005e-06 2024-09-17T21:29:44.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:44.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973176+0000", 2024-09-17T21:29:44.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0450000000000001e-06 2024-09-17T21:29:44.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:44.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973225+0000", 2024-09-17T21:29:44.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8899000000000001e-05 2024-09-17T21:29:44.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:44.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973248+0000", 2024-09-17T21:29:44.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3113e-05 2024-09-17T21:29:44.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:44.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973908+0000", 2024-09-17T21:29:44.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066012799999999995 2024-09-17T21:29:44.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:44.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:44.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:44.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973948+0000", 2024-09-17T21:29:44.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9190999999999998e-05 2024-09-17T21:29:44.438 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:44.438 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:44.438 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:44.438 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:44.438 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:44.438 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:44.442 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:29:44.492 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:29:44.526 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:44.525+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:44.588 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:44.588 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:44.588 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:29:44.589 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:44.595 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:44.592+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:44.598 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:29:44.651 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:29:44.651 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:29:44.668 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:29:44.668 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:29:44.684 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:29:44.685 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:29:44.696 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:44.694+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:44.701 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:29:44.701 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:29:44.705 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:44.705 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:44.714 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:29:44.718 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:29:44.718 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:29:44.735 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:29:44.735 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:29:44.752 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:29:44.752 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:29:44.769 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:29:44.769 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:29:44.796 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:44.794+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:44.818 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:44.818 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:44.827 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:29:44.897 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:44.895+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:44.945 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:44.946 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:44.955 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:29:44.998 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:44.996+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:45.098 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:45.096+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:45.099 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:45.099 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:45.107 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:29:45.199 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:45.197+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:45.204 INFO:teuthology.orchestra.run.smithi129.stdout:1 write 0x855ad36 thru 0x8564a3b (0x9d06 bytes) 2024-09-17T21:29:45.207 INFO:teuthology.orchestra.run.smithi129.stdout:2 writesame 0x577bdb3 thru 0x57829af data_size 0xf(0x6bfd bytes) 2024-09-17T21:29:45.210 INFO:teuthology.orchestra.run.smithi129.stdout:3 write 0x4fb2834 thru 0x4fba23e (0x7a0b bytes) 2024-09-17T21:29:45.216 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:45.216 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:45.224 INFO:teuthology.orchestra.run.smithi129.stdout:4 write 0xd0ece07 thru 0xd0f330d (0x6507 bytes) 2024-09-17T21:29:45.225 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:29:45.228 INFO:teuthology.orchestra.run.smithi129.stdout:truncating to largest ever: 0xb1ddeb 2024-09-17T21:29:45.238 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T21:29:45.238 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:29:45.238 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-17T21:29:45.238 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-17T21:29:45.242 INFO:teuthology.orchestra.run.smithi129.stdout:5 trunc from 0xd0f330e to 0xb1ddeb 2024-09-17T21:29:45.255 INFO:teuthology.orchestra.run.smithi129.stdout:6 read 0x75e7e7 thru 0x76798f (0x91a9 bytes) 2024-09-17T21:29:45.277 INFO:teuthology.orchestra.run.smithi129.stdout:7 write 0xe7196a4 thru 0xe727cde (0xe63b bytes) 2024-09-17T21:29:45.280 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:45.280 INFO:teuthology.orchestra.run.smithi129.stdout:14 punch from 0x69177c5 to 0x6925206, (0xda41 bytes) 2024-09-17T21:29:45.280 INFO:teuthology.orchestra.run.smithi129.stdout:15 write 0xb9b3ca8 thru 0xb9c07f6 (0xcb4f bytes) 2024-09-17T21:29:45.285 INFO:teuthology.orchestra.run.smithi129.stdout:16 punch from 0x73222a7 to 0x732b171, (0x8eca bytes) 2024-09-17T21:29:45.285 INFO:teuthology.orchestra.run.smithi129.stdout:truncating to largest ever: 0x448912d 2024-09-17T21:29:45.297 INFO:teuthology.orchestra.run.smithi129.stdout:17 trunc from 0xe727cdf to 0x448912d 2024-09-17T21:29:45.300 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:45.298+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:45.307 INFO:teuthology.orchestra.run.smithi129.stdout:18 punch from 0x301f322 to 0x301f46c, (0x14a bytes) 2024-09-17T21:29:45.317 INFO:teuthology.orchestra.run.smithi129.stdout:19 write 0x7ff448f thru 0x7ffabe1 (0x6753 bytes) 2024-09-17T21:29:45.320 INFO:teuthology.orchestra.run.smithi129.stdout:21 read 0x4e999f thru 0x4f5d55 (0xc3b7 bytes) 2024-09-17T21:29:45.320 INFO:teuthology.orchestra.run.smithi129.stdout:22 punch from 0x2d504bc to 0x2d5a37b, (0x9ebf bytes) 2024-09-17T21:29:45.320 INFO:teuthology.orchestra.run.smithi129.stdout:23 punch from 0x727d1fc to 0x727f2d3, (0x20d7 bytes) 2024-09-17T21:29:45.333 INFO:teuthology.orchestra.run.smithi129.stdout:25 write 0xe3af88e thru 0xe3b5680 (0x5df3 bytes) 2024-09-17T21:29:45.337 INFO:teuthology.orchestra.run.smithi129.stdout:26 read 0x9fe6a46 thru 0x9fed0d1 (0x668c bytes) 2024-09-17T21:29:45.350 INFO:teuthology.orchestra.run.smithi129.stdout:27 trunc from 0xe3b5681 to 0x416081c 2024-09-17T21:29:45.365 INFO:teuthology.orchestra.run.smithi129.stdout:28 punch from 0x279a741 to 0x279bf1d, (0x17dc bytes) 2024-09-17T21:29:45.365 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:45.365 INFO:teuthology.orchestra.run.smithi129.stdout:30 writesame 0x16251f thru 0x16454b data_size 0x1(0x202d bytes) 2024-09-17T21:29:45.374 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:45.374 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:45.383 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:29:45.384 INFO:teuthology.orchestra.run.smithi129.stdout:31 writesame 0xa885d20 thru 0xa88639c data_size 0x1(0x67d bytes) 2024-09-17T21:29:45.384 INFO:teuthology.orchestra.run.smithi129.stdout:33 write 0x75a4c4d thru 0x75aed97 (0xa14b bytes) 2024-09-17T21:29:45.387 INFO:teuthology.orchestra.run.smithi129.stdout:34 write 0x5fa253a thru 0x5fa72d3 (0x4d9a bytes) 2024-09-17T21:29:45.390 INFO:teuthology.orchestra.run.smithi129.stdout:38 read 0x94ef7a7 thru 0x94fbb58 (0xc3b2 bytes) 2024-09-17T21:29:45.390 INFO:teuthology.orchestra.run.smithi129.stdout:39 read 0x344be0d thru 0x3453652 (0x7846 bytes) 2024-09-17T21:29:45.390 INFO:teuthology.orchestra.run.smithi129.stdout:41 read 0x1649c14 thru 0x165028f (0x667c bytes) 2024-09-17T21:29:45.397 INFO:teuthology.orchestra.run.smithi129.stdout:44 write 0xc14a73b thru 0xc14e7c4 (0x408a bytes) 2024-09-17T21:29:45.400 INFO:teuthology.orchestra.run.smithi129.stdout:47 write 0x6118214 thru 0x61239e4 (0xb7d1 bytes) 2024-09-17T21:29:45.405 INFO:teuthology.orchestra.run.smithi129.stdout:48 write 0xb36e6f0 thru 0xb375cba (0x75cb bytes) 2024-09-17T21:29:45.407 INFO:teuthology.orchestra.run.smithi129.stdout:truncating to largest ever: 0x71a7f54 2024-09-17T21:29:45.414 INFO:teuthology.orchestra.run.smithi129.stdout:49 trunc from 0xc14e7c5 to 0x71a7f54 2024-09-17T21:29:45.423 INFO:teuthology.orchestra.run.smithi129.stdout:50 read 0x12a79bb thru 0x12b62c4 (0xe90a bytes) 2024-09-17T21:29:45.423 INFO:teuthology.orchestra.run.smithi129.stdout:truncating to largest ever: 0x79057eb 2024-09-17T21:29:45.424 INFO:teuthology.orchestra.run.smithi129.stdout:51 trunc from 0x71a7f54 to 0x79057eb 2024-09-17T21:29:45.427 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:45.425+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:45.438 INFO:teuthology.orchestra.run.smithi129.stdout:52 write 0xa160286 thru 0xa166e08 (0x6b83 bytes) 2024-09-17T21:29:45.450 INFO:teuthology.orchestra.run.smithi129.stdout:55 trunc from 0xa166e09 to 0x23c4e80 2024-09-17T21:29:45.481 INFO:teuthology.orchestra.run.smithi129.stdout:56 writesame 0xdc70dec thru 0xdc79304 data_size 0x1(0x8519 bytes) 2024-09-17T21:29:45.481 INFO:teuthology.orchestra.run.smithi129.stdout:57 read 0x12bf5c2 thru 0x12c32f4 (0x3d33 bytes) 2024-09-17T21:29:45.481 INFO:teuthology.orchestra.run.smithi129.stdout:58 read 0xd4d0314 thru 0xd4d33ba (0x30a7 bytes) 2024-09-17T21:29:45.491 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:45.491 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:45.494 INFO:teuthology.orchestra.run.smithi129.stdout:59 trunc from 0xdc79305 to 0x271e4bb 2024-09-17T21:29:45.499 INFO:teuthology.orchestra.run.smithi129.stdout:60 write 0x1e176c1 thru 0x1e24f70 (0xd8b0 bytes) 2024-09-17T21:29:45.500 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:29:45.503 INFO:teuthology.orchestra.run.smithi129.stdout:61 read 0x224e5af thru 0x2255b6c (0x75be bytes) 2024-09-17T21:29:45.503 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:45.503 INFO:teuthology.orchestra.run.smithi129.stdout:66 punch from 0x1aec12 to 0x1b8c62, (0xa050 bytes) 2024-09-17T21:29:45.518 INFO:teuthology.orchestra.run.smithi129.stdout:67 writesame 0xa876c58 thru 0xa884bd9 data_size 0x7a(0xdf82 bytes) 2024-09-17T21:29:45.518 INFO:teuthology.orchestra.run.smithi129.stdout:68 read 0x347fedc thru 0x348f9aa (0xfacf bytes) 2024-09-17T21:29:45.519 INFO:teuthology.orchestra.run.smithi129.stdout:69 write 0x37ecdc9 thru 0x37f38f4 (0x6b2c bytes) 2024-09-17T21:29:45.521 INFO:teuthology.orchestra.run.smithi129.stdout:70 writesame 0x5c44f42 thru 0x5c46a97 data_size 0x1(0x1b56 bytes) 2024-09-17T21:29:45.527 INFO:teuthology.orchestra.run.smithi129.stdout:73 write 0xa9c747 thru 0xaa68d5 (0xa18f bytes) 2024-09-17T21:29:45.531 INFO:teuthology.orchestra.run.smithi129.stdout:74 read 0x91f227f thru 0x91f91bb (0x6f3d bytes) 2024-09-17T21:29:45.531 INFO:teuthology.orchestra.run.smithi129.stdout:75 read 0x81366c5 thru 0x8137b12 (0x144e bytes) 2024-09-17T21:29:45.531 INFO:teuthology.orchestra.run.smithi129.stdout:76 read 0x31bf6b thru 0x32b631 (0xf6c7 bytes) 2024-09-17T21:29:45.540 INFO:teuthology.orchestra.run.smithi129.stdout:77 write 0xd945c79 thru 0xd951ec5 (0xc24d bytes) 2024-09-17T21:29:45.542 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:45.540+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:45.543 INFO:teuthology.orchestra.run.smithi129.stdout:78 writesame 0xd4cfbc5 thru 0xd4d5aac data_size 0x4(0x5ee8 bytes) 2024-09-17T21:29:45.548 INFO:teuthology.orchestra.run.smithi129.stdout:79 write 0xd092b6a thru 0xd094c66 (0x20fd bytes) 2024-09-17T21:29:45.551 INFO:teuthology.orchestra.run.smithi129.stdout:81 read 0x129330 thru 0x137df6 (0xeac7 bytes) 2024-09-17T21:29:45.551 INFO:teuthology.orchestra.run.smithi129.stdout:82 read 0x8324ff9 thru 0x83318a7 (0xc8af bytes) 2024-09-17T21:29:45.551 INFO:teuthology.orchestra.run.smithi129.stdout:83 punch from 0xd3009f to 0xd36fd5, (0x6f36 bytes) 2024-09-17T21:29:45.552 INFO:teuthology.orchestra.run.smithi129.stdout:84 read 0xc52b9b thru 0xc5367d (0xae3 bytes) 2024-09-17T21:29:45.567 INFO:teuthology.orchestra.run.smithi129.stdout:85 trunc from 0xd951ec6 to 0x4a2dc0 2024-09-17T21:29:45.571 INFO:teuthology.orchestra.run.smithi129.stdout:moved 0 / 82 (0%) 2024-09-17T21:29:45.571 INFO:teuthology.orchestra.run.smithi129.stdout:avg 10.25 2024-09-17T21:29:45.571 INFO:teuthology.orchestra.run.smithi129.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-09-17T21:29:45.571 INFO:teuthology.orchestra.run.smithi129.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-17T21:29:45.571 INFO:teuthology.orchestra.run.smithi129.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-17T21:29:45.571 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:29:45.571 INFO:teuthology.orchestra.run.smithi129.stdout:oload 110 2024-09-17T21:29:45.571 INFO:teuthology.orchestra.run.smithi129.stdout:max_change 0.05 2024-09-17T21:29:45.571 INFO:teuthology.orchestra.run.smithi129.stdout:max_change_osds 4 2024-09-17T21:29:45.571 INFO:teuthology.orchestra.run.smithi129.stdout:average_utilization 0.0003 2024-09-17T21:29:45.572 INFO:teuthology.orchestra.run.smithi129.stdout:overload_utilization 0.0003 2024-09-17T21:29:45.572 INFO:teuthology.orchestra.run.smithi129.stderr:no change 2024-09-17T21:29:45.579 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:45.579 INFO:teuthology.orchestra.run.smithi129.stdout:88 punch from 0x465bbf to 0x472dac, (0xd1ed bytes) 2024-09-17T21:29:45.579 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:45.586 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:29:45.587 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-17T21:29:45.588 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-17T21:29:45.589 INFO:teuthology.orchestra.run.smithi129.stdout:91 writesame 0x3ed946e thru 0x3ee343f data_size 0x9(0x9fd2 bytes) 2024-09-17T21:29:45.589 INFO:teuthology.orchestra.run.smithi129.stdout:92 read 0x2daf404 thru 0x2dbdb5e (0xe75b bytes) 2024-09-17T21:29:45.589 INFO:teuthology.orchestra.run.smithi129.stdout:93 writesame 0x1dcd388 thru 0x1dd5ecb data_size 0x6(0x8b44 bytes) 2024-09-17T21:29:45.603 INFO:teuthology.orchestra.run.smithi129.stdout:94 write 0x6c05738 thru 0x6c05b88 (0x451 bytes) 2024-09-17T21:29:45.604 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:45.600+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:29:45.605 INFO:teuthology.orchestra.run.smithi129.stdout:95 punch from 0x63e921f to 0x63f37ac, (0xa58d bytes) 2024-09-17T21:29:45.605 INFO:teuthology.orchestra.run.smithi129.stdout:96 write 0x54362e6 thru 0x544196d (0xb688 bytes) 2024-09-17T21:29:45.609 INFO:teuthology.orchestra.run.smithi129.stdout:97 read 0x2194b14 thru 0x219673b (0x1c28 bytes) 2024-09-17T21:29:45.609 INFO:teuthology.orchestra.run.smithi129.stdout:102 read 0x17bf2af thru 0x17c97b1 (0xa503 bytes) 2024-09-17T21:29:45.610 INFO:teuthology.orchestra.run.smithi129.stdout:103 write 0x2192eae thru 0x21971ff (0x4352 bytes) 2024-09-17T21:29:45.613 INFO:teuthology.orchestra.run.smithi129.stdout:104 read 0x440a71d thru 0x4417701 (0xcfe5 bytes) 2024-09-17T21:29:45.613 INFO:teuthology.orchestra.run.smithi129.stdout:105 punch from 0x1e82c81 to 0x1e8f18c, (0xc50b bytes) 2024-09-17T21:29:45.626 INFO:teuthology.orchestra.run.smithi129.stdout:107 writesame 0xc8f164a thru 0xc8fc94a data_size 0x8d(0xb301 bytes) 2024-09-17T21:29:45.626 INFO:teuthology.orchestra.run.smithi129.stdout:109 write 0x886e6c4 thru 0x886ecde (0x61b bytes) 2024-09-17T21:29:45.629 INFO:teuthology.orchestra.run.smithi129.stdout:111 write 0xbb2dadb thru 0xbb319e8 (0x3f0e bytes) 2024-09-17T21:29:45.632 INFO:teuthology.orchestra.run.smithi129.stdout:112 read 0x72f3a72 thru 0x72f738e (0x391d bytes) 2024-09-17T21:29:45.632 INFO:teuthology.orchestra.run.smithi129.stdout:114 write 0xa1a3fb6 thru 0xa1ad5ce (0x9619 bytes) 2024-09-17T21:29:45.635 INFO:teuthology.orchestra.run.smithi129.stdout:116 read 0x4a25d47 thru 0x4a3209a (0xc354 bytes) 2024-09-17T21:29:45.635 INFO:teuthology.orchestra.run.smithi129.stdout:truncating to largest ever: 0xed4835f 2024-09-17T21:29:45.638 INFO:teuthology.orchestra.run.smithi129.stdout:118 trunc from 0xc8fc94b to 0xed4835f 2024-09-17T21:29:45.651 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:45.651 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:45.655 INFO:teuthology.orchestra.run.smithi129.stdout:119 trunc from 0xed4835f to 0x56105e0 2024-09-17T21:29:45.661 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:29:45.667 INFO:teuthology.orchestra.run.smithi129.stdout:120 read 0x17e2ff7 thru 0x17ec5f8 (0x9602 bytes) 2024-09-17T21:29:45.667 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:45.667 INFO:teuthology.orchestra.run.smithi129.stdout:122 punch from 0xc3b74d to 0xc45cae, (0xa561 bytes) 2024-09-17T21:29:45.667 INFO:teuthology.orchestra.run.smithi129.stdout:123 read 0x2f8449b thru 0x2f84cae (0x814 bytes) 2024-09-17T21:29:45.667 INFO:teuthology.orchestra.run.smithi129.stdout:124 write 0x1a98068 thru 0x1aa2606 (0xa59f bytes) 2024-09-17T21:29:45.670 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:45.670 INFO:teuthology.orchestra.run.smithi129.stdout:127 punch from 0x424549d to 0x4246163, (0xcc6 bytes) 2024-09-17T21:29:45.670 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:45.687 INFO:teuthology.orchestra.run.smithi129.stdout:129 write 0xe3d8429 thru 0xe3e2131 (0x9d09 bytes) 2024-09-17T21:29:45.692 INFO:teuthology.orchestra.run.smithi129.stdout:131 punch from 0x6e9894d to 0x6ea6706, (0xddb9 bytes) 2024-09-17T21:29:45.692 INFO:teuthology.orchestra.run.smithi129.stdout:132 write 0x6640226 thru 0x664b9b4 (0xb78f bytes) 2024-09-17T21:29:45.696 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:45.696 INFO:teuthology.orchestra.run.smithi129.stdout:134 read 0x5f39468 thru 0x5f48ed1 (0xfa6a bytes) 2024-09-17T21:29:45.696 INFO:teuthology.orchestra.run.smithi129.stdout:136 read 0x3138624 thru 0x313c029 (0x3a06 bytes) 2024-09-17T21:29:45.696 INFO:teuthology.orchestra.run.smithi129.stdout:139 read 0xba95a6c thru 0xbaa15ec (0xbb81 bytes) 2024-09-17T21:29:45.696 INFO:teuthology.orchestra.run.smithi129.stdout:140 write 0x9818180 thru 0x981b7a2 (0x3623 bytes) 2024-09-17T21:29:45.700 INFO:teuthology.orchestra.run.smithi129.stdout:141 writesame 0x3dcbf26 thru 0x3dda64a data_size 0x1(0xe725 bytes) 2024-09-17T21:29:45.704 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:45.702+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:45.729 INFO:teuthology.orchestra.run.smithi129.stdout:142 writesame 0xb476ff7 thru 0xb47e03d data_size 0x43(0x7047 bytes) 2024-09-17T21:29:45.733 INFO:teuthology.orchestra.run.smithi129.stdout:144 write 0x23a373d thru 0x23a6ebe (0x3782 bytes) 2024-09-17T21:29:45.735 INFO:teuthology.orchestra.run.smithi129.stdout:146 writesame 0x9976120 thru 0x99858f2 data_size 0x1(0xf7d3 bytes) 2024-09-17T21:29:45.764 INFO:teuthology.orchestra.run.smithi129.stdout:147 read 0xcd535d0 thru 0xcd552fc (0x1d2d bytes) 2024-09-17T21:29:45.764 INFO:teuthology.orchestra.run.smithi129.stdout:148 write 0xcd4a2a5 thru 0xcd57e04 (0xdb60 bytes) 2024-09-17T21:29:45.767 INFO:teuthology.orchestra.run.smithi129.stdout:149 read 0x377ea69 thru 0x378a264 (0xb7fc bytes) 2024-09-17T21:29:45.767 INFO:teuthology.orchestra.run.smithi129.stdout:153 write 0xd9d828d thru 0xd9e113f (0x8eb3 bytes) 2024-09-17T21:29:45.767 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:45.767 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:45.774 INFO:teuthology.orchestra.run.smithi129.stdout:154 writesame 0x91b09f8 thru 0x91bba8d data_size 0x7(0xb096 bytes) 2024-09-17T21:29:45.776 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:29:45.779 INFO:teuthology.orchestra.run.smithi129.stdout:155 punch from 0xc801a15 to 0xc80e7bd, (0xcda8 bytes) 2024-09-17T21:29:45.779 INFO:teuthology.orchestra.run.smithi129.stdout:156 read 0xca9cbe thru 0xcaa61a (0x95d bytes) 2024-09-17T21:29:45.779 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:45.779 INFO:teuthology.orchestra.run.smithi129.stdout:158 read 0xcc26722 thru 0xcc2ad2f (0x460e bytes) 2024-09-17T21:29:45.779 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:45.796 INFO:teuthology.orchestra.run.smithi129.stdout:160 trunc from 0xe3e2132 to 0x2780ea 2024-09-17T21:29:45.804 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:45.801+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:45.810 INFO:teuthology.orchestra.run.smithi129.stdout:161 read 0x6ecb9 thru 0x7e284 (0xf5cc bytes) 2024-09-17T21:29:45.810 INFO:teuthology.orchestra.run.smithi129.stdout:162 read 0x11ee91 thru 0x121a39 (0x2ba9 bytes) 2024-09-17T21:29:45.810 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:45.810 INFO:teuthology.orchestra.run.smithi129.stdout:164 read 0x272d6a thru 0x273abc (0xd53 bytes) 2024-09-17T21:29:45.811 INFO:teuthology.orchestra.run.smithi129.stdout:165 read 0x120552 thru 0x12d042 (0xcaf1 bytes) 2024-09-17T21:29:45.833 INFO:teuthology.orchestra.run.smithi129.stdout:166 write 0xb59fbd0 thru 0xb5a7394 (0x77c5 bytes) 2024-09-17T21:29:45.836 INFO:teuthology.orchestra.run.smithi129.stdout:167 read 0x13d6e06 thru 0x13e2dc1 (0xbfbc bytes) 2024-09-17T21:29:45.836 INFO:teuthology.orchestra.run.smithi129.stdout:168 writesame 0x2ee85c2 thru 0x2eef0af data_size 0x1(0x6aee bytes) 2024-09-17T21:29:45.851 INFO:teuthology.orchestra.run.smithi129.stdout:170 read 0x1620515 thru 0x162eb74 (0xe660 bytes) 2024-09-17T21:29:45.851 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:45.851 INFO:teuthology.orchestra.run.smithi129.stdout:175 write 0x9841be5 thru 0x9848bb2 (0x6fce bytes) 2024-09-17T21:29:45.854 INFO:teuthology.orchestra.run.smithi129.stdout:176 read 0x601701b thru 0x601726b (0x251 bytes) 2024-09-17T21:29:45.854 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:45.854 INFO:teuthology.orchestra.run.smithi129.stdout:180 read 0x5fa8fa6 thru 0x5faf51c (0x6577 bytes) 2024-09-17T21:29:45.854 INFO:teuthology.orchestra.run.smithi129.stdout:182 writesame 0x42f627c thru 0x42ffca7 data_size 0x5c(0x9a2c bytes) 2024-09-17T21:29:45.858 INFO:teuthology.orchestra.run.smithi129.stdout:183 write 0x3d95111 thru 0x3da3403 (0xe2f3 bytes) 2024-09-17T21:29:45.861 INFO:teuthology.orchestra.run.smithi129.stdout:184 write 0x900a14a thru 0x90120d1 (0x7f88 bytes) 2024-09-17T21:29:45.864 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:45.871 INFO:teuthology.orchestra.run.smithi129.stdout:188 write 0xc950ab2 thru 0xc951152 (0x6a1 bytes) 2024-09-17T21:29:45.878 INFO:teuthology.orchestra.run.smithi129.stdout:189 trunc from 0xc951153 to 0x95a8967 2024-09-17T21:29:45.888 INFO:teuthology.orchestra.run.smithi129.stdout:190 read 0x1ccdadb thru 0x1cd2f97 (0x54bd bytes) 2024-09-17T21:29:45.888 INFO:teuthology.orchestra.run.smithi129.stdout:191 punch from 0x732d0bb to 0x732d7cf, (0x714 bytes) 2024-09-17T21:29:45.891 INFO:teuthology.orchestra.run.smithi129.stdout:192 trunc from 0x95a8967 to 0x71ca8fa 2024-09-17T21:29:45.905 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:45.902+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:45.907 INFO:teuthology.orchestra.run.smithi129.stdout:193 writesame 0x773e313 thru 0x774ad4b data_size 0x1(0xca39 bytes) 2024-09-17T21:29:45.907 INFO:teuthology.orchestra.run.smithi129.stdout:194 read 0x5c21f0a thru 0x5c2efa4 (0xd09b bytes) 2024-09-17T21:29:45.907 INFO:teuthology.orchestra.run.smithi129.stdout:195 read 0x6ef2c0b thru 0x6ef6020 (0x3416 bytes) 2024-09-17T21:29:45.907 INFO:teuthology.orchestra.run.smithi129.stdout:197 writesame 0x7e6954 thru 0x7f616c data_size 0x3(0xf819 bytes) 2024-09-17T21:29:45.918 INFO:teuthology.orchestra.run.smithi129.stdout:198 writesame 0x6595c92 thru 0x6599673 data_size 0x1f(0x39e2 bytes) 2024-09-17T21:29:45.921 INFO:teuthology.orchestra.run.smithi129.stdout:199 write 0x33d464a thru 0x33e11d6 (0xcb8d bytes) 2024-09-17T21:29:45.928 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:45.928 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:45.937 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:29:45.938 INFO:teuthology.orchestra.run.smithi129.stdout:200 write 0xd12a0de thru 0xd12e371 (0x4294 bytes) 2024-09-17T21:29:45.940 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:45.940 INFO:teuthology.orchestra.run.smithi129.stdout:204 clone 1 order 21 su 8192 sc 7 2024-09-17T21:29:45.941 INFO:teuthology.orchestra.run.smithi129.stdout:moved 0 / 82 (0%) 2024-09-17T21:29:45.941 INFO:teuthology.orchestra.run.smithi129.stdout:avg 10.25 2024-09-17T21:29:45.941 INFO:teuthology.orchestra.run.smithi129.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-09-17T21:29:45.942 INFO:teuthology.orchestra.run.smithi129.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-17T21:29:45.942 INFO:teuthology.orchestra.run.smithi129.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-17T21:29:45.942 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:29:45.942 INFO:teuthology.orchestra.run.smithi129.stdout:oload 1000 2024-09-17T21:29:45.942 INFO:teuthology.orchestra.run.smithi129.stdout:max_change 3 2024-09-17T21:29:45.942 INFO:teuthology.orchestra.run.smithi129.stdout:max_change_osds 4 2024-09-17T21:29:45.942 INFO:teuthology.orchestra.run.smithi129.stdout:average_utilization 0.0003 2024-09-17T21:29:45.942 INFO:teuthology.orchestra.run.smithi129.stdout:overload_utilization 0.0029 2024-09-17T21:29:45.942 INFO:teuthology.orchestra.run.smithi129.stderr:no change 2024-09-17T21:29:45.954 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-17T21:29:45.954 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-09-17T21:29:46.006 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:46.004+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:46.041 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:46.041 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:46.050 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:29:46.107 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:46.105+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:46.193 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:46.194 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:46.202 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:29:46.246 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:46.244+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:46.279 INFO:teuthology.orchestra.run.smithi129.stdout:moved 0 / 82 (0%) 2024-09-17T21:29:46.279 INFO:teuthology.orchestra.run.smithi129.stdout:avg 10.25 2024-09-17T21:29:46.279 INFO:teuthology.orchestra.run.smithi129.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-09-17T21:29:46.279 INFO:teuthology.orchestra.run.smithi129.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-17T21:29:46.279 INFO:teuthology.orchestra.run.smithi129.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-17T21:29:46.279 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:29:46.279 INFO:teuthology.orchestra.run.smithi129.stdout:oload 1000 2024-09-17T21:29:46.279 INFO:teuthology.orchestra.run.smithi129.stdout:max_change 3 2024-09-17T21:29:46.279 INFO:teuthology.orchestra.run.smithi129.stdout:max_change_osds 4 2024-09-17T21:29:46.280 INFO:teuthology.orchestra.run.smithi129.stdout:average_utilization 0.0003 2024-09-17T21:29:46.280 INFO:teuthology.orchestra.run.smithi129.stdout:overload_utilization 0.0029 2024-09-17T21:29:46.280 INFO:teuthology.orchestra.run.smithi129.stderr:no change 2024-09-17T21:29:46.292 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-17T21:29:46.292 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-17T21:29:46.305 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:46.305 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:46.308 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:46.306+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:46.313 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:29:46.409 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:46.407+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:46.458 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:46.458 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:46.467 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:29:46.479 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is unset 2024-09-17T21:29:46.537 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:46.535+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:46.585 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:46.585 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:46.594 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:29:46.636 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:46.633+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:46.637 INFO:teuthology.orchestra.run.smithi129.stdout:moved 0 / 82 (0%) 2024-09-17T21:29:46.637 INFO:teuthology.orchestra.run.smithi129.stdout:avg 10.25 2024-09-17T21:29:46.638 INFO:teuthology.orchestra.run.smithi129.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-09-17T21:29:46.638 INFO:teuthology.orchestra.run.smithi129.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-17T21:29:46.638 INFO:teuthology.orchestra.run.smithi129.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-17T21:29:46.638 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:29:46.638 INFO:teuthology.orchestra.run.smithi129.stdout:oload 110 2024-09-17T21:29:46.638 INFO:teuthology.orchestra.run.smithi129.stdout:max_change 3 2024-09-17T21:29:46.638 INFO:teuthology.orchestra.run.smithi129.stdout:max_change_osds 4 2024-09-17T21:29:46.638 INFO:teuthology.orchestra.run.smithi129.stdout:average_utilization 0.0003 2024-09-17T21:29:46.638 INFO:teuthology.orchestra.run.smithi129.stdout:overload_utilization 0.0003 2024-09-17T21:29:46.638 INFO:teuthology.orchestra.run.smithi129.stderr:no change 2024-09-17T21:29:46.652 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-17T21:29:46.652 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-17T21:29:46.674 INFO:teuthology.orchestra.run.smithi129.stdout:206 write 0x26545f9 thru 0x2658ce1 (0x46e9 bytes) 2024-09-17T21:29:46.685 INFO:teuthology.orchestra.run.smithi129.stdout:207 read 0x524600 thru 0x52bdc4 (0x77c5 bytes) 2024-09-17T21:29:46.686 INFO:teuthology.orchestra.run.smithi129.stdout:209 punch from 0xccd8feb to 0xccda434, (0x1449 bytes) 2024-09-17T21:29:46.686 INFO:teuthology.orchestra.run.smithi129.stdout:212 read 0x9a3ecf6 thru 0x9a475c0 (0x88cb bytes) 2024-09-17T21:29:46.686 INFO:teuthology.orchestra.run.smithi129.stdout:213 writesame 0x1d1c7c5 thru 0x1d24815 data_size 0x1(0x8051 bytes) 2024-09-17T21:29:46.696 INFO:teuthology.orchestra.run.smithi129.stdout:214 read 0x155bcb thru 0x160178 (0xa5ae bytes) 2024-09-17T21:29:46.698 INFO:teuthology.orchestra.run.smithi129.stdout:215 write 0x950a4af thru 0x9513310 (0x8e62 bytes) 2024-09-17T21:29:46.712 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:46.709+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:46.713 INFO:teuthology.orchestra.run.smithi129.stdout:218 write 0xe194937 thru 0xe1a3653 (0xed1d bytes) 2024-09-17T21:29:46.722 INFO:teuthology.orchestra.run.smithi129.stdout:219 read 0x34a4eee thru 0x34b2114 (0xd227 bytes) 2024-09-17T21:29:46.730 INFO:teuthology.orchestra.run.smithi129.stdout:220 trunc from 0xe1a3654 to 0x7a149dc 2024-09-17T21:29:46.764 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:46.764 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:46.769 INFO:teuthology.orchestra.run.smithi129.stdout:221 write 0x9467424 thru 0x9476c7a (0xf857 bytes) 2024-09-17T21:29:46.775 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:29:46.782 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:29:46.785 INFO:teuthology.orchestra.run.smithi129.stdout:222 trunc from 0x9476c7b to 0x7d6127e 2024-09-17T21:29:46.809 INFO:teuthology.orchestra.run.smithi129.stdout:223 punch from 0x4e732aa to 0x4e78043, (0x4d99 bytes) 2024-09-17T21:29:46.819 INFO:teuthology.orchestra.run.smithi129.stdout:224 write 0xa709915 thru 0xa718431 (0xeb1d bytes) 2024-09-17T21:29:46.820 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:46.817+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:46.835 INFO:teuthology.orchestra.run.smithi129.stdout:225 writesame 0x2f10f70 thru 0x2f15584 data_size 0xb(0x4615 bytes) 2024-09-17T21:29:46.900 INFO:teuthology.orchestra.run.smithi129.stdout:226 trunc from 0xa718432 to 0x4438862 2024-09-17T21:29:46.914 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:46.911+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:46.935 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:46.935 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:46.949 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:29:46.958 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:29:46.958 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:29:46.986 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:29:46.987 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:29:47.015 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:47.013+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:47.017 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:29:47.017 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:29:47.044 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:29:47.045 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:29:47.050 INFO:teuthology.orchestra.run.smithi129.stdout:227 punch from 0x25d123e to 0x25e0ce1, (0xfaa3 bytes) 2024-09-17T21:29:47.060 INFO:teuthology.orchestra.run.smithi129.stdout:229 trunc from 0x4438862 to 0xc6e76b4 2024-09-17T21:29:47.062 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:29:47.062 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:29:47.065 INFO:teuthology.orchestra.run.smithi129.stdout:230 write 0x206db7e thru 0x2077d9d (0xa220 bytes) 2024-09-17T21:29:47.081 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:29:47.081 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:29:47.083 INFO:teuthology.orchestra.run.smithi129.stdout:231 trunc from 0xc6e76b4 to 0x18a4b52 2024-09-17T21:29:47.105 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:29:47.105 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:29:47.111 INFO:teuthology.orchestra.run.smithi129.stdout:233 trunc from 0x18a4b52 to 0x280ddab 2024-09-17T21:29:47.116 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:47.114+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:47.116 INFO:teuthology.orchestra.run.smithi129.stdout:235 read 0x12346cf thru 0x1236e10 (0x2742 bytes) 2024-09-17T21:29:47.117 INFO:teuthology.orchestra.run.smithi129.stdout:238 write 0x21b7c82 thru 0x21c2805 (0xab84 bytes) 2024-09-17T21:29:47.128 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:29:47.128 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:29:47.130 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:47.130 INFO:teuthology.orchestra.run.smithi129.stdout:240 read 0x855db0 thru 0x8586fc (0x294d bytes) 2024-09-17T21:29:47.130 INFO:teuthology.orchestra.run.smithi129.stdout:241 write 0xf8bc64 thru 0xf93450 (0x77ed bytes) 2024-09-17T21:29:47.133 INFO:teuthology.orchestra.run.smithi129.stdout:243 punch from 0x1dcf6fd to 0x1ddf25e, (0xfb61 bytes) 2024-09-17T21:29:47.133 INFO:teuthology.orchestra.run.smithi129.stdout:244 punch from 0x2633eb9 to 0x263c7f9, (0x8940 bytes) 2024-09-17T21:29:47.133 INFO:teuthology.orchestra.run.smithi129.stdout:246 write 0x803f1b thru 0x804370 (0x456 bytes) 2024-09-17T21:29:47.142 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:47.142 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:47.142 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:29:47.142 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:47.154 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:29:47.161 INFO:teuthology.orchestra.run.smithi129.stdout:moved 0 / 82 (0%) 2024-09-17T21:29:47.161 INFO:teuthology.orchestra.run.smithi129.stdout:avg 10.25 2024-09-17T21:29:47.161 INFO:teuthology.orchestra.run.smithi129.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-09-17T21:29:47.161 INFO:teuthology.orchestra.run.smithi129.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-17T21:29:47.161 INFO:teuthology.orchestra.run.smithi129.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-17T21:29:47.161 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:29:47.162 INFO:teuthology.orchestra.run.smithi129.stdout:oload 1000 2024-09-17T21:29:47.162 INFO:teuthology.orchestra.run.smithi129.stdout:max_change 3 2024-09-17T21:29:47.162 INFO:teuthology.orchestra.run.smithi129.stdout:max_change_osds 4 2024-09-17T21:29:47.162 INFO:teuthology.orchestra.run.smithi129.stdout:average_utilization 0.0003 2024-09-17T21:29:47.162 INFO:teuthology.orchestra.run.smithi129.stdout:overload_utilization 0.0029 2024-09-17T21:29:47.162 INFO:teuthology.orchestra.run.smithi129.stderr:no change 2024-09-17T21:29:47.174 INFO:teuthology.orchestra.run.smithi129.stdout:247 write 0x75e9aad thru 0x75ece36 (0x338a bytes) 2024-09-17T21:29:47.181 INFO:teuthology.orchestra.run.smithi129.stdout:248 punch from 0x2725a63 to 0x272bc86, (0x6223 bytes) 2024-09-17T21:29:47.182 INFO:teuthology.orchestra.run.smithi129.stdout:249 write 0x1652162 thru 0x165c1b3 (0xa052 bytes) 2024-09-17T21:29:47.185 INFO:teuthology.orchestra.run.smithi129.stdout:250 read 0x61d870 thru 0x621f9d (0x472e bytes) 2024-09-17T21:29:47.199 INFO:teuthology.orchestra.run.smithi129.stdout:251 write 0xca5de0a thru 0xca64ab1 (0x6ca8 bytes) 2024-09-17T21:29:47.208 INFO:teuthology.orchestra.run.smithi129.stdout:253 trunc from 0xca64ab2 to 0xcf05f27 2024-09-17T21:29:47.213 INFO:teuthology.orchestra.run.smithi129.stdout:254 read 0x8a136cd thru 0x8a15891 (0x21c5 bytes) 2024-09-17T21:29:47.213 INFO:teuthology.orchestra.run.smithi129.stdout:255 writesame 0x7a5c501 thru 0x7a62299 data_size 0x31(0x5d99 bytes) 2024-09-17T21:29:47.216 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:47.214+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:47.223 INFO:teuthology.orchestra.run.smithi129.stdout:257 read 0x90cc20a thru 0x90cd648 (0x143f bytes) 2024-09-17T21:29:47.227 INFO:teuthology.orchestra.run.smithi129.stdout:258 trunc from 0xcf05f27 to 0x92d7909 2024-09-17T21:29:47.240 INFO:teuthology.orchestra.run.smithi129.stdout:259 write 0x1854e5b thru 0x185c706 (0x78ac bytes) 2024-09-17T21:29:47.246 INFO:teuthology.orchestra.run.smithi129.stdout:261 write 0x3f810e1 thru 0x3f83aa7 (0x29c7 bytes) 2024-09-17T21:29:47.255 INFO:teuthology.orchestra.run.smithi129.stdout:263 trunc from 0x92d7909 to 0xc88607f 2024-09-17T21:29:47.261 INFO:teuthology.orchestra.run.smithi129.stdout:264 read 0xc08a791 thru 0xc096d9a (0xc60a bytes) 2024-09-17T21:29:47.261 INFO:teuthology.orchestra.run.smithi129.stdout:265 write 0x484416c thru 0x4853c45 (0xfada bytes) 2024-09-17T21:29:47.274 INFO:teuthology.orchestra.run.smithi129.stdout:266 read 0x6a4025d thru 0x6a4f0f7 (0xee9b bytes) 2024-09-17T21:29:47.274 INFO:teuthology.orchestra.run.smithi129.stdout:267 clone 2 order 18 su 2048 sc 5 2024-09-17T21:29:47.287 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:47.287 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:47.287 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:29:47.287 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:29:47.287 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:47.296 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:29:47.318 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:47.316+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:47.419 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:47.416+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:47.441 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:47.441 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:29:47.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:29:47.441 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:29:47.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.441 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2279 3.6 e29/25 3:6250b857:::rbd_object_map.11202c1b6cb4:head v 29'71, mlcod=29'67)", 2024-09-17T21:29:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.001355+0000", 2024-09-17T21:29:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.43706373599999998, 2024-09-17T21:29:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00466571, 2024-09-17T21:29:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.001355+0000", 2024-09-17T21:29:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.001355+0000", 2024-09-17T21:29:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.001356+0000", 2024-09-17T21:29:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:29:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.001361+0000", 2024-09-17T21:29:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2809999999999999e-06 2024-09-17T21:29:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.001372+0000", 2024-09-17T21:29:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1005000000000001e-05 2024-09-17T21:29:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.001379+0000", 2024-09-17T21:29:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8369999999999999e-06 2024-09-17T21:29:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.001427+0000", 2024-09-17T21:29:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8358e-05 2024-09-17T21:29:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.001450+0000", 2024-09-17T21:29:47.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2137000000000002e-05 2024-09-17T21:29:47.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.005977+0000", 2024-09-17T21:29:47.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045272089999999999 2024-09-17T21:29:47.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.006021+0000", 2024-09-17T21:29:47.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4214000000000001e-05 2024-09-17T21:29:47.446 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.446 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.446 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.446 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2281 3.1a e29/25 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:head v 29'119, mlcod=29'117)", 2024-09-17T21:29:47.446 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.002958+0000", 2024-09-17T21:29:47.446 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.43546062899999999, 2024-09-17T21:29:47.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0060199750000000003, 2024-09-17T21:29:47.446 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.446 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.446 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.446 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.002958+0000", 2024-09-17T21:29:47.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.002958+0000", 2024-09-17T21:29:47.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.002959+0000", 2024-09-17T21:29:47.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8200000000000008e-07 2024-09-17T21:29:47.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.003646+0000", 2024-09-17T21:29:47.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000686676 2024-09-17T21:29:47.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.003655+0000", 2024-09-17T21:29:47.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3719999999999994e-06 2024-09-17T21:29:47.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.003766+0000", 2024-09-17T21:29:47.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000110833 2024-09-17T21:29:47.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.003805+0000", 2024-09-17T21:29:47.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8974999999999999e-05 2024-09-17T21:29:47.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.003817+0000", 2024-09-17T21:29:47.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2255e-05 2024-09-17T21:29:47.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.008946+0000", 2024-09-17T21:29:47.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0051288369999999998 2024-09-17T21:29:47.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.008978+0000", 2024-09-17T21:29:47.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2045000000000002e-05 2024-09-17T21:29:47.450 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.450 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.450 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.451 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2282 3.6 e29/25 3:6250b857:::rbd_object_map.11202c1b6cb4:head v 29'72, mlcod=29'69)", 2024-09-17T21:29:47.451 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.003463+0000", 2024-09-17T21:29:47.451 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.43495572900000001, 2024-09-17T21:29:47.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0033672950000000002, 2024-09-17T21:29:47.451 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.451 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.451 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.451 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.003463+0000", 2024-09-17T21:29:47.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.003463+0000", 2024-09-17T21:29:47.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.003464+0000", 2024-09-17T21:29:47.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8899999999999998e-07 2024-09-17T21:29:47.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.003471+0000", 2024-09-17T21:29:47.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1459999999999999e-06 2024-09-17T21:29:47.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.003486+0000", 2024-09-17T21:29:47.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4594e-05 2024-09-17T21:29:47.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.003495+0000", 2024-09-17T21:29:47.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3219999999999999e-06 2024-09-17T21:29:47.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.003564+0000", 2024-09-17T21:29:47.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8604999999999998e-05 2024-09-17T21:29:47.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.003622+0000", 2024-09-17T21:29:47.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7760999999999998e-05 2024-09-17T21:29:47.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.006732+0000", 2024-09-17T21:29:47.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0031104679999999999 2024-09-17T21:29:47.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.006831+0000", 2024-09-17T21:29:47.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8510000000000004e-05 2024-09-17T21:29:47.454 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.455 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.455 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.455 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:2381 3.19 3:9aa62639:::rbd_data.11202c1b6cb4.0000000000000014:head [set-alloc-hint object_size 2097152 write_size 2097152,write 851968~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T21:29:47.455 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.123610+0000", 2024-09-17T21:29:47.455 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.314808637, 2024-09-17T21:29:47.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0025793819999999999, 2024-09-17T21:29:47.455 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.455 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.455 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.455 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:47.455 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:47.455 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:47.456 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2381 2024-09-17T21:29:47.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.456 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.123610+0000", 2024-09-17T21:29:47.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.123610+0000", 2024-09-17T21:29:47.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.123612+0000", 2024-09-17T21:29:47.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3230000000000001e-06 2024-09-17T21:29:47.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.123622+0000", 2024-09-17T21:29:47.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8130000000000007e-06 2024-09-17T21:29:47.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.123632+0000", 2024-09-17T21:29:47.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0944999999999999e-05 2024-09-17T21:29:47.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.123640+0000", 2024-09-17T21:29:47.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3719999999999999e-06 2024-09-17T21:29:47.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.123712+0000", 2024-09-17T21:29:47.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1735000000000004e-05 2024-09-17T21:29:47.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.123798+0000", 2024-09-17T21:29:47.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6543000000000001e-05 2024-09-17T21:29:47.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-17T21:29:47.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.123959+0000", 2024-09-17T21:29:47.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000161154 2024-09-17T21:29:47.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:29:47.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.126005+0000", 2024-09-17T21:29:47.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020455619999999999 2024-09-17T21:29:47.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:47.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.126083+0000", 2024-09-17T21:29:47.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8656e-05 2024-09-17T21:29:47.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.126142+0000", 2024-09-17T21:29:47.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8856e-05 2024-09-17T21:29:47.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.126190+0000", 2024-09-17T21:29:47.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7423000000000003e-05 2024-09-17T21:29:47.460 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.460 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.460 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.460 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:2680 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1203995~229] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T21:29:47.461 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.146485+0000", 2024-09-17T21:29:47.461 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.29193374900000002, 2024-09-17T21:29:47.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0041581209999999999, 2024-09-17T21:29:47.461 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.461 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.461 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:47.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:47.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:47.461 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2680 2024-09-17T21:29:47.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.461 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.146485+0000", 2024-09-17T21:29:47.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.146485+0000", 2024-09-17T21:29:47.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.146487+0000", 2024-09-17T21:29:47.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8959999999999999e-06 2024-09-17T21:29:47.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.147017+0000", 2024-09-17T21:29:47.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053010500000000001 2024-09-17T21:29:47.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.147034+0000", 2024-09-17T21:29:47.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7028e-05 2024-09-17T21:29:47.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.147044+0000", 2024-09-17T21:29:47.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3090000000000001e-06 2024-09-17T21:29:47.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.147123+0000", 2024-09-17T21:29:47.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9065000000000003e-05 2024-09-17T21:29:47.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.147280+0000", 2024-09-17T21:29:47.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00015779600000000001 2024-09-17T21:29:47.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-17T21:29:47.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.147556+0000", 2024-09-17T21:29:47.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00027551700000000003 2024-09-17T21:29:47.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:29:47.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.149981+0000", 2024-09-17T21:29:47.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0024251490000000001 2024-09-17T21:29:47.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:47.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.150524+0000", 2024-09-17T21:29:47.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00054332899999999995 2024-09-17T21:29:47.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.150601+0000", 2024-09-17T21:29:47.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6209000000000002e-05 2024-09-17T21:29:47.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.150643+0000", 2024-09-17T21:29:47.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2718e-05 2024-09-17T21:29:47.466 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.466 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.466 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.466 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2694 3.6 e29/25 3:6250b857:::rbd_object_map.11202c1b6cb4:head v 29'103, mlcod=29'99)", 2024-09-17T21:29:47.466 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.174943+0000", 2024-09-17T21:29:47.466 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.263476194, 2024-09-17T21:29:47.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0025509859999999999, 2024-09-17T21:29:47.466 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.466 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.466 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.467 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.174943+0000", 2024-09-17T21:29:47.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.174943+0000", 2024-09-17T21:29:47.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.174943+0000", 2024-09-17T21:29:47.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5300000000000004e-07 2024-09-17T21:29:47.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.174949+0000", 2024-09-17T21:29:47.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3879999999999999e-06 2024-09-17T21:29:47.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.174961+0000", 2024-09-17T21:29:47.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2088e-05 2024-09-17T21:29:47.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.174967+0000", 2024-09-17T21:29:47.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.263e-06 2024-09-17T21:29:47.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.175008+0000", 2024-09-17T21:29:47.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0735000000000002e-05 2024-09-17T21:29:47.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.175028+0000", 2024-09-17T21:29:47.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0095e-05 2024-09-17T21:29:47.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.177449+0000", 2024-09-17T21:29:47.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0024211520000000002 2024-09-17T21:29:47.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.177494+0000", 2024-09-17T21:29:47.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4611999999999999e-05 2024-09-17T21:29:47.470 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.470 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.470 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.470 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2716 3.6 e29/25 3:6250b857:::rbd_object_map.11202c1b6cb4:head v 29'106, mlcod=29'104)", 2024-09-17T21:29:47.470 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.199085+0000", 2024-09-17T21:29:47.470 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.23933423400000001, 2024-09-17T21:29:47.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0034940729999999999, 2024-09-17T21:29:47.471 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.471 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.471 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.471 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.199085+0000", 2024-09-17T21:29:47.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.199085+0000", 2024-09-17T21:29:47.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.199086+0000", 2024-09-17T21:29:47.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6499999999999998e-07 2024-09-17T21:29:47.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.199093+0000", 2024-09-17T21:29:47.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7659999999999998e-06 2024-09-17T21:29:47.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.199105+0000", 2024-09-17T21:29:47.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1728e-05 2024-09-17T21:29:47.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.199111+0000", 2024-09-17T21:29:47.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.947e-06 2024-09-17T21:29:47.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.199172+0000", 2024-09-17T21:29:47.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1216999999999993e-05 2024-09-17T21:29:47.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.199200+0000", 2024-09-17T21:29:47.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8104000000000001e-05 2024-09-17T21:29:47.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.202549+0000", 2024-09-17T21:29:47.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003348494 2024-09-17T21:29:47.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.202579+0000", 2024-09-17T21:29:47.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9952000000000001e-05 2024-09-17T21:29:47.474 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.474 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.475 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.475 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2717 3.6 e29/25 3:6250b857:::rbd_object_map.11202c1b6cb4:head v 29'107, mlcod=29'104)", 2024-09-17T21:29:47.475 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.199968+0000", 2024-09-17T21:29:47.475 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.23845148499999999, 2024-09-17T21:29:47.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0030262570000000001, 2024-09-17T21:29:47.475 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.475 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.475 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.475 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.199968+0000", 2024-09-17T21:29:47.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.199968+0000", 2024-09-17T21:29:47.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.199968+0000", 2024-09-17T21:29:47.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3099999999999997e-07 2024-09-17T21:29:47.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.199973+0000", 2024-09-17T21:29:47.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.972e-06 2024-09-17T21:29:47.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.199982+0000", 2024-09-17T21:29:47.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4959999999999996e-06 2024-09-17T21:29:47.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.199987+0000", 2024-09-17T21:29:47.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8810000000000004e-06 2024-09-17T21:29:47.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.200024+0000", 2024-09-17T21:29:47.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7632000000000002e-05 2024-09-17T21:29:47.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.200041+0000", 2024-09-17T21:29:47.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6906e-05 2024-09-17T21:29:47.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.202947+0000", 2024-09-17T21:29:47.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0029056490000000002 2024-09-17T21:29:47.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.202994+0000", 2024-09-17T21:29:47.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6990000000000002e-05 2024-09-17T21:29:47.479 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.479 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.479 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.479 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2736 3.6 e29/25 3:6250b857:::rbd_object_map.11202c1b6cb4:head v 29'113, mlcod=29'110)", 2024-09-17T21:29:47.479 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.213467+0000", 2024-09-17T21:29:47.479 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.22495211700000001, 2024-09-17T21:29:47.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002823393, 2024-09-17T21:29:47.479 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.479 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.480 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.480 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.213467+0000", 2024-09-17T21:29:47.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.213467+0000", 2024-09-17T21:29:47.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.213468+0000", 2024-09-17T21:29:47.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5899999999999996e-07 2024-09-17T21:29:47.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.213474+0000", 2024-09-17T21:29:47.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5760000000000002e-06 2024-09-17T21:29:47.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.213485+0000", 2024-09-17T21:29:47.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.119e-05 2024-09-17T21:29:47.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.213491+0000", 2024-09-17T21:29:47.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:29:47.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.213535+0000", 2024-09-17T21:29:47.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3693999999999999e-05 2024-09-17T21:29:47.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.213551+0000", 2024-09-17T21:29:47.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6345e-05 2024-09-17T21:29:47.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.216249+0000", 2024-09-17T21:29:47.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0026976449999999998 2024-09-17T21:29:47.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.216290+0000", 2024-09-17T21:29:47.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1783999999999998e-05 2024-09-17T21:29:47.483 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.483 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.483 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.483 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2737 3.6 e29/25 3:6250b857:::rbd_object_map.11202c1b6cb4:head v 29'114, mlcod=29'110)", 2024-09-17T21:29:47.483 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.214285+0000", 2024-09-17T21:29:47.483 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.22413395799999999, 2024-09-17T21:29:47.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045922560000000003, 2024-09-17T21:29:47.484 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.484 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.484 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.484 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.214285+0000", 2024-09-17T21:29:47.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.214285+0000", 2024-09-17T21:29:47.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.214286+0000", 2024-09-17T21:29:47.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7999999999999995e-07 2024-09-17T21:29:47.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.214291+0000", 2024-09-17T21:29:47.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7879999999999997e-06 2024-09-17T21:29:47.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.214299+0000", 2024-09-17T21:29:47.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0320000000000003e-06 2024-09-17T21:29:47.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.214303+0000", 2024-09-17T21:29:47.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3989999999999997e-06 2024-09-17T21:29:47.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.214346+0000", 2024-09-17T21:29:47.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2852000000000002e-05 2024-09-17T21:29:47.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.214364+0000", 2024-09-17T21:29:47.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8550999999999999e-05 2024-09-17T21:29:47.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.218834+0000", 2024-09-17T21:29:47.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0044697030000000002 2024-09-17T21:29:47.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.218877+0000", 2024-09-17T21:29:47.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3251000000000003e-05 2024-09-17T21:29:47.487 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.487 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.487 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.488 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2738 3.6 e29/25 3:6250b857:::rbd_object_map.11202c1b6cb4:head v 29'115, mlcod=29'110)", 2024-09-17T21:29:47.488 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.214955+0000", 2024-09-17T21:29:47.488 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.223464143, 2024-09-17T21:29:47.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003959733, 2024-09-17T21:29:47.488 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.488 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.488 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.488 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.214955+0000", 2024-09-17T21:29:47.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.214955+0000", 2024-09-17T21:29:47.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.214956+0000", 2024-09-17T21:29:47.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5899999999999996e-07 2024-09-17T21:29:47.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.214961+0000", 2024-09-17T21:29:47.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2460000000000003e-06 2024-09-17T21:29:47.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.214970+0000", 2024-09-17T21:29:47.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6460000000000005e-06 2024-09-17T21:29:47.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.214977+0000", 2024-09-17T21:29:47.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1539999999999999e-06 2024-09-17T21:29:47.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.215027+0000", 2024-09-17T21:29:47.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0074000000000002e-05 2024-09-17T21:29:47.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.215046+0000", 2024-09-17T21:29:47.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9828999999999999e-05 2024-09-17T21:29:47.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.218880+0000", 2024-09-17T21:29:47.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038335259999999999 2024-09-17T21:29:47.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.218915+0000", 2024-09-17T21:29:47.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4598999999999999e-05 2024-09-17T21:29:47.492 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.492 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.492 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.492 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2739 3.8 e29/25 3:163d28f2:::rbd_data.11202c1b6cb4.000000000000003b:head v 29'4, mlcod=29'2)", 2024-09-17T21:29:47.492 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.215797+0000", 2024-09-17T21:29:47.492 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.222622138, 2024-09-17T21:29:47.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0031238590000000001, 2024-09-17T21:29:47.492 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.492 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.493 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.493 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.215797+0000", 2024-09-17T21:29:47.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.215797+0000", 2024-09-17T21:29:47.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.215798+0000", 2024-09-17T21:29:47.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8999999999999996e-07 2024-09-17T21:29:47.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.215806+0000", 2024-09-17T21:29:47.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7539999999999995e-06 2024-09-17T21:29:47.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.215816+0000", 2024-09-17T21:29:47.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4180000000000003e-06 2024-09-17T21:29:47.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.215822+0000", 2024-09-17T21:29:47.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1840000000000004e-06 2024-09-17T21:29:47.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.215863+0000", 2024-09-17T21:29:47.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1037e-05 2024-09-17T21:29:47.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.215875+0000", 2024-09-17T21:29:47.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2012e-05 2024-09-17T21:29:47.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.218879+0000", 2024-09-17T21:29:47.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0030041410000000001 2024-09-17T21:29:47.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.218921+0000", 2024-09-17T21:29:47.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1622999999999998e-05 2024-09-17T21:29:47.496 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.496 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.496 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.496 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:2740 3.19 3:98002694:::rbd_data.11202c1b6cb4.000000000000003c:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1548288~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T21:29:47.497 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.216616+0000", 2024-09-17T21:29:47.497 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.221803053, 2024-09-17T21:29:47.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0031304269999999999, 2024-09-17T21:29:47.497 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.497 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.497 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.497 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:47.497 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:47.497 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:47.497 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2740 2024-09-17T21:29:47.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.497 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.216616+0000", 2024-09-17T21:29:47.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.216616+0000", 2024-09-17T21:29:47.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.216618+0000", 2024-09-17T21:29:47.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:29:47.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.216630+0000", 2024-09-17T21:29:47.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2567e-05 2024-09-17T21:29:47.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.216645+0000", 2024-09-17T21:29:47.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4537e-05 2024-09-17T21:29:47.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.216654+0000", 2024-09-17T21:29:47.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3700000000000001e-06 2024-09-17T21:29:47.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.216701+0000", 2024-09-17T21:29:47.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6768999999999999e-05 2024-09-17T21:29:47.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.216756+0000", 2024-09-17T21:29:47.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5381999999999999e-05 2024-09-17T21:29:47.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-17T21:29:47.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.216839+0000", 2024-09-17T21:29:47.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2799000000000005e-05 2024-09-17T21:29:47.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:47.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.219643+0000", 2024-09-17T21:29:47.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0028041680000000001 2024-09-17T21:29:47.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:29:47.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.219660+0000", 2024-09-17T21:29:47.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7215999999999999e-05 2024-09-17T21:29:47.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.219707+0000", 2024-09-17T21:29:47.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6344999999999998e-05 2024-09-17T21:29:47.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.219746+0000", 2024-09-17T21:29:47.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9552999999999999e-05 2024-09-17T21:29:47.502 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.502 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.502 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.502 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2779 3.6 e29/25 3:6250b857:::rbd_object_map.11202c1b6cb4:head v 29'124, mlcod=29'121)", 2024-09-17T21:29:47.502 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.262251+0000", 2024-09-17T21:29:47.503 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.17616759900000001, 2024-09-17T21:29:47.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0027320460000000001, 2024-09-17T21:29:47.503 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.503 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.503 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.503 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.262251+0000", 2024-09-17T21:29:47.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.262251+0000", 2024-09-17T21:29:47.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.262252+0000", 2024-09-17T21:29:47.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0399999999999996e-07 2024-09-17T21:29:47.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.262256+0000", 2024-09-17T21:29:47.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8659999999999999e-06 2024-09-17T21:29:47.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.262266+0000", 2024-09-17T21:29:47.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0492000000000001e-05 2024-09-17T21:29:47.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.262271+0000", 2024-09-17T21:29:47.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6380000000000003e-06 2024-09-17T21:29:47.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.262311+0000", 2024-09-17T21:29:47.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0216999999999998e-05 2024-09-17T21:29:47.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.262323+0000", 2024-09-17T21:29:47.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:29:47.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.264945+0000", 2024-09-17T21:29:47.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0026215819999999999 2024-09-17T21:29:47.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.264983+0000", 2024-09-17T21:29:47.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8597000000000003e-05 2024-09-17T21:29:47.507 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.507 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.507 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.507 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2780 3.6 e29/25 3:6250b857:::rbd_object_map.11202c1b6cb4:head v 29'125, mlcod=29'121)", 2024-09-17T21:29:47.507 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.262853+0000", 2024-09-17T21:29:47.507 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.17556645000000001, 2024-09-17T21:29:47.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0044931490000000001, 2024-09-17T21:29:47.507 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.507 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.507 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.508 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.262853+0000", 2024-09-17T21:29:47.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.262853+0000", 2024-09-17T21:29:47.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.262853+0000", 2024-09-17T21:29:47.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3099999999999997e-07 2024-09-17T21:29:47.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.262858+0000", 2024-09-17T21:29:47.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:29:47.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.262867+0000", 2024-09-17T21:29:47.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.884e-06 2024-09-17T21:29:47.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.262872+0000", 2024-09-17T21:29:47.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:29:47.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.262910+0000", 2024-09-17T21:29:47.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7829e-05 2024-09-17T21:29:47.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.262927+0000", 2024-09-17T21:29:47.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7462999999999999e-05 2024-09-17T21:29:47.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.267302+0000", 2024-09-17T21:29:47.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043745570000000003 2024-09-17T21:29:47.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.267346+0000", 2024-09-17T21:29:47.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3875999999999998e-05 2024-09-17T21:29:47.511 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.511 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.511 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.511 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2781 3.6 e29/25 3:6250b857:::rbd_object_map.11202c1b6cb4:head v 29'126, mlcod=29'121)", 2024-09-17T21:29:47.511 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.263450+0000", 2024-09-17T21:29:47.512 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.17496863800000001, 2024-09-17T21:29:47.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003913845, 2024-09-17T21:29:47.512 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.512 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.512 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.512 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.263450+0000", 2024-09-17T21:29:47.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.263450+0000", 2024-09-17T21:29:47.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.263451+0000", 2024-09-17T21:29:47.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6899999999999997e-07 2024-09-17T21:29:47.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.263456+0000", 2024-09-17T21:29:47.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.395e-06 2024-09-17T21:29:47.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.263468+0000", 2024-09-17T21:29:47.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1783000000000001e-05 2024-09-17T21:29:47.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.263474+0000", 2024-09-17T21:29:47.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1680000000000001e-06 2024-09-17T21:29:47.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.263519+0000", 2024-09-17T21:29:47.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4688000000000003e-05 2024-09-17T21:29:47.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.263539+0000", 2024-09-17T21:29:47.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0112e-05 2024-09-17T21:29:47.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.267348+0000", 2024-09-17T21:29:47.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003808428 2024-09-17T21:29:47.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.267364+0000", 2024-09-17T21:29:47.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6602000000000001e-05 2024-09-17T21:29:47.516 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.516 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.516 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.516 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2782 3.6 e29/25 3:6250b857:::rbd_object_map.11202c1b6cb4:head v 29'127, mlcod=29'122)", 2024-09-17T21:29:47.516 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.264038+0000", 2024-09-17T21:29:47.516 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.17438081499999999, 2024-09-17T21:29:47.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003349313, 2024-09-17T21:29:47.516 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.516 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.517 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.517 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.264038+0000", 2024-09-17T21:29:47.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.264038+0000", 2024-09-17T21:29:47.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.264039+0000", 2024-09-17T21:29:47.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3e-07 2024-09-17T21:29:47.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.264044+0000", 2024-09-17T21:29:47.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8400000000000002e-06 2024-09-17T21:29:47.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.264055+0000", 2024-09-17T21:29:47.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1347e-05 2024-09-17T21:29:47.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.264062+0000", 2024-09-17T21:29:47.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:29:47.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.264102+0000", 2024-09-17T21:29:47.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9607000000000002e-05 2024-09-17T21:29:47.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.264119+0000", 2024-09-17T21:29:47.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.783e-05 2024-09-17T21:29:47.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.267365+0000", 2024-09-17T21:29:47.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0032455069999999999 2024-09-17T21:29:47.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.267388+0000", 2024-09-17T21:29:47.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2609e-05 2024-09-17T21:29:47.520 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.520 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.520 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.520 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2783 3.6 e29/25 3:6250b857:::rbd_object_map.11202c1b6cb4:head v 29'128, mlcod=29'122)", 2024-09-17T21:29:47.520 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.264532+0000", 2024-09-17T21:29:47.521 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.17388715699999999, 2024-09-17T21:29:47.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0055353670000000002, 2024-09-17T21:29:47.521 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.521 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.521 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.521 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.264532+0000", 2024-09-17T21:29:47.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.264532+0000", 2024-09-17T21:29:47.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.264533+0000", 2024-09-17T21:29:47.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6700000000000003e-07 2024-09-17T21:29:47.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.264538+0000", 2024-09-17T21:29:47.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2839999999999997e-06 2024-09-17T21:29:47.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.264547+0000", 2024-09-17T21:29:47.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0440000000000001e-06 2024-09-17T21:29:47.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.264551+0000", 2024-09-17T21:29:47.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2150000000000003e-06 2024-09-17T21:29:47.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.264600+0000", 2024-09-17T21:29:47.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8690000000000003e-05 2024-09-17T21:29:47.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.264620+0000", 2024-09-17T21:29:47.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0071000000000001e-05 2024-09-17T21:29:47.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270026+0000", 2024-09-17T21:29:47.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00540633 2024-09-17T21:29:47.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270067+0000", 2024-09-17T21:29:47.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1066000000000003e-05 2024-09-17T21:29:47.524 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.524 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.524 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.525 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2784 3.6 e29/25 3:6250b857:::rbd_object_map.11202c1b6cb4:head v 29'129, mlcod=29'122)", 2024-09-17T21:29:47.525 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.265203+0000", 2024-09-17T21:29:47.525 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.173216121, 2024-09-17T21:29:47.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0048967029999999996, 2024-09-17T21:29:47.525 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.525 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.525 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.525 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.265203+0000", 2024-09-17T21:29:47.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.265203+0000", 2024-09-17T21:29:47.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.265203+0000", 2024-09-17T21:29:47.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0200000000000002e-07 2024-09-17T21:29:47.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.265207+0000", 2024-09-17T21:29:47.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5439999999999999e-06 2024-09-17T21:29:47.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.265216+0000", 2024-09-17T21:29:47.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7569999999999993e-06 2024-09-17T21:29:47.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.265220+0000", 2024-09-17T21:29:47.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5120000000000002e-06 2024-09-17T21:29:47.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.265260+0000", 2024-09-17T21:29:47.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9716999999999999e-05 2024-09-17T21:29:47.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.265276+0000", 2024-09-17T21:29:47.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5922000000000001e-05 2024-09-17T21:29:47.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270069+0000", 2024-09-17T21:29:47.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004792943 2024-09-17T21:29:47.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270100+0000", 2024-09-17T21:29:47.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0806000000000003e-05 2024-09-17T21:29:47.529 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.529 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.529 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.529 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:2790 3.0 3:02f4e340:::rbd_data.11202c1b6cb4.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write 335872~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T21:29:47.529 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.270496+0000", 2024-09-17T21:29:47.529 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.16792262499999999, 2024-09-17T21:29:47.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0026542250000000001, 2024-09-17T21:29:47.529 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.530 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.530 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:47.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:47.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:47.530 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2790 2024-09-17T21:29:47.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.530 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270496+0000", 2024-09-17T21:29:47.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270496+0000", 2024-09-17T21:29:47.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270498+0000", 2024-09-17T21:29:47.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8109999999999999e-06 2024-09-17T21:29:47.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270509+0000", 2024-09-17T21:29:47.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0560999999999999e-05 2024-09-17T21:29:47.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270523+0000", 2024-09-17T21:29:47.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4286e-05 2024-09-17T21:29:47.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270531+0000", 2024-09-17T21:29:47.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.977e-06 2024-09-17T21:29:47.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270594+0000", 2024-09-17T21:29:47.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2762999999999995e-05 2024-09-17T21:29:47.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270702+0000", 2024-09-17T21:29:47.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000108556 2024-09-17T21:29:47.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:29:47.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270820+0000", 2024-09-17T21:29:47.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000117367 2024-09-17T21:29:47.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:47.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.271957+0000", 2024-09-17T21:29:47.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0011376579999999999 2024-09-17T21:29:47.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:29:47.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.273036+0000", 2024-09-17T21:29:47.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00107895 2024-09-17T21:29:47.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.273082+0000", 2024-09-17T21:29:47.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5754000000000003e-05 2024-09-17T21:29:47.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.273151+0000", 2024-09-17T21:29:47.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8541999999999995e-05 2024-09-17T21:29:47.535 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.535 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.535 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.535 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.535 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.535 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:47.536 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:47.534+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:47.537 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:29:47.556 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone1 from 0xc88607f (overlap 0x18a4b52) to 0x13b6f74 2024-09-17T21:29:47.630 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-17T21:29:47.637 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:47.634+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:47.675 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:47.675 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:47.675 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:29:47.675 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:47.687 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:29:47.737 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:47.735+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:47.799 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:47.799 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:47.799 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:29:47.799 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:29:47.799 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:47.808 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:29:47.838 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:47.836+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:47.866 INFO:teuthology.orchestra.run.smithi129.stdout:268 write 0x294634d thru 0x295007f (0x9d33 bytes) 2024-09-17T21:29:47.889 INFO:teuthology.orchestra.run.smithi129.stdout:269 write 0x8b0fc90 thru 0x8b1b875 (0xbbe6 bytes) 2024-09-17T21:29:47.899 INFO:teuthology.orchestra.run.smithi129.stdout:270 read 0x56ae90c thru 0x56b1b11 (0x3206 bytes) 2024-09-17T21:29:47.899 INFO:teuthology.orchestra.run.smithi129.stdout:271 write 0x52451b3 thru 0x52493bb (0x4209 bytes) 2024-09-17T21:29:47.917 INFO:teuthology.orchestra.run.smithi129.stdout:272 writesame 0x9580f14 thru 0x958afe8 data_size 0x13(0xa0d5 bytes) 2024-09-17T21:29:47.923 INFO:teuthology.orchestra.run.smithi129.stdout:273 read 0x6811355 thru 0x68166d7 (0x5383 bytes) 2024-09-17T21:29:47.924 INFO:teuthology.orchestra.run.smithi129.stdout:275 punch from 0x7456a46 to 0x745edc3, (0x837d bytes) 2024-09-17T21:29:47.924 INFO:teuthology.orchestra.run.smithi129.stdout:278 read 0x4ea46b4 thru 0x4ea7b12 (0x345f bytes) 2024-09-17T21:29:47.924 INFO:teuthology.orchestra.run.smithi129.stdout:279 write 0x47d6168 thru 0x47dcd71 (0x6c0a bytes) 2024-09-17T21:29:47.933 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:47.933 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:29:47.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:29:47.933 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:29:47.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.934 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4269.0:1 2.3 e21/20 2:c4c92e5a:::rbd_trash:head v 21'1, mlcod=0'0)", 2024-09-17T21:29:47.934 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:32.973144+0000", 2024-09-17T21:29:47.934 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.958500923999999, 2024-09-17T21:29:47.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00080378800000000005, 2024-09-17T21:29:47.934 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.934 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.934 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.934 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973144+0000", 2024-09-17T21:29:47.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973144+0000", 2024-09-17T21:29:47.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973144+0000", 2024-09-17T21:29:47.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7599999999999998e-07 2024-09-17T21:29:47.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973159+0000", 2024-09-17T21:29:47.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4997999999999999e-05 2024-09-17T21:29:47.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973168+0000", 2024-09-17T21:29:47.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0380000000000005e-06 2024-09-17T21:29:47.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973176+0000", 2024-09-17T21:29:47.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0450000000000001e-06 2024-09-17T21:29:47.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973225+0000", 2024-09-17T21:29:47.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8899000000000001e-05 2024-09-17T21:29:47.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973248+0000", 2024-09-17T21:29:47.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3113e-05 2024-09-17T21:29:47.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973908+0000", 2024-09-17T21:29:47.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066012799999999995 2024-09-17T21:29:47.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973948+0000", 2024-09-17T21:29:47.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9190999999999998e-05 2024-09-17T21:29:47.938 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.938 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.938 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.938 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2790 3.0 e29/25 3:02f4e340:::rbd_data.11202c1b6cb4.0000000000000028:head v 29'3, mlcod=29'1)", 2024-09-17T21:29:47.938 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.270914+0000", 2024-09-17T21:29:47.938 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.66073051100000002, 2024-09-17T21:29:47.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000928888, 2024-09-17T21:29:47.938 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.938 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.939 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.939 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270914+0000", 2024-09-17T21:29:47.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270914+0000", 2024-09-17T21:29:47.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270915+0000", 2024-09-17T21:29:47.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.172e-06 2024-09-17T21:29:47.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270930+0000", 2024-09-17T21:29:47.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4629e-05 2024-09-17T21:29:47.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270947+0000", 2024-09-17T21:29:47.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7204999999999999e-05 2024-09-17T21:29:47.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270964+0000", 2024-09-17T21:29:47.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6569999999999999e-05 2024-09-17T21:29:47.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.271034+0000", 2024-09-17T21:29:47.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0226000000000002e-05 2024-09-17T21:29:47.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.271058+0000", 2024-09-17T21:29:47.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3808e-05 2024-09-17T21:29:47.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.271808+0000", 2024-09-17T21:29:47.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00075016199999999996 2024-09-17T21:29:47.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.271843+0000", 2024-09-17T21:29:47.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5116000000000001e-05 2024-09-17T21:29:47.942 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.942 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.942 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.943 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2843 3.0 e30/25 3:04479396:::rbd_id.image_client.0-clone2:head v 30'4, mlcod=29'2)", 2024-09-17T21:29:47.943 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.530358+0000", 2024-09-17T21:29:47.943 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.40128639100000002, 2024-09-17T21:29:47.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066505500000000001, 2024-09-17T21:29:47.943 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.943 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.943 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.943 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.530358+0000", 2024-09-17T21:29:47.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.530358+0000", 2024-09-17T21:29:47.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.530358+0000", 2024-09-17T21:29:47.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4700000000000001e-07 2024-09-17T21:29:47.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.530362+0000", 2024-09-17T21:29:47.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.546e-06 2024-09-17T21:29:47.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.530374+0000", 2024-09-17T21:29:47.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.201e-05 2024-09-17T21:29:47.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.530380+0000", 2024-09-17T21:29:47.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1879999999999999e-06 2024-09-17T21:29:47.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.530441+0000", 2024-09-17T21:29:47.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0427999999999998e-05 2024-09-17T21:29:47.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.530464+0000", 2024-09-17T21:29:47.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2935999999999999e-05 2024-09-17T21:29:47.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.530990+0000", 2024-09-17T21:29:47.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00052651300000000005 2024-09-17T21:29:47.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.531023+0000", 2024-09-17T21:29:47.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3287000000000002e-05 2024-09-17T21:29:47.947 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.947 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.947 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.947 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2892 3.0 e30/25 3:02f4e340:::rbd_data.11202c1b6cb4.0000000000000028:head v 30'6, mlcod=29'3)", 2024-09-17T21:29:47.947 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.562368+0000", 2024-09-17T21:29:47.947 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.36927700699999999, 2024-09-17T21:29:47.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000923331, 2024-09-17T21:29:47.947 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:47.947 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:47.948 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:47.948 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:47.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:47.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.562368+0000", 2024-09-17T21:29:47.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:47.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.562368+0000", 2024-09-17T21:29:47.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:47.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:47.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.562369+0000", 2024-09-17T21:29:47.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.344e-06 2024-09-17T21:29:47.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:47.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.562385+0000", 2024-09-17T21:29:47.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5815999999999999e-05 2024-09-17T21:29:47.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:47.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.562400+0000", 2024-09-17T21:29:47.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.488e-05 2024-09-17T21:29:47.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:47.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.562406+0000", 2024-09-17T21:29:47.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3149999999999997e-06 2024-09-17T21:29:47.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:47.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.562462+0000", 2024-09-17T21:29:47.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6271999999999999e-05 2024-09-17T21:29:47.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:47.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.562486+0000", 2024-09-17T21:29:47.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3909000000000001e-05 2024-09-17T21:29:47.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:47.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.563258+0000", 2024-09-17T21:29:47.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00077164499999999995 2024-09-17T21:29:47.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:47.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:47.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:47.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.563291+0000", 2024-09-17T21:29:47.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3149999999999999e-05 2024-09-17T21:29:47.951 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.951 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.951 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.951 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:47.951 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:47.952 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:47.953 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:29:47.953 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:47.954 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:47.954 INFO:teuthology.orchestra.run.smithi129.stdout:283 write 0x8cf78a0 thru 0x8cff99c (0x80fd bytes) 2024-09-17T21:29:47.954 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:47.950+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:47.955 INFO:teuthology.orchestra.run.smithi129.stdout:284 read 0x4feb909 thru 0x4ff7082 (0xb77a bytes) 2024-09-17T21:29:47.957 INFO:teuthology.orchestra.run.smithi129.stdout:285 punch from 0xa74cb24 to 0xa759c64, (0xd140 bytes) 2024-09-17T21:29:47.957 INFO:teuthology.orchestra.run.smithi129.stdout:287 read 0x372cddc thru 0x3739ca1 (0xcec6 bytes) 2024-09-17T21:29:47.957 INFO:teuthology.orchestra.run.smithi129.stdout:288 punch from 0x36568d8 to 0x36588e6, (0x200e bytes) 2024-09-17T21:29:47.957 INFO:teuthology.orchestra.run.smithi129.stdout:289 write 0x24c21a3 thru 0x24c3a31 (0x188f bytes) 2024-09-17T21:29:47.978 INFO:teuthology.orchestra.run.smithi129.stdout:290 clone 3 order 22 su 131072 sc 6 2024-09-17T21:29:48.053 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:48.051+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:48.061 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:48.061 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:48.061 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:29:48.061 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:48.069 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:29:48.154 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:48.152+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:48.212 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:48.212 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:48.213 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:29:48.213 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:29:48.213 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:48.221 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:29:48.262 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:48.260+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:48.355 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:48.353+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:48.366 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:29:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:29:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:29:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:3841 3.18 3:1ae66b4d:::rbd_data.11202c1b6cb4.0000000000000029:4 [read 1828864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.906210+0000", 2024-09-17T21:29:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.45749705800000001, 2024-09-17T21:29:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0066371349999999997, 2024-09-17T21:29:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3841 2024-09-17T21:29:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.906210+0000", 2024-09-17T21:29:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.906210+0000", 2024-09-17T21:29:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.906210+0000", 2024-09-17T21:29:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.22e-07 2024-09-17T21:29:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.906212+0000", 2024-09-17T21:29:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6670000000000001e-06 2024-09-17T21:29:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.906218+0000", 2024-09-17T21:29:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7189999999999998e-06 2024-09-17T21:29:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.906222+0000", 2024-09-17T21:29:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:29:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.912749+0000", 2024-09-17T21:29:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065269719999999998 2024-09-17T21:29:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.912793+0000", 2024-09-17T21:29:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3560000000000003e-05 2024-09-17T21:29:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.912847+0000", 2024-09-17T21:29:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4422000000000002e-05 2024-09-17T21:29:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:3842 3.18 3:1ae66b4d:::rbd_data.11202c1b6cb4.0000000000000029:4 [read 1841152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.906230+0000", 2024-09-17T21:29:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.45747744099999998, 2024-09-17T21:29:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068354410000000003, 2024-09-17T21:29:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3842 2024-09-17T21:29:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.906230+0000", 2024-09-17T21:29:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.906230+0000", 2024-09-17T21:29:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.906230+0000", 2024-09-17T21:29:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3200000000000001e-07 2024-09-17T21:29:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.906232+0000", 2024-09-17T21:29:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5629999999999999e-06 2024-09-17T21:29:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.906237+0000", 2024-09-17T21:29:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5400000000000003e-06 2024-09-17T21:29:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.906242+0000", 2024-09-17T21:29:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6079999999999998e-06 2024-09-17T21:29:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.912887+0000", 2024-09-17T21:29:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006645354 2024-09-17T21:29:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.912979+0000", 2024-09-17T21:29:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1707999999999997e-05 2024-09-17T21:29:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.913065+0000", 2024-09-17T21:29:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6335999999999995e-05 2024-09-17T21:29:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:3843 3.18 3:1ae66b4d:::rbd_data.11202c1b6cb4.0000000000000029:4 [read 1855488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.906249+0000", 2024-09-17T21:29:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.45745821199999998, 2024-09-17T21:29:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069480829999999999, 2024-09-17T21:29:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3843 2024-09-17T21:29:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.906249+0000", 2024-09-17T21:29:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.906249+0000", 2024-09-17T21:29:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.906249+0000", 2024-09-17T21:29:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:29:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.906256+0000", 2024-09-17T21:29:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2200000000000003e-06 2024-09-17T21:29:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.906263+0000", 2024-09-17T21:29:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0789999999999997e-06 2024-09-17T21:29:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.906267+0000", 2024-09-17T21:29:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3930000000000001e-06 2024-09-17T21:29:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.913100+0000", 2024-09-17T21:29:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068327040000000002 2024-09-17T21:29:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.913150+0000", 2024-09-17T21:29:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0407e-05 2024-09-17T21:29:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.913197+0000", 2024-09-17T21:29:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.689e-05 2024-09-17T21:29:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:4280 3.b 3:d1c05e84:::rbd_data.11202c1b6cb4.0000000000000024:4 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.932295+0000", 2024-09-17T21:29:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.43141200000000002, 2024-09-17T21:29:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0066332470000000001, 2024-09-17T21:29:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 4280 2024-09-17T21:29:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932295+0000", 2024-09-17T21:29:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932295+0000", 2024-09-17T21:29:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932296+0000", 2024-09-17T21:29:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:29:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932298+0000", 2024-09-17T21:29:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.492e-06 2024-09-17T21:29:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932305+0000", 2024-09-17T21:29:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9299999999999997e-06 2024-09-17T21:29:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932310+0000", 2024-09-17T21:29:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5390000000000001e-06 2024-09-17T21:29:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.938770+0000", 2024-09-17T21:29:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0064601779999999996 2024-09-17T21:29:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.938843+0000", 2024-09-17T21:29:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3184999999999998e-05 2024-09-17T21:29:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.938928+0000", 2024-09-17T21:29:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5395999999999994e-05 2024-09-17T21:29:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:4281 3.b 3:d1c05e84:::rbd_data.11202c1b6cb4.0000000000000024:4 [read 272384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.932342+0000", 2024-09-17T21:29:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.43136530899999997, 2024-09-17T21:29:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067926169999999999, 2024-09-17T21:29:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 4281 2024-09-17T21:29:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932342+0000", 2024-09-17T21:29:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932342+0000", 2024-09-17T21:29:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932342+0000", 2024-09-17T21:29:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6400000000000002e-07 2024-09-17T21:29:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932345+0000", 2024-09-17T21:29:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6029999999999999e-06 2024-09-17T21:29:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932352+0000", 2024-09-17T21:29:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2169999999999997e-06 2024-09-17T21:29:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932357+0000", 2024-09-17T21:29:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5600000000000004e-06 2024-09-17T21:29:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.938961+0000", 2024-09-17T21:29:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0066041859999999997 2024-09-17T21:29:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939046+0000", 2024-09-17T21:29:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5487000000000003e-05 2024-09-17T21:29:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939134+0000", 2024-09-17T21:29:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7999999999999998e-05 2024-09-17T21:29:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:4282 3.b 3:d1c05e84:::rbd_data.11202c1b6cb4.0000000000000024:4 [read 284672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.932391+0000", 2024-09-17T21:29:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.43131604200000001, 2024-09-17T21:29:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069148969999999997, 2024-09-17T21:29:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 4282 2024-09-17T21:29:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932391+0000", 2024-09-17T21:29:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932391+0000", 2024-09-17T21:29:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932392+0000", 2024-09-17T21:29:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2600000000000002e-07 2024-09-17T21:29:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932394+0000", 2024-09-17T21:29:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5849999999999998e-06 2024-09-17T21:29:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932401+0000", 2024-09-17T21:29:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1709999999999996e-06 2024-09-17T21:29:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932407+0000", 2024-09-17T21:29:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4190000000000001e-06 2024-09-17T21:29:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939164+0000", 2024-09-17T21:29:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067568519999999998 2024-09-17T21:29:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939224+0000", 2024-09-17T21:29:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0232999999999997e-05 2024-09-17T21:29:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939306+0000", 2024-09-17T21:29:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2010999999999998e-05 2024-09-17T21:29:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:4283 3.b 3:d1c05e84:::rbd_data.11202c1b6cb4.0000000000000024:4 [read 299008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.932414+0000", 2024-09-17T21:29:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.43129278100000001, 2024-09-17T21:29:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0070075909999999996, 2024-09-17T21:29:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 4283 2024-09-17T21:29:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932414+0000", 2024-09-17T21:29:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932414+0000", 2024-09-17T21:29:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932415+0000", 2024-09-17T21:29:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.37e-07 2024-09-17T21:29:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932417+0000", 2024-09-17T21:29:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:29:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932423+0000", 2024-09-17T21:29:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.891e-06 2024-09-17T21:29:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932428+0000", 2024-09-17T21:29:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:29:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939332+0000", 2024-09-17T21:29:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069046259999999996 2024-09-17T21:29:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939374+0000", 2024-09-17T21:29:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1777999999999999e-05 2024-09-17T21:29:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939422+0000", 2024-09-17T21:29:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7799000000000001e-05 2024-09-17T21:29:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:4284 3.b 3:d1c05e84:::rbd_data.11202c1b6cb4.0000000000000024:4 [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.932434+0000", 2024-09-17T21:29:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.43127263500000002, 2024-09-17T21:29:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071635249999999996, 2024-09-17T21:29:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 4284 2024-09-17T21:29:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932434+0000", 2024-09-17T21:29:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932434+0000", 2024-09-17T21:29:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932435+0000", 2024-09-17T21:29:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.53e-07 2024-09-17T21:29:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932436+0000", 2024-09-17T21:29:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3260000000000001e-06 2024-09-17T21:29:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932439+0000", 2024-09-17T21:29:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2550000000000002e-06 2024-09-17T21:29:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932442+0000", 2024-09-17T21:29:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0079999999999998e-06 2024-09-17T21:29:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939450+0000", 2024-09-17T21:29:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0070071730000000002 2024-09-17T21:29:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939507+0000", 2024-09-17T21:29:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6950999999999997e-05 2024-09-17T21:29:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939598+0000", 2024-09-17T21:29:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1459000000000006e-05 2024-09-17T21:29:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:4285 3.b 3:d1c05e84:::rbd_data.11202c1b6cb4.0000000000000024:4 [read 325632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.932446+0000", 2024-09-17T21:29:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.43126136399999998, 2024-09-17T21:29:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0073844649999999998, 2024-09-17T21:29:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 4285 2024-09-17T21:29:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932446+0000", 2024-09-17T21:29:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932446+0000", 2024-09-17T21:29:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932446+0000", 2024-09-17T21:29:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.16e-07 2024-09-17T21:29:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932450+0000", 2024-09-17T21:29:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.614e-06 2024-09-17T21:29:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932453+0000", 2024-09-17T21:29:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6109999999999998e-06 2024-09-17T21:29:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932456+0000", 2024-09-17T21:29:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6560000000000001e-06 2024-09-17T21:29:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939639+0000", 2024-09-17T21:29:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071832470000000002 2024-09-17T21:29:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939724+0000", 2024-09-17T21:29:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4583e-05 2024-09-17T21:29:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939830+0000", 2024-09-17T21:29:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106538 2024-09-17T21:29:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:4286 3.b 3:d1c05e84:::rbd_data.11202c1b6cb4.0000000000000024:4 [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.932461+0000", 2024-09-17T21:29:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.431245875, 2024-09-17T21:29:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0075573749999999999, 2024-09-17T21:29:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 4286 2024-09-17T21:29:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932461+0000", 2024-09-17T21:29:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932461+0000", 2024-09-17T21:29:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932462+0000", 2024-09-17T21:29:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:29:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932463+0000", 2024-09-17T21:29:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7990000000000001e-06 2024-09-17T21:29:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932469+0000", 2024-09-17T21:29:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2569999999999998e-06 2024-09-17T21:29:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932522+0000", 2024-09-17T21:29:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3242999999999998e-05 2024-09-17T21:29:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939859+0000", 2024-09-17T21:29:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0073368289999999996 2024-09-17T21:29:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939931+0000", 2024-09-17T21:29:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2399999999999998e-05 2024-09-17T21:29:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940019+0000", 2024-09-17T21:29:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7343999999999996e-05 2024-09-17T21:29:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:4287 3.b 3:d1c05e84:::rbd_data.11202c1b6cb4.0000000000000024:4 [read 354304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.932532+0000", 2024-09-17T21:29:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.43117472499999998, 2024-09-17T21:29:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.007688912, 2024-09-17T21:29:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 4287 2024-09-17T21:29:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932532+0000", 2024-09-17T21:29:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932532+0000", 2024-09-17T21:29:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932533+0000", 2024-09-17T21:29:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3099999999999997e-07 2024-09-17T21:29:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932536+0000", 2024-09-17T21:29:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8499999999999998e-06 2024-09-17T21:29:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932544+0000", 2024-09-17T21:29:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1410000000000005e-06 2024-09-17T21:29:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932550+0000", 2024-09-17T21:29:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7270000000000004e-06 2024-09-17T21:29:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940054+0000", 2024-09-17T21:29:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0075042490000000002 2024-09-17T21:29:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940141+0000", 2024-09-17T21:29:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6708999999999999e-05 2024-09-17T21:29:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940221+0000", 2024-09-17T21:29:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0505000000000003e-05 2024-09-17T21:29:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:4288 3.b 3:d1c05e84:::rbd_data.11202c1b6cb4.0000000000000024:4 [read 366592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.932557+0000", 2024-09-17T21:29:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.43115039700000002, 2024-09-17T21:29:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0078542879999999992, 2024-09-17T21:29:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 4288 2024-09-17T21:29:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932557+0000", 2024-09-17T21:29:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932557+0000", 2024-09-17T21:29:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932557+0000", 2024-09-17T21:29:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9100000000000004e-07 2024-09-17T21:29:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932559+0000", 2024-09-17T21:29:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8619999999999999e-06 2024-09-17T21:29:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932565+0000", 2024-09-17T21:29:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5940000000000001e-06 2024-09-17T21:29:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932569+0000", 2024-09-17T21:29:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:29:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940253+0000", 2024-09-17T21:29:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0076837859999999997 2024-09-17T21:29:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940329+0000", 2024-09-17T21:29:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5545999999999998e-05 2024-09-17T21:29:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940411+0000", 2024-09-17T21:29:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2367000000000006e-05 2024-09-17T21:29:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:4329 3.2 3:420dab1b:::rbd_data.11202c1b6cb4.0000000000000027:4 [read 362496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.932924+0000", 2024-09-17T21:29:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.43078299199999998, 2024-09-17T21:29:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067678679999999998, 2024-09-17T21:29:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 4329 2024-09-17T21:29:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932924+0000", 2024-09-17T21:29:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932924+0000", 2024-09-17T21:29:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932924+0000", 2024-09-17T21:29:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:29:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932926+0000", 2024-09-17T21:29:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9180000000000001e-06 2024-09-17T21:29:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932932+0000", 2024-09-17T21:29:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5520000000000004e-06 2024-09-17T21:29:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932936+0000", 2024-09-17T21:29:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3499999999999999e-06 2024-09-17T21:29:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939525+0000", 2024-09-17T21:29:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065889879999999996 2024-09-17T21:29:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939604+0000", 2024-09-17T21:29:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8596000000000004e-05 2024-09-17T21:29:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939692+0000", 2024-09-17T21:29:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8096999999999994e-05 2024-09-17T21:29:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:4330 3.2 3:420dab1b:::rbd_data.11202c1b6cb4.0000000000000027:4 [read 374784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.932942+0000", 2024-09-17T21:29:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.430764906, 2024-09-17T21:29:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068094749999999997, 2024-09-17T21:29:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 4330 2024-09-17T21:29:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932942+0000", 2024-09-17T21:29:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932942+0000", 2024-09-17T21:29:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932943+0000", 2024-09-17T21:29:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6100000000000002e-07 2024-09-17T21:29:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932944+0000", 2024-09-17T21:29:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6390000000000001e-06 2024-09-17T21:29:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932951+0000", 2024-09-17T21:29:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0840000000000003e-06 2024-09-17T21:29:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.932956+0000", 2024-09-17T21:29:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7759999999999997e-06 2024-09-17T21:29:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939660+0000", 2024-09-17T21:29:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067041080000000003 2024-09-17T21:29:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939717+0000", 2024-09-17T21:29:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6372000000000002e-05 2024-09-17T21:29:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939752+0000", 2024-09-17T21:29:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5135000000000002e-05 2024-09-17T21:29:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:4353 3.18 3:1ae66b4d:::rbd_data.11202c1b6cb4.0000000000000029:4 [read 299008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.933138+0000", 2024-09-17T21:29:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.43056944800000002, 2024-09-17T21:29:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067004400000000002, 2024-09-17T21:29:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 4353 2024-09-17T21:29:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933138+0000", 2024-09-17T21:29:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933138+0000", 2024-09-17T21:29:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933138+0000", 2024-09-17T21:29:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4299999999999999e-07 2024-09-17T21:29:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933152+0000", 2024-09-17T21:29:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.384e-05 2024-09-17T21:29:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933158+0000", 2024-09-17T21:29:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5429999999999999e-06 2024-09-17T21:29:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933163+0000", 2024-09-17T21:29:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8849999999999998e-06 2024-09-17T21:29:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939745+0000", 2024-09-17T21:29:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065815329999999997 2024-09-17T21:29:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939788+0000", 2024-09-17T21:29:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3556000000000001e-05 2024-09-17T21:29:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939838+0000", 2024-09-17T21:29:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9740000000000001e-05 2024-09-17T21:29:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:4354 3.18 3:1ae66b4d:::rbd_data.11202c1b6cb4.0000000000000029:4 [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.933171+0000", 2024-09-17T21:29:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.430535579, 2024-09-17T21:29:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068081239999999996, 2024-09-17T21:29:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 4354 2024-09-17T21:29:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933171+0000", 2024-09-17T21:29:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933171+0000", 2024-09-17T21:29:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933172+0000", 2024-09-17T21:29:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1099999999999996e-07 2024-09-17T21:29:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933174+0000", 2024-09-17T21:29:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.142e-06 2024-09-17T21:29:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933180+0000", 2024-09-17T21:29:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7679999999999997e-06 2024-09-17T21:29:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933189+0000", 2024-09-17T21:29:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1300000000000007e-06 2024-09-17T21:29:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939871+0000", 2024-09-17T21:29:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0066821199999999997 2024-09-17T21:29:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939927+0000", 2024-09-17T21:29:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5581999999999997e-05 2024-09-17T21:29:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.939980+0000", 2024-09-17T21:29:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2871000000000003e-05 2024-09-17T21:29:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:4355 3.18 3:1ae66b4d:::rbd_data.11202c1b6cb4.0000000000000029:4 [read 325632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.933196+0000", 2024-09-17T21:29:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.430511278, 2024-09-17T21:29:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069757960000000003, 2024-09-17T21:29:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 4355 2024-09-17T21:29:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933196+0000", 2024-09-17T21:29:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933196+0000", 2024-09-17T21:29:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933196+0000", 2024-09-17T21:29:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7500000000000001e-07 2024-09-17T21:29:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933198+0000", 2024-09-17T21:29:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9719999999999999e-06 2024-09-17T21:29:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933204+0000", 2024-09-17T21:29:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6640000000000003e-06 2024-09-17T21:29:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933213+0000", 2024-09-17T21:29:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8289999999999997e-06 2024-09-17T21:29:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940003+0000", 2024-09-17T21:29:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067905810000000004 2024-09-17T21:29:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940065+0000", 2024-09-17T21:29:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1684000000000001e-05 2024-09-17T21:29:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940172+0000", 2024-09-17T21:29:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106691 2024-09-17T21:29:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:4356 3.18 3:1ae66b4d:::rbd_data.11202c1b6cb4.0000000000000029:4 [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.933219+0000", 2024-09-17T21:29:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.430487707, 2024-09-17T21:29:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0072910290000000001, 2024-09-17T21:29:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 4356 2024-09-17T21:29:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933219+0000", 2024-09-17T21:29:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933219+0000", 2024-09-17T21:29:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933220+0000", 2024-09-17T21:29:48.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:29:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933222+0000", 2024-09-17T21:29:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.95e-06 2024-09-17T21:29:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933228+0000", 2024-09-17T21:29:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9050000000000002e-06 2024-09-17T21:29:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933232+0000", 2024-09-17T21:29:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4870000000000004e-06 2024-09-17T21:29:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940207+0000", 2024-09-17T21:29:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069746399999999998 2024-09-17T21:29:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940290+0000", 2024-09-17T21:29:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3014000000000002e-05 2024-09-17T21:29:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940510+0000", 2024-09-17T21:29:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022062899999999999 2024-09-17T21:29:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:4357 3.18 3:1ae66b4d:::rbd_data.11202c1b6cb4.0000000000000029:4 [read 354304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.933238+0000", 2024-09-17T21:29:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.43046887700000003, 2024-09-17T21:29:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0074035070000000001, 2024-09-17T21:29:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 4357 2024-09-17T21:29:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933238+0000", 2024-09-17T21:29:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933238+0000", 2024-09-17T21:29:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933239+0000", 2024-09-17T21:29:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8800000000000003e-07 2024-09-17T21:29:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933240+0000", 2024-09-17T21:29:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7770000000000001e-06 2024-09-17T21:29:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933247+0000", 2024-09-17T21:29:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5400000000000001e-06 2024-09-17T21:29:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933252+0000", 2024-09-17T21:29:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:29:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940540+0000", 2024-09-17T21:29:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0072883990000000001 2024-09-17T21:29:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940597+0000", 2024-09-17T21:29:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7028999999999999e-05 2024-09-17T21:29:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940642+0000", 2024-09-17T21:29:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4413999999999999e-05 2024-09-17T21:29:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:4358 3.18 3:1ae66b4d:::rbd_data.11202c1b6cb4.0000000000000029:4 [read 366592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.933258+0000", 2024-09-17T21:29:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.43044895, 2024-09-17T21:29:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0075000860000000004, 2024-09-17T21:29:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 4358 2024-09-17T21:29:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933258+0000", 2024-09-17T21:29:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933258+0000", 2024-09-17T21:29:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933258+0000", 2024-09-17T21:29:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:29:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933260+0000", 2024-09-17T21:29:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.827e-06 2024-09-17T21:29:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933265+0000", 2024-09-17T21:29:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1909999999999999e-06 2024-09-17T21:29:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.933270+0000", 2024-09-17T21:29:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4700000000000004e-06 2024-09-17T21:29:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940670+0000", 2024-09-17T21:29:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.007400495 2024-09-17T21:29:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940717+0000", 2024-09-17T21:29:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.693e-05 2024-09-17T21:29:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.940758+0000", 2024-09-17T21:29:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0831000000000003e-05 2024-09-17T21:29:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.459 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.459 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.459 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:48.459 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:48.459 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:48.460 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:29:48.461 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:48.458+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:48.496 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:29:48.561 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:48.559+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:48.567 INFO:teuthology.orchestra.run.smithi129.stdout:flattening image image_client.0-clone2 2024-09-17T21:29:48.577 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:48.577 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:29:48.577 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:29:48.577 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:48.586 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:29:48.662 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:48.659+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:48.690 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:48.691 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:29:48.691 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:29:48.691 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:29:48.691 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:48.699 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:29:48.763 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:48.760+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:48.813 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:29:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:29:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:29:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:2885 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.556534+0000", 2024-09-17T21:29:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.252857989, 2024-09-17T21:29:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010091376000000001, 2024-09-17T21:29:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 2885 2024-09-17T21:29:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556534+0000", 2024-09-17T21:29:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556534+0000", 2024-09-17T21:29:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556535+0000", 2024-09-17T21:29:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.344e-06 2024-09-17T21:29:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556541+0000", 2024-09-17T21:29:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7660000000000004e-06 2024-09-17T21:29:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556566+0000", 2024-09-17T21:29:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4992999999999999e-05 2024-09-17T21:29:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556588+0000", 2024-09-17T21:29:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1746000000000002e-05 2024-09-17T21:29:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557425+0000", 2024-09-17T21:29:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00083782099999999999 2024-09-17T21:29:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557491+0000", 2024-09-17T21:29:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5804999999999998e-05 2024-09-17T21:29:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-17T21:29:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557635+0000", 2024-09-17T21:29:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000143318 2024-09-17T21:29:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.561398+0000", 2024-09-17T21:29:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037635799999999999 2024-09-17T21:29:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.566546+0000", 2024-09-17T21:29:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051477839999999999 2024-09-17T21:29:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.566586+0000", 2024-09-17T21:29:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0531000000000002e-05 2024-09-17T21:29:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.566625+0000", 2024-09-17T21:29:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8688e-05 2024-09-17T21:29:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:2881 3.a e30/25 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:head v 30'10, mlcod=29'7)", 2024-09-17T21:29:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.557303+0000", 2024-09-17T21:29:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.2520891670000001, 2024-09-17T21:29:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0092338070000000001, 2024-09-17T21:29:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557303+0000", 2024-09-17T21:29:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557303+0000", 2024-09-17T21:29:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557303+0000", 2024-09-17T21:29:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.61e-07 2024-09-17T21:29:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557310+0000", 2024-09-17T21:29:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.55e-06 2024-09-17T21:29:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557320+0000", 2024-09-17T21:29:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8479999999999996e-06 2024-09-17T21:29:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557326+0000", 2024-09-17T21:29:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3099999999999997e-06 2024-09-17T21:29:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557541+0000", 2024-09-17T21:29:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000215085 2024-09-17T21:29:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557559+0000", 2024-09-17T21:29:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8131e-05 2024-09-17T21:29:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.566482+0000", 2024-09-17T21:29:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0089228929999999994 2024-09-17T21:29:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.566536+0000", 2024-09-17T21:29:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4228999999999999e-05 2024-09-17T21:29:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:2887 3.11 e30/25 3:8afabca4:::rbd_data.11202c1b6cb4.0000000000000023:head v 30'19, mlcod=29'16)", 2024-09-17T21:29:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.557973+0000", 2024-09-17T21:29:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.2514186110000001, 2024-09-17T21:29:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0085376440000000005, 2024-09-17T21:29:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557973+0000", 2024-09-17T21:29:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557973+0000", 2024-09-17T21:29:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557974+0000", 2024-09-17T21:29:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2799999999999996e-07 2024-09-17T21:29:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557980+0000", 2024-09-17T21:29:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8309999999999997e-06 2024-09-17T21:29:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557988+0000", 2024-09-17T21:29:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8429999999999999e-06 2024-09-17T21:29:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557995+0000", 2024-09-17T21:29:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.737e-06 2024-09-17T21:29:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.558049+0000", 2024-09-17T21:29:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3436999999999997e-05 2024-09-17T21:29:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.558067+0000", 2024-09-17T21:29:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8828999999999999e-05 2024-09-17T21:29:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.566469+0000", 2024-09-17T21:29:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0084015200000000009 2024-09-17T21:29:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.566511+0000", 2024-09-17T21:29:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1819000000000001e-05 2024-09-17T21:29:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:2889 3.3 3:c67c06f8:::rbd_data.11202c1b6cb4.0000000000000025:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.558405+0000", 2024-09-17T21:29:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.2509869499999999, 2024-09-17T21:29:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0081367499999999999, 2024-09-17T21:29:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 2889 2024-09-17T21:29:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.558405+0000", 2024-09-17T21:29:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.558405+0000", 2024-09-17T21:29:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.558406+0000", 2024-09-17T21:29:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4799999999999997e-07 2024-09-17T21:29:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.558409+0000", 2024-09-17T21:29:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7639999999999999e-06 2024-09-17T21:29:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.558419+0000", 2024-09-17T21:29:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6590000000000002e-06 2024-09-17T21:29:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.558424+0000", 2024-09-17T21:29:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8690000000000003e-06 2024-09-17T21:29:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.558455+0000", 2024-09-17T21:29:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1470000000000002e-05 2024-09-17T21:29:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.558496+0000", 2024-09-17T21:29:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0515000000000001e-05 2024-09-17T21:29:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.558582+0000", 2024-09-17T21:29:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5607999999999998e-05 2024-09-17T21:29:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.561342+0000", 2024-09-17T21:29:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027599819999999998 2024-09-17T21:29:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.566450+0000", 2024-09-17T21:29:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051082080000000004 2024-09-17T21:29:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.566503+0000", 2024-09-17T21:29:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2935000000000001e-05 2024-09-17T21:29:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.566542+0000", 2024-09-17T21:29:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8892e-05 2024-09-17T21:29:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:3787 3.3 3:c67c06f8:::rbd_data.11202c1b6cb4.0000000000000025:4 [read 1871872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.902981+0000", 2024-09-17T21:29:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.90641103099999998, 2024-09-17T21:29:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0076664239999999998, 2024-09-17T21:29:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 3787 2024-09-17T21:29:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.902981+0000", 2024-09-17T21:29:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.902981+0000", 2024-09-17T21:29:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.902981+0000", 2024-09-17T21:29:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:29:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.902983+0000", 2024-09-17T21:29:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7179999999999999e-06 2024-09-17T21:29:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.902990+0000", 2024-09-17T21:29:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6730000000000003e-06 2024-09-17T21:29:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.902994+0000", 2024-09-17T21:29:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:29:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.910559+0000", 2024-09-17T21:29:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075648080000000001 2024-09-17T21:29:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.910604+0000", 2024-09-17T21:29:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4997999999999998e-05 2024-09-17T21:29:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.910647+0000", 2024-09-17T21:29:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3037999999999997e-05 2024-09-17T21:29:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4203 3.3 3:c67c06f8:::rbd_data.11202c1b6cb4.0000000000000025:4 [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.927246+0000", 2024-09-17T21:29:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.882146188, 2024-09-17T21:29:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007837643, 2024-09-17T21:29:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4203 2024-09-17T21:29:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.927246+0000", 2024-09-17T21:29:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.927246+0000", 2024-09-17T21:29:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.927246+0000", 2024-09-17T21:29:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:29:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.927248+0000", 2024-09-17T21:29:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:29:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.927254+0000", 2024-09-17T21:29:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1940000000000003e-06 2024-09-17T21:29:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.927259+0000", 2024-09-17T21:29:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8189999999999999e-06 2024-09-17T21:29:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.934888+0000", 2024-09-17T21:29:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0076290860000000002 2024-09-17T21:29:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.934976+0000", 2024-09-17T21:29:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7571000000000006e-05 2024-09-17T21:29:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.935083+0000", 2024-09-17T21:29:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010733799999999999 2024-09-17T21:29:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4204 3.3 3:c67c06f8:::rbd_data.11202c1b6cb4.0000000000000025:4 [read 354304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.927266+0000", 2024-09-17T21:29:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.88212556900000005, 2024-09-17T21:29:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079409200000000006, 2024-09-17T21:29:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4204 2024-09-17T21:29:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.927266+0000", 2024-09-17T21:29:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.927266+0000", 2024-09-17T21:29:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.927267+0000", 2024-09-17T21:29:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:29:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.927269+0000", 2024-09-17T21:29:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.911e-06 2024-09-17T21:29:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.927280+0000", 2024-09-17T21:29:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1076e-05 2024-09-17T21:29:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.927286+0000", 2024-09-17T21:29:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4840000000000001e-06 2024-09-17T21:29:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.935119+0000", 2024-09-17T21:29:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0078329860000000001 2024-09-17T21:29:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.935164+0000", 2024-09-17T21:29:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4540999999999999e-05 2024-09-17T21:29:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.935207+0000", 2024-09-17T21:29:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3535000000000002e-05 2024-09-17T21:29:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4205 3.3 3:c67c06f8:::rbd_data.11202c1b6cb4.0000000000000025:4 [read 366592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.927294+0000", 2024-09-17T21:29:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.88209811100000002, 2024-09-17T21:29:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0081091440000000004, 2024-09-17T21:29:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4205 2024-09-17T21:29:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.927294+0000", 2024-09-17T21:29:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.927294+0000", 2024-09-17T21:29:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.927294+0000", 2024-09-17T21:29:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:29:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.927296+0000", 2024-09-17T21:29:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2120000000000002e-06 2024-09-17T21:29:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.927302+0000", 2024-09-17T21:29:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7320000000000003e-06 2024-09-17T21:29:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.927306+0000", 2024-09-17T21:29:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3270000000000002e-06 2024-09-17T21:29:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.935243+0000", 2024-09-17T21:29:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079367110000000008 2024-09-17T21:29:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.935323+0000", 2024-09-17T21:29:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0322999999999997e-05 2024-09-17T21:29:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.935403+0000", 2024-09-17T21:29:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9420999999999997e-05 2024-09-17T21:29:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4291 3.3 3:c67c06f8:::rbd_data.11202c1b6cb4.0000000000000025:4 [read 219136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.928115+0000", 2024-09-17T21:29:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.88127628300000005, 2024-09-17T21:29:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0078884220000000008, 2024-09-17T21:29:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4291 2024-09-17T21:29:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928115+0000", 2024-09-17T21:29:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928115+0000", 2024-09-17T21:29:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928116+0000", 2024-09-17T21:29:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5400000000000002e-07 2024-09-17T21:29:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928118+0000", 2024-09-17T21:29:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9240000000000001e-06 2024-09-17T21:29:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928123+0000", 2024-09-17T21:29:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5369999999999997e-06 2024-09-17T21:29:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928128+0000", 2024-09-17T21:29:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2189999999999998e-06 2024-09-17T21:29:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.935799+0000", 2024-09-17T21:29:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0076715380000000003 2024-09-17T21:29:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.935899+0000", 2024-09-17T21:29:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010025 2024-09-17T21:29:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.936004+0000", 2024-09-17T21:29:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010459999999999999 2024-09-17T21:29:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4292 3.3 3:c67c06f8:::rbd_data.11202c1b6cb4.0000000000000025:4 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.928134+0000", 2024-09-17T21:29:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.88125742900000004, 2024-09-17T21:29:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0080614120000000004, 2024-09-17T21:29:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4292 2024-09-17T21:29:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928134+0000", 2024-09-17T21:29:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928134+0000", 2024-09-17T21:29:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928135+0000", 2024-09-17T21:29:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7e-07 2024-09-17T21:29:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928137+0000", 2024-09-17T21:29:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9800000000000001e-06 2024-09-17T21:29:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928143+0000", 2024-09-17T21:29:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8989999999999998e-06 2024-09-17T21:29:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928147+0000", 2024-09-17T21:29:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6630000000000001e-06 2024-09-17T21:29:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.936046+0000", 2024-09-17T21:29:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0078984840000000008 2024-09-17T21:29:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.936118+0000", 2024-09-17T21:29:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2090999999999998e-05 2024-09-17T21:29:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.936196+0000", 2024-09-17T21:29:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7924999999999997e-05 2024-09-17T21:29:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4293 3.3 3:c67c06f8:::rbd_data.11202c1b6cb4.0000000000000025:4 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.928153+0000", 2024-09-17T21:29:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.88123870000000004, 2024-09-17T21:29:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0082119940000000002, 2024-09-17T21:29:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4293 2024-09-17T21:29:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928153+0000", 2024-09-17T21:29:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928153+0000", 2024-09-17T21:29:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928153+0000", 2024-09-17T21:29:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5400000000000002e-07 2024-09-17T21:29:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928155+0000", 2024-09-17T21:29:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.748e-06 2024-09-17T21:29:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928161+0000", 2024-09-17T21:29:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.789e-06 2024-09-17T21:29:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928167+0000", 2024-09-17T21:29:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6919999999999999e-06 2024-09-17T21:29:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.936220+0000", 2024-09-17T21:29:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.008052844 2024-09-17T21:29:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.936280+0000", 2024-09-17T21:29:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0322999999999998e-05 2024-09-17T21:29:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.936365+0000", 2024-09-17T21:29:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5243999999999999e-05 2024-09-17T21:29:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4294 3.3 3:c67c06f8:::rbd_data.11202c1b6cb4.0000000000000025:4 [read 260096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.928173+0000", 2024-09-17T21:29:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.88121898899999995, 2024-09-17T21:29:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0085827869999999997, 2024-09-17T21:29:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4294 2024-09-17T21:29:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928173+0000", 2024-09-17T21:29:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928173+0000", 2024-09-17T21:29:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928173+0000", 2024-09-17T21:29:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T21:29:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928175+0000", 2024-09-17T21:29:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.875e-06 2024-09-17T21:29:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928181+0000", 2024-09-17T21:29:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2190000000000001e-06 2024-09-17T21:29:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928186+0000", 2024-09-17T21:29:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5619999999999997e-06 2024-09-17T21:29:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.936390+0000", 2024-09-17T21:29:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0082045390000000003 2024-09-17T21:29:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.936652+0000", 2024-09-17T21:29:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026142599999999999 2024-09-17T21:29:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.936756+0000", 2024-09-17T21:29:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010380800000000001 2024-09-17T21:29:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4295 3.3 3:c67c06f8:::rbd_data.11202c1b6cb4.0000000000000025:4 [read 274432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.928197+0000", 2024-09-17T21:29:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.88119503499999996, 2024-09-17T21:29:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0087866790000000004, 2024-09-17T21:29:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4295 2024-09-17T21:29:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928197+0000", 2024-09-17T21:29:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928197+0000", 2024-09-17T21:29:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928197+0000", 2024-09-17T21:29:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.27e-07 2024-09-17T21:29:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928201+0000", 2024-09-17T21:29:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.72e-06 2024-09-17T21:29:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928207+0000", 2024-09-17T21:29:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3999999999999997e-06 2024-09-17T21:29:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928212+0000", 2024-09-17T21:29:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7489999999999998e-06 2024-09-17T21:29:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.936791+0000", 2024-09-17T21:29:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0085784659999999999 2024-09-17T21:29:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.936877+0000", 2024-09-17T21:29:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6364000000000002e-05 2024-09-17T21:29:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.936983+0000", 2024-09-17T21:29:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010655299999999999 2024-09-17T21:29:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4296 3.3 3:c67c06f8:::rbd_data.11202c1b6cb4.0000000000000025:4 [read 288768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.928218+0000", 2024-09-17T21:29:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.88117329499999997, 2024-09-17T21:29:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0089815229999999999, 2024-09-17T21:29:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4296 2024-09-17T21:29:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928218+0000", 2024-09-17T21:29:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928218+0000", 2024-09-17T21:29:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928219+0000", 2024-09-17T21:29:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:29:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928221+0000", 2024-09-17T21:29:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.674e-06 2024-09-17T21:29:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928228+0000", 2024-09-17T21:29:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2479999999999999e-06 2024-09-17T21:29:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928232+0000", 2024-09-17T21:29:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2220000000000004e-06 2024-09-17T21:29:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.937028+0000", 2024-09-17T21:29:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0087956630000000004 2024-09-17T21:29:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.937104+0000", 2024-09-17T21:29:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6654000000000002e-05 2024-09-17T21:29:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.937200+0000", 2024-09-17T21:29:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5687999999999996e-05 2024-09-17T21:29:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4297 3.3 3:c67c06f8:::rbd_data.11202c1b6cb4.0000000000000025:4 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.928239+0000", 2024-09-17T21:29:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.88115319000000003, 2024-09-17T21:29:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0091748490000000005, 2024-09-17T21:29:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4297 2024-09-17T21:29:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928239+0000", 2024-09-17T21:29:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928239+0000", 2024-09-17T21:29:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928239+0000", 2024-09-17T21:29:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:29:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928241+0000", 2024-09-17T21:29:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T21:29:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928247+0000", 2024-09-17T21:29:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3040000000000001e-06 2024-09-17T21:29:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928252+0000", 2024-09-17T21:29:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8099999999999997e-06 2024-09-17T21:29:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.937229+0000", 2024-09-17T21:29:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0089767189999999993 2024-09-17T21:29:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.937301+0000", 2024-09-17T21:29:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2566999999999998e-05 2024-09-17T21:29:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.937413+0000", 2024-09-17T21:29:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000112184 2024-09-17T21:29:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4298 3.3 3:c67c06f8:::rbd_data.11202c1b6cb4.0000000000000025:4 [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.928274+0000", 2024-09-17T21:29:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.88111733699999994, 2024-09-17T21:29:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093153550000000009, 2024-09-17T21:29:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4298 2024-09-17T21:29:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928274+0000", 2024-09-17T21:29:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928274+0000", 2024-09-17T21:29:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928275+0000", 2024-09-17T21:29:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2800000000000002e-07 2024-09-17T21:29:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928277+0000", 2024-09-17T21:29:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1959999999999998e-06 2024-09-17T21:29:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928284+0000", 2024-09-17T21:29:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5610000000000004e-06 2024-09-17T21:29:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928289+0000", 2024-09-17T21:29:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7039999999999999e-06 2024-09-17T21:29:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.937460+0000", 2024-09-17T21:29:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0091711420000000002 2024-09-17T21:29:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.937506+0000", 2024-09-17T21:29:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5569999999999999e-05 2024-09-17T21:29:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.937590+0000", 2024-09-17T21:29:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3753999999999998e-05 2024-09-17T21:29:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4299 3.3 3:c67c06f8:::rbd_data.11202c1b6cb4.0000000000000025:4 [read 329728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.928297+0000", 2024-09-17T21:29:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.88109485899999995, 2024-09-17T21:29:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0096249049999999996, 2024-09-17T21:29:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4299 2024-09-17T21:29:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928297+0000", 2024-09-17T21:29:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928297+0000", 2024-09-17T21:29:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928297+0000", 2024-09-17T21:29:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:29:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928299+0000", 2024-09-17T21:29:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.883e-06 2024-09-17T21:29:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928306+0000", 2024-09-17T21:29:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4230000000000002e-06 2024-09-17T21:29:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928312+0000", 2024-09-17T21:29:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8590000000000001e-06 2024-09-17T21:29:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.937620+0000", 2024-09-17T21:29:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093080669999999997 2024-09-17T21:29:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.937698+0000", 2024-09-17T21:29:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8872e-05 2024-09-17T21:29:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.937922+0000", 2024-09-17T21:29:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00022332499999999999 2024-09-17T21:29:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4300 3.3 3:c67c06f8:::rbd_data.11202c1b6cb4.0000000000000025:4 [read 342016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.928321+0000", 2024-09-17T21:29:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.88107110700000002, 2024-09-17T21:29:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0098389870000000004, 2024-09-17T21:29:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4300 2024-09-17T21:29:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928321+0000", 2024-09-17T21:29:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928321+0000", 2024-09-17T21:29:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928321+0000", 2024-09-17T21:29:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T21:29:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928323+0000", 2024-09-17T21:29:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8470000000000001e-06 2024-09-17T21:29:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928330+0000", 2024-09-17T21:29:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2049999999999996e-06 2024-09-17T21:29:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928334+0000", 2024-09-17T21:29:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-06 2024-09-17T21:29:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.937951+0000", 2024-09-17T21:29:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0096171390000000002 2024-09-17T21:29:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.938004+0000", 2024-09-17T21:29:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2660000000000001e-05 2024-09-17T21:29:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.938160+0000", 2024-09-17T21:29:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015553799999999999 2024-09-17T21:29:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4301 3.3 3:c67c06f8:::rbd_data.11202c1b6cb4.0000000000000025:4 [read 356352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.928340+0000", 2024-09-17T21:29:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.88105130799999998, 2024-09-17T21:29:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010019916, 2024-09-17T21:29:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4301 2024-09-17T21:29:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928340+0000", 2024-09-17T21:29:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928340+0000", 2024-09-17T21:29:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928341+0000", 2024-09-17T21:29:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7599999999999998e-07 2024-09-17T21:29:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928343+0000", 2024-09-17T21:29:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9630000000000001e-06 2024-09-17T21:29:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928350+0000", 2024-09-17T21:29:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8059999999999997e-06 2024-09-17T21:29:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928354+0000", 2024-09-17T21:29:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8629999999999999e-06 2024-09-17T21:29:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.938201+0000", 2024-09-17T21:29:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0098462359999999995 2024-09-17T21:29:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.938275+0000", 2024-09-17T21:29:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4660999999999995e-05 2024-09-17T21:29:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.938360+0000", 2024-09-17T21:29:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5011000000000003e-05 2024-09-17T21:29:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4302 3.3 3:c67c06f8:::rbd_data.11202c1b6cb4.0000000000000025:4 [read 370688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.928361+0000", 2024-09-17T21:29:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.88103098999999996, 2024-09-17T21:29:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010229703, 2024-09-17T21:29:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4302 2024-09-17T21:29:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928361+0000", 2024-09-17T21:29:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928361+0000", 2024-09-17T21:29:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928361+0000", 2024-09-17T21:29:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.72e-07 2024-09-17T21:29:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928363+0000", 2024-09-17T21:29:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9750000000000001e-06 2024-09-17T21:29:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928369+0000", 2024-09-17T21:29:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:29:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.928374+0000", 2024-09-17T21:29:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6929999999999998e-06 2024-09-17T21:29:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.938397+0000", 2024-09-17T21:29:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010023565 2024-09-17T21:29:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.938506+0000", 2024-09-17T21:29:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000108513 2024-09-17T21:29:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.938590+0000", 2024-09-17T21:29:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4843000000000001e-05 2024-09-17T21:29:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:48.909 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:48.910 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:29:48.911 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:48.908+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:49.011 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:49.008+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:49.110 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:49.111 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:29:49.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.111 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10808 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.111 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.082672+0000", 2024-09-17T21:29:49.111 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.016648947000000001, 2024-09-17T21:29:49.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016667112000000001, 2024-09-17T21:29:49.111 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.111 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.111 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.111 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.111 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.112 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.112 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10808 2024-09-17T21:29:49.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.112 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082672+0000", 2024-09-17T21:29:49.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082672+0000", 2024-09-17T21:29:49.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082672+0000", 2024-09-17T21:29:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9699999999999996e-07 2024-09-17T21:29:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082675+0000", 2024-09-17T21:29:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.633e-06 2024-09-17T21:29:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082682+0000", 2024-09-17T21:29:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1160000000000001e-06 2024-09-17T21:29:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082687+0000", 2024-09-17T21:29:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2220000000000001e-06 2024-09-17T21:29:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10882 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.083612+0000", 2024-09-17T21:29:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.015708636000000002, 2024-09-17T21:29:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015759295, 2024-09-17T21:29:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10882 2024-09-17T21:29:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083612+0000", 2024-09-17T21:29:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083612+0000", 2024-09-17T21:29:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083613+0000", 2024-09-17T21:29:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:29:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083614+0000", 2024-09-17T21:29:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.984e-06 2024-09-17T21:29:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083621+0000", 2024-09-17T21:29:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3439999999999997e-06 2024-09-17T21:29:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083626+0000", 2024-09-17T21:29:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7620000000000003e-06 2024-09-17T21:29:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11102 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.095170+0000", 2024-09-17T21:29:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0041509210000000001, 2024-09-17T21:29:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042261299999999998, 2024-09-17T21:29:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11102 2024-09-17T21:29:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095170+0000", 2024-09-17T21:29:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095170+0000", 2024-09-17T21:29:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095170+0000", 2024-09-17T21:29:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6300000000000001e-07 2024-09-17T21:29:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095172+0000", 2024-09-17T21:29:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9089999999999998e-06 2024-09-17T21:29:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095178+0000", 2024-09-17T21:29:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2419999999999997e-06 2024-09-17T21:29:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095185+0000", 2024-09-17T21:29:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9970000000000001e-06 2024-09-17T21:29:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11176 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.096657+0000", 2024-09-17T21:29:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.002663295, 2024-09-17T21:29:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027611340000000002, 2024-09-17T21:29:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11176 2024-09-17T21:29:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096657+0000", 2024-09-17T21:29:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096657+0000", 2024-09-17T21:29:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096658+0000", 2024-09-17T21:29:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.99e-07 2024-09-17T21:29:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096660+0000", 2024-09-17T21:29:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0870000000000001e-06 2024-09-17T21:29:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096668+0000", 2024-09-17T21:29:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5240000000000003e-06 2024-09-17T21:29:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096678+0000", 2024-09-17T21:29:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0587e-05 2024-09-17T21:29:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10809 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.082695+0000", 2024-09-17T21:29:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.016625844000000001, 2024-09-17T21:29:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016747015000000001, 2024-09-17T21:29:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10809 2024-09-17T21:29:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082695+0000", 2024-09-17T21:29:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082695+0000", 2024-09-17T21:29:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082695+0000", 2024-09-17T21:29:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:29:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082697+0000", 2024-09-17T21:29:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0470000000000001e-06 2024-09-17T21:29:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082704+0000", 2024-09-17T21:29:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.117e-06 2024-09-17T21:29:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082708+0000", 2024-09-17T21:29:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.853e-06 2024-09-17T21:29:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10883 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.083632+0000", 2024-09-17T21:29:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.015689116999999999, 2024-09-17T21:29:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015834237000000001, 2024-09-17T21:29:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10883 2024-09-17T21:29:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083632+0000", 2024-09-17T21:29:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083632+0000", 2024-09-17T21:29:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083632+0000", 2024-09-17T21:29:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T21:29:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083634+0000", 2024-09-17T21:29:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:29:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083640+0000", 2024-09-17T21:29:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1800000000000001e-06 2024-09-17T21:29:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083644+0000", 2024-09-17T21:29:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3209999999999998e-06 2024-09-17T21:29:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11103 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.095193+0000", 2024-09-17T21:29:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00412751, 2024-09-17T21:29:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042952420000000003, 2024-09-17T21:29:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11103 2024-09-17T21:29:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095193+0000", 2024-09-17T21:29:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095193+0000", 2024-09-17T21:29:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095194+0000", 2024-09-17T21:29:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:29:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095204+0000", 2024-09-17T21:29:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0036e-05 2024-09-17T21:29:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095211+0000", 2024-09-17T21:29:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.272e-06 2024-09-17T21:29:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095216+0000", 2024-09-17T21:29:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0170000000000004e-06 2024-09-17T21:29:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11177 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.096698+0000", 2024-09-17T21:29:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0026227780000000001, 2024-09-17T21:29:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028130669999999998, 2024-09-17T21:29:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11177 2024-09-17T21:29:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096698+0000", 2024-09-17T21:29:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096698+0000", 2024-09-17T21:29:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096699+0000", 2024-09-17T21:29:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-17T21:29:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096706+0000", 2024-09-17T21:29:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7049999999999999e-06 2024-09-17T21:29:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096714+0000", 2024-09-17T21:29:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6650000000000003e-06 2024-09-17T21:29:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096725+0000", 2024-09-17T21:29:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1309e-05 2024-09-17T21:29:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10810 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.082714+0000", 2024-09-17T21:29:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.016606293000000001, 2024-09-17T21:29:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016819119, 2024-09-17T21:29:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10810 2024-09-17T21:29:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082714+0000", 2024-09-17T21:29:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082714+0000", 2024-09-17T21:29:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082715+0000", 2024-09-17T21:29:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:29:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082717+0000", 2024-09-17T21:29:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9080000000000002e-06 2024-09-17T21:29:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082722+0000", 2024-09-17T21:29:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6969999999999998e-06 2024-09-17T21:29:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082727+0000", 2024-09-17T21:29:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5539999999999999e-06 2024-09-17T21:29:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10884 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.083650+0000", 2024-09-17T21:29:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.015670706999999999, 2024-09-17T21:29:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015907725000000001, 2024-09-17T21:29:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10884 2024-09-17T21:29:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083650+0000", 2024-09-17T21:29:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083650+0000", 2024-09-17T21:29:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083650+0000", 2024-09-17T21:29:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:29:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083652+0000", 2024-09-17T21:29:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.66e-06 2024-09-17T21:29:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083658+0000", 2024-09-17T21:29:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4949999999999999e-06 2024-09-17T21:29:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083670+0000", 2024-09-17T21:29:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2024e-05 2024-09-17T21:29:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11104 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.095224+0000", 2024-09-17T21:29:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0040972220000000002, 2024-09-17T21:29:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043570579999999996, 2024-09-17T21:29:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11104 2024-09-17T21:29:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095224+0000", 2024-09-17T21:29:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095224+0000", 2024-09-17T21:29:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095224+0000", 2024-09-17T21:29:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.58e-07 2024-09-17T21:29:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095226+0000", 2024-09-17T21:29:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2749999999999998e-06 2024-09-17T21:29:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095233+0000", 2024-09-17T21:29:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.968e-06 2024-09-17T21:29:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095238+0000", 2024-09-17T21:29:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5569999999999997e-06 2024-09-17T21:29:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11178 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.096735+0000", 2024-09-17T21:29:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00258544, 2024-09-17T21:29:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028814800000000001, 2024-09-17T21:29:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11178 2024-09-17T21:29:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096735+0000", 2024-09-17T21:29:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096735+0000", 2024-09-17T21:29:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096736+0000", 2024-09-17T21:29:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6400000000000002e-07 2024-09-17T21:29:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096738+0000", 2024-09-17T21:29:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3659999999999999e-06 2024-09-17T21:29:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096745+0000", 2024-09-17T21:29:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2609999999999996e-06 2024-09-17T21:29:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096751+0000", 2024-09-17T21:29:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3380000000000004e-06 2024-09-17T21:29:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10811 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.082734+0000", 2024-09-17T21:29:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.016586269000000001, 2024-09-17T21:29:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016949474999999999, 2024-09-17T21:29:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10811 2024-09-17T21:29:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082734+0000", 2024-09-17T21:29:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082734+0000", 2024-09-17T21:29:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082735+0000", 2024-09-17T21:29:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.89e-07 2024-09-17T21:29:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082741+0000", 2024-09-17T21:29:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5899999999999998e-06 2024-09-17T21:29:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082747+0000", 2024-09-17T21:29:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.976e-06 2024-09-17T21:29:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082751+0000", 2024-09-17T21:29:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4719999999999997e-06 2024-09-17T21:29:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10885 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.083676+0000", 2024-09-17T21:29:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.015644247999999999, 2024-09-17T21:29:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016052024000000002, 2024-09-17T21:29:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10885 2024-09-17T21:29:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083676+0000", 2024-09-17T21:29:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083676+0000", 2024-09-17T21:29:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083677+0000", 2024-09-17T21:29:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.46e-07 2024-09-17T21:29:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083683+0000", 2024-09-17T21:29:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3589999999999996e-06 2024-09-17T21:29:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083690+0000", 2024-09-17T21:29:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8600000000000004e-06 2024-09-17T21:29:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083703+0000", 2024-09-17T21:29:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2490999999999999e-05 2024-09-17T21:29:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11105 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.095244+0000", 2024-09-17T21:29:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.004076479, 2024-09-17T21:29:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045189560000000002, 2024-09-17T21:29:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11105 2024-09-17T21:29:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095244+0000", 2024-09-17T21:29:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095244+0000", 2024-09-17T21:29:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095245+0000", 2024-09-17T21:29:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.46e-07 2024-09-17T21:29:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095246+0000", 2024-09-17T21:29:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7829999999999999e-06 2024-09-17T21:29:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095252+0000", 2024-09-17T21:29:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.614e-06 2024-09-17T21:29:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095257+0000", 2024-09-17T21:29:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5750000000000002e-06 2024-09-17T21:29:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11179 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.096760+0000", 2024-09-17T21:29:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0025610030000000001, 2024-09-17T21:29:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030300510000000002, 2024-09-17T21:29:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11179 2024-09-17T21:29:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096760+0000", 2024-09-17T21:29:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096760+0000", 2024-09-17T21:29:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096760+0000", 2024-09-17T21:29:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4299999999999998e-07 2024-09-17T21:29:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096762+0000", 2024-09-17T21:29:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1739999999999999e-06 2024-09-17T21:29:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096769+0000", 2024-09-17T21:29:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5860000000000001e-06 2024-09-17T21:29:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096774+0000", 2024-09-17T21:29:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4079999999999997e-06 2024-09-17T21:29:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10812 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.082758+0000", 2024-09-17T21:29:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.016563043999999999, 2024-09-17T21:29:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.01705475, 2024-09-17T21:29:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10812 2024-09-17T21:29:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082758+0000", 2024-09-17T21:29:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082758+0000", 2024-09-17T21:29:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082758+0000", 2024-09-17T21:29:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4799999999999999e-07 2024-09-17T21:29:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082760+0000", 2024-09-17T21:29:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9659999999999999e-06 2024-09-17T21:29:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082766+0000", 2024-09-17T21:29:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8470000000000001e-06 2024-09-17T21:29:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082770+0000", 2024-09-17T21:29:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5290000000000002e-06 2024-09-17T21:29:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10886 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.083710+0000", 2024-09-17T21:29:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.015610427, 2024-09-17T21:29:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016147410000000001, 2024-09-17T21:29:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10886 2024-09-17T21:29:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083710+0000", 2024-09-17T21:29:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083710+0000", 2024-09-17T21:29:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083711+0000", 2024-09-17T21:29:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9500000000000003e-07 2024-09-17T21:29:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083713+0000", 2024-09-17T21:29:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0849999999999999e-06 2024-09-17T21:29:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083719+0000", 2024-09-17T21:29:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9970000000000004e-06 2024-09-17T21:29:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083723+0000", 2024-09-17T21:29:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.369e-06 2024-09-17T21:29:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10974 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.086906+0000", 2024-09-17T21:29:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012414909, 2024-09-17T21:29:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012975458, 2024-09-17T21:29:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10974 2024-09-17T21:29:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086906+0000", 2024-09-17T21:29:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086906+0000", 2024-09-17T21:29:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086906+0000", 2024-09-17T21:29:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.89e-07 2024-09-17T21:29:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086908+0000", 2024-09-17T21:29:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:29:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086913+0000", 2024-09-17T21:29:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3430000000000003e-06 2024-09-17T21:29:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086918+0000", 2024-09-17T21:29:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:29:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11106 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.095262+0000", 2024-09-17T21:29:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0040583750000000004, 2024-09-17T21:29:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0046413979999999997, 2024-09-17T21:29:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11106 2024-09-17T21:29:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095262+0000", 2024-09-17T21:29:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095262+0000", 2024-09-17T21:29:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095263+0000", 2024-09-17T21:29:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:29:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095265+0000", 2024-09-17T21:29:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.835e-06 2024-09-17T21:29:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095270+0000", 2024-09-17T21:29:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3059999999999997e-06 2024-09-17T21:29:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095274+0000", 2024-09-17T21:29:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1099999999999996e-06 2024-09-17T21:29:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11180 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.096781+0000", 2024-09-17T21:29:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.002539557, 2024-09-17T21:29:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031459610000000001, 2024-09-17T21:29:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11180 2024-09-17T21:29:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096781+0000", 2024-09-17T21:29:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096781+0000", 2024-09-17T21:29:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096782+0000", 2024-09-17T21:29:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:29:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096783+0000", 2024-09-17T21:29:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9089999999999998e-06 2024-09-17T21:29:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096790+0000", 2024-09-17T21:29:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5490000000000003e-06 2024-09-17T21:29:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096795+0000", 2024-09-17T21:29:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8890000000000001e-06 2024-09-17T21:29:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10813 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.082786+0000", 2024-09-17T21:29:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.016534864999999999, 2024-09-17T21:29:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017172545000000001, 2024-09-17T21:29:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10813 2024-09-17T21:29:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082786+0000", 2024-09-17T21:29:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082786+0000", 2024-09-17T21:29:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082786+0000", 2024-09-17T21:29:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.75e-07 2024-09-17T21:29:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082794+0000", 2024-09-17T21:29:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8329999999999994e-06 2024-09-17T21:29:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082801+0000", 2024-09-17T21:29:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.596e-06 2024-09-17T21:29:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082806+0000", 2024-09-17T21:29:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7380000000000002e-06 2024-09-17T21:29:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10887 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.083730+0000", 2024-09-17T21:29:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.015591183999999999, 2024-09-17T21:29:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016275563999999999, 2024-09-17T21:29:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10887 2024-09-17T21:29:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083730+0000", 2024-09-17T21:29:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083730+0000", 2024-09-17T21:29:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083730+0000", 2024-09-17T21:29:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:29:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083732+0000", 2024-09-17T21:29:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8679999999999999e-06 2024-09-17T21:29:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083737+0000", 2024-09-17T21:29:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4859999999999997e-06 2024-09-17T21:29:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083743+0000", 2024-09-17T21:29:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7320000000000003e-06 2024-09-17T21:29:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10975 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.086932+0000", 2024-09-17T21:29:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012388367000000001, 2024-09-17T21:29:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013118474999999999, 2024-09-17T21:29:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10975 2024-09-17T21:29:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086932+0000", 2024-09-17T21:29:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086932+0000", 2024-09-17T21:29:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086933+0000", 2024-09-17T21:29:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0399999999999996e-07 2024-09-17T21:29:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086935+0000", 2024-09-17T21:29:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:29:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086941+0000", 2024-09-17T21:29:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3350000000000003e-06 2024-09-17T21:29:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086947+0000", 2024-09-17T21:29:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0850000000000002e-06 2024-09-17T21:29:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11107 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.095298+0000", 2024-09-17T21:29:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0040224600000000003, 2024-09-17T21:29:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004803135, 2024-09-17T21:29:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11107 2024-09-17T21:29:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095298+0000", 2024-09-17T21:29:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095298+0000", 2024-09-17T21:29:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095299+0000", 2024-09-17T21:29:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:29:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095301+0000", 2024-09-17T21:29:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-06 2024-09-17T21:29:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095307+0000", 2024-09-17T21:29:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9789999999999998e-06 2024-09-17T21:29:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095325+0000", 2024-09-17T21:29:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8502999999999999e-05 2024-09-17T21:29:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11181 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.096802+0000", 2024-09-17T21:29:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0025189499999999998, 2024-09-17T21:29:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033452460000000001, 2024-09-17T21:29:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11181 2024-09-17T21:29:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096802+0000", 2024-09-17T21:29:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096802+0000", 2024-09-17T21:29:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096802+0000", 2024-09-17T21:29:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.32e-07 2024-09-17T21:29:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096804+0000", 2024-09-17T21:29:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.984e-06 2024-09-17T21:29:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096811+0000", 2024-09-17T21:29:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6220000000000003e-06 2024-09-17T21:29:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096816+0000", 2024-09-17T21:29:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1749999999999996e-06 2024-09-17T21:29:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10814 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.082813+0000", 2024-09-17T21:29:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.016507810000000001, 2024-09-17T21:29:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017382267999999999, 2024-09-17T21:29:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10814 2024-09-17T21:29:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082813+0000", 2024-09-17T21:29:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082813+0000", 2024-09-17T21:29:49.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082813+0000", 2024-09-17T21:29:49.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7700000000000005e-07 2024-09-17T21:29:49.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082815+0000", 2024-09-17T21:29:49.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.052e-06 2024-09-17T21:29:49.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082821+0000", 2024-09-17T21:29:49.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8100000000000003e-06 2024-09-17T21:29:49.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082826+0000", 2024-09-17T21:29:49.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5970000000000002e-06 2024-09-17T21:29:49.204 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.204 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.204 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.205 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10888 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.205 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.083750+0000", 2024-09-17T21:29:49.205 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.015571061000000001, 2024-09-17T21:29:49.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016491525, 2024-09-17T21:29:49.205 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.205 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.205 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.205 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.205 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.205 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.205 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10888 2024-09-17T21:29:49.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.206 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083750+0000", 2024-09-17T21:29:49.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083750+0000", 2024-09-17T21:29:49.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083750+0000", 2024-09-17T21:29:49.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2399999999999999e-07 2024-09-17T21:29:49.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083757+0000", 2024-09-17T21:29:49.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1060000000000002e-06 2024-09-17T21:29:49.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083764+0000", 2024-09-17T21:29:49.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5130000000000002e-06 2024-09-17T21:29:49.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083768+0000", 2024-09-17T21:29:49.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4939999999999997e-06 2024-09-17T21:29:49.208 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.208 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.208 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.208 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10976 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.208 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.086955+0000", 2024-09-17T21:29:49.208 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012365950000000001, 2024-09-17T21:29:49.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013335928, 2024-09-17T21:29:49.209 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.209 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.209 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.209 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.209 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.209 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.209 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10976 2024-09-17T21:29:49.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.209 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086955+0000", 2024-09-17T21:29:49.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086955+0000", 2024-09-17T21:29:49.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086955+0000", 2024-09-17T21:29:49.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7100000000000002e-07 2024-09-17T21:29:49.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086957+0000", 2024-09-17T21:29:49.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0269999999999998e-06 2024-09-17T21:29:49.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086963+0000", 2024-09-17T21:29:49.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6540000000000004e-06 2024-09-17T21:29:49.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086975+0000", 2024-09-17T21:29:49.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2451e-05 2024-09-17T21:29:49.211 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.211 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.211 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.212 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11108 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.212 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.095336+0000", 2024-09-17T21:29:49.212 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00398468, 2024-09-17T21:29:49.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005002027, 2024-09-17T21:29:49.212 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.212 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.212 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.212 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.212 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.212 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.212 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11108 2024-09-17T21:29:49.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.212 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095336+0000", 2024-09-17T21:29:49.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095336+0000", 2024-09-17T21:29:49.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095337+0000", 2024-09-17T21:29:49.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-17T21:29:49.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095339+0000", 2024-09-17T21:29:49.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6879999999999999e-06 2024-09-17T21:29:49.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095348+0000", 2024-09-17T21:29:49.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0929999999999992e-06 2024-09-17T21:29:49.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095353+0000", 2024-09-17T21:29:49.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0150000000000003e-06 2024-09-17T21:29:49.215 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.215 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.215 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.215 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11195 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.215 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.099666+0000", 2024-09-17T21:29:49.215 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4294967295.9996548, 2024-09-17T21:29:49.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00072661299999999998, 2024-09-17T21:29:49.215 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.215 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.216 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.216 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.216 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.216 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.216 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11195 2024-09-17T21:29:49.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.216 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099666+0000", 2024-09-17T21:29:49.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099666+0000", 2024-09-17T21:29:49.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099667+0000", 2024-09-17T21:29:49.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4300000000000001e-07 2024-09-17T21:29:49.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099671+0000", 2024-09-17T21:29:49.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3649999999999997e-06 2024-09-17T21:29:49.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099689+0000", 2024-09-17T21:29:49.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7493e-05 2024-09-17T21:29:49.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099695+0000", 2024-09-17T21:29:49.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7390000000000002e-06 2024-09-17T21:29:49.218 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.218 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.218 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.219 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10815 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.219 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.082838+0000", 2024-09-17T21:29:49.219 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.016482836000000001, 2024-09-17T21:29:49.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017600707, 2024-09-17T21:29:49.219 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.219 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.219 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.219 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.219 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.219 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.219 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10815 2024-09-17T21:29:49.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.219 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082838+0000", 2024-09-17T21:29:49.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082838+0000", 2024-09-17T21:29:49.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082838+0000", 2024-09-17T21:29:49.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:29:49.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082840+0000", 2024-09-17T21:29:49.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1579999999999999e-06 2024-09-17T21:29:49.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082847+0000", 2024-09-17T21:29:49.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0299999999999999e-06 2024-09-17T21:29:49.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082857+0000", 2024-09-17T21:29:49.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0322e-05 2024-09-17T21:29:49.222 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.222 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.222 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.222 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10889 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.222 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.083775+0000", 2024-09-17T21:29:49.222 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.015545705, 2024-09-17T21:29:49.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016713763, 2024-09-17T21:29:49.222 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.222 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.222 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.222 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.222 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.223 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.223 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10889 2024-09-17T21:29:49.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.223 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083775+0000", 2024-09-17T21:29:49.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083775+0000", 2024-09-17T21:29:49.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.224 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083776+0000", 2024-09-17T21:29:49.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3099999999999998e-07 2024-09-17T21:29:49.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.224 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083777+0000", 2024-09-17T21:29:49.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8989999999999999e-06 2024-09-17T21:29:49.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.224 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083783+0000", 2024-09-17T21:29:49.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6250000000000004e-06 2024-09-17T21:29:49.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083788+0000", 2024-09-17T21:29:49.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:29:49.225 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.225 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.225 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.225 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10977 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.225 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.086982+0000", 2024-09-17T21:29:49.225 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012338379999999999, 2024-09-17T21:29:49.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013552211999999999, 2024-09-17T21:29:49.226 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.226 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.226 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.226 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.226 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.226 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.226 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10977 2024-09-17T21:29:49.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.226 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086982+0000", 2024-09-17T21:29:49.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086982+0000", 2024-09-17T21:29:49.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086983+0000", 2024-09-17T21:29:49.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T21:29:49.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086989+0000", 2024-09-17T21:29:49.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6449999999999999e-06 2024-09-17T21:29:49.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086996+0000", 2024-09-17T21:29:49.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4570000000000002e-06 2024-09-17T21:29:49.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087005+0000", 2024-09-17T21:29:49.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8580000000000006e-06 2024-09-17T21:29:49.228 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.228 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.228 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.229 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11109 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.229 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.095361+0000", 2024-09-17T21:29:49.229 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0039597460000000001, 2024-09-17T21:29:49.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052613160000000003, 2024-09-17T21:29:49.229 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.229 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.229 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.229 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.229 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.229 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.229 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11109 2024-09-17T21:29:49.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.230 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095361+0000", 2024-09-17T21:29:49.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095361+0000", 2024-09-17T21:29:49.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095361+0000", 2024-09-17T21:29:49.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7399999999999998e-07 2024-09-17T21:29:49.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095364+0000", 2024-09-17T21:29:49.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4080000000000001e-06 2024-09-17T21:29:49.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095370+0000", 2024-09-17T21:29:49.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:29:49.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095374+0000", 2024-09-17T21:29:49.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:29:49.232 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.232 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.232 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.232 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11196 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.232 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.099715+0000", 2024-09-17T21:29:49.232 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4294967295.9996061, 2024-09-17T21:29:49.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00095758900000000001, 2024-09-17T21:29:49.232 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.232 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.233 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.233 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.233 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.233 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.233 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11196 2024-09-17T21:29:49.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.233 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099715+0000", 2024-09-17T21:29:49.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099715+0000", 2024-09-17T21:29:49.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099715+0000", 2024-09-17T21:29:49.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:29:49.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099718+0000", 2024-09-17T21:29:49.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7760000000000002e-06 2024-09-17T21:29:49.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099726+0000", 2024-09-17T21:29:49.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T21:29:49.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099732+0000", 2024-09-17T21:29:49.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.631e-06 2024-09-17T21:29:49.235 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.235 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.235 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.236 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10816 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.236 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.082864+0000", 2024-09-17T21:29:49.236 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.016457184999999999, 2024-09-17T21:29:49.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017858005999999999, 2024-09-17T21:29:49.236 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.236 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.236 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.236 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.236 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.236 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.236 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10816 2024-09-17T21:29:49.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.236 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082864+0000", 2024-09-17T21:29:49.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082864+0000", 2024-09-17T21:29:49.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082864+0000", 2024-09-17T21:29:49.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:29:49.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082866+0000", 2024-09-17T21:29:49.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0600000000000002e-06 2024-09-17T21:29:49.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082872+0000", 2024-09-17T21:29:49.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8819999999999998e-06 2024-09-17T21:29:49.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082877+0000", 2024-09-17T21:29:49.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.707e-06 2024-09-17T21:29:49.239 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.239 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.239 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.239 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10904 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.239 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.086263+0000", 2024-09-17T21:29:49.239 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.013058014, 2024-09-17T21:29:49.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014509032999999999, 2024-09-17T21:29:49.239 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.239 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.240 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.240 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.240 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.240 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.240 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10904 2024-09-17T21:29:49.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.240 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086263+0000", 2024-09-17T21:29:49.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086263+0000", 2024-09-17T21:29:49.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086263+0000", 2024-09-17T21:29:49.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1699999999999997e-07 2024-09-17T21:29:49.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086267+0000", 2024-09-17T21:29:49.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7969999999999998e-06 2024-09-17T21:29:49.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086275+0000", 2024-09-17T21:29:49.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.695e-06 2024-09-17T21:29:49.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086279+0000", 2024-09-17T21:29:49.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3409999999999996e-06 2024-09-17T21:29:49.242 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.242 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.242 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.242 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10978 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.242 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.087012+0000", 2024-09-17T21:29:49.242 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012308262, 2024-09-17T21:29:49.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013804878, 2024-09-17T21:29:49.243 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.243 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.243 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.243 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.243 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.243 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.243 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10978 2024-09-17T21:29:49.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.243 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087012+0000", 2024-09-17T21:29:49.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087012+0000", 2024-09-17T21:29:49.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087013+0000", 2024-09-17T21:29:49.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.32e-07 2024-09-17T21:29:49.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087015+0000", 2024-09-17T21:29:49.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0490000000000002e-06 2024-09-17T21:29:49.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087021+0000", 2024-09-17T21:29:49.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9630000000000003e-06 2024-09-17T21:29:49.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087025+0000", 2024-09-17T21:29:49.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5449999999999997e-06 2024-09-17T21:29:49.245 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.245 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.245 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11110 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.095380+0000", 2024-09-17T21:29:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0039403240000000003, 2024-09-17T21:29:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056515869999999996, 2024-09-17T21:29:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11110 2024-09-17T21:29:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095380+0000", 2024-09-17T21:29:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095380+0000", 2024-09-17T21:29:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095381+0000", 2024-09-17T21:29:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6199999999999999e-07 2024-09-17T21:29:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095383+0000", 2024-09-17T21:29:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9149999999999999e-06 2024-09-17T21:29:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095389+0000", 2024-09-17T21:29:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9429999999999997e-06 2024-09-17T21:29:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095393+0000", 2024-09-17T21:29:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6059999999999996e-06 2024-09-17T21:29:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11197 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.099739+0000", 2024-09-17T21:29:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4294967295.9995823, 2024-09-17T21:29:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013429169999999999, 2024-09-17T21:29:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11197 2024-09-17T21:29:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099739+0000", 2024-09-17T21:29:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099739+0000", 2024-09-17T21:29:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099739+0000", 2024-09-17T21:29:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.63e-07 2024-09-17T21:29:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099741+0000", 2024-09-17T21:29:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8959999999999999e-06 2024-09-17T21:29:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099749+0000", 2024-09-17T21:29:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9140000000000008e-06 2024-09-17T21:29:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099753+0000", 2024-09-17T21:29:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4299999999999999e-06 2024-09-17T21:29:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10817 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.082883+0000", 2024-09-17T21:29:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.016437595999999999, 2024-09-17T21:29:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018248265, 2024-09-17T21:29:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10817 2024-09-17T21:29:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082883+0000", 2024-09-17T21:29:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082883+0000", 2024-09-17T21:29:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082884+0000", 2024-09-17T21:29:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0200000000000002e-07 2024-09-17T21:29:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082885+0000", 2024-09-17T21:29:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.807e-06 2024-09-17T21:29:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082891+0000", 2024-09-17T21:29:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6389999999999997e-06 2024-09-17T21:29:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082895+0000", 2024-09-17T21:29:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4109999999999998e-06 2024-09-17T21:29:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10905 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.086293+0000", 2024-09-17T21:29:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.013027584, 2024-09-17T21:29:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014886191, 2024-09-17T21:29:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10905 2024-09-17T21:29:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086293+0000", 2024-09-17T21:29:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086293+0000", 2024-09-17T21:29:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086294+0000", 2024-09-17T21:29:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:29:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086295+0000", 2024-09-17T21:29:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7719999999999999e-06 2024-09-17T21:29:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086299+0000", 2024-09-17T21:29:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1790000000000001e-06 2024-09-17T21:29:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086304+0000", 2024-09-17T21:29:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9409999999999998e-06 2024-09-17T21:29:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10979 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.087032+0000", 2024-09-17T21:29:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012289115, 2024-09-17T21:29:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014196780000000001, 2024-09-17T21:29:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10979 2024-09-17T21:29:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087032+0000", 2024-09-17T21:29:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087032+0000", 2024-09-17T21:29:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087032+0000", 2024-09-17T21:29:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:29:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087034+0000", 2024-09-17T21:29:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6750000000000001e-06 2024-09-17T21:29:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087040+0000", 2024-09-17T21:29:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.178e-06 2024-09-17T21:29:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087044+0000", 2024-09-17T21:29:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5299999999999998e-06 2024-09-17T21:29:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11111 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.095400+0000", 2024-09-17T21:29:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0039211339999999997, 2024-09-17T21:29:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0058735619999999997, 2024-09-17T21:29:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11111 2024-09-17T21:29:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095400+0000", 2024-09-17T21:29:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095400+0000", 2024-09-17T21:29:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095400+0000", 2024-09-17T21:29:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.03e-07 2024-09-17T21:29:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095408+0000", 2024-09-17T21:29:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5689999999999999e-06 2024-09-17T21:29:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095422+0000", 2024-09-17T21:29:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4045e-05 2024-09-17T21:29:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095426+0000", 2024-09-17T21:29:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7809999999999996e-06 2024-09-17T21:29:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11198 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.099773+0000", 2024-09-17T21:29:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4294967295.9995484, 2024-09-17T21:29:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015520810000000001, 2024-09-17T21:29:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11198 2024-09-17T21:29:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099773+0000", 2024-09-17T21:29:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099773+0000", 2024-09-17T21:29:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099773+0000", 2024-09-17T21:29:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4299999999999998e-07 2024-09-17T21:29:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099775+0000", 2024-09-17T21:29:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1739999999999999e-06 2024-09-17T21:29:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099781+0000", 2024-09-17T21:29:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2720000000000003e-06 2024-09-17T21:29:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099947+0000", 2024-09-17T21:29:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016547600000000001 2024-09-17T21:29:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10818 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.082902+0000", 2024-09-17T21:29:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.016418997000000001, 2024-09-17T21:29:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018473446000000001, 2024-09-17T21:29:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10818 2024-09-17T21:29:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082902+0000", 2024-09-17T21:29:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082902+0000", 2024-09-17T21:29:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082902+0000", 2024-09-17T21:29:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2500000000000001e-07 2024-09-17T21:29:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082904+0000", 2024-09-17T21:29:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.787e-06 2024-09-17T21:29:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082910+0000", 2024-09-17T21:29:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7089999999999999e-06 2024-09-17T21:29:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082914+0000", 2024-09-17T21:29:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3830000000000002e-06 2024-09-17T21:29:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10906 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.086312+0000", 2024-09-17T21:29:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.013008860000000001, 2024-09-17T21:29:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015111641, 2024-09-17T21:29:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10906 2024-09-17T21:29:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086312+0000", 2024-09-17T21:29:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086312+0000", 2024-09-17T21:29:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086312+0000", 2024-09-17T21:29:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9900000000000001e-07 2024-09-17T21:29:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086314+0000", 2024-09-17T21:29:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.973e-06 2024-09-17T21:29:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086320+0000", 2024-09-17T21:29:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6570000000000002e-06 2024-09-17T21:29:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086324+0000", 2024-09-17T21:29:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3590000000000001e-06 2024-09-17T21:29:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10980 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.087050+0000", 2024-09-17T21:29:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012270351000000001, 2024-09-17T21:29:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014418024999999999, 2024-09-17T21:29:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10980 2024-09-17T21:29:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087050+0000", 2024-09-17T21:29:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087050+0000", 2024-09-17T21:29:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087051+0000", 2024-09-17T21:29:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:29:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087053+0000", 2024-09-17T21:29:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7680000000000001e-06 2024-09-17T21:29:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087059+0000", 2024-09-17T21:29:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199999999999999e-06 2024-09-17T21:29:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087064+0000", 2024-09-17T21:29:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6739999999999996e-06 2024-09-17T21:29:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11199 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.099981+0000", 2024-09-17T21:29:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4294967295.9993396, 2024-09-17T21:29:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001534182, 2024-09-17T21:29:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11199 2024-09-17T21:29:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099981+0000", 2024-09-17T21:29:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099981+0000", 2024-09-17T21:29:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099988+0000", 2024-09-17T21:29:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6059999999999999e-06 2024-09-17T21:29:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099998+0000", 2024-09-17T21:29:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0511e-05 2024-09-17T21:29:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100008+0000", 2024-09-17T21:29:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0104e-05 2024-09-17T21:29:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100015+0000", 2024-09-17T21:29:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7789999999999998e-06 2024-09-17T21:29:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10819 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.082920+0000", 2024-09-17T21:29:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.016400806, 2024-09-17T21:29:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018641999999999999, 2024-09-17T21:29:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10819 2024-09-17T21:29:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082920+0000", 2024-09-17T21:29:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082920+0000", 2024-09-17T21:29:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082920+0000", 2024-09-17T21:29:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5299999999999999e-07 2024-09-17T21:29:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082922+0000", 2024-09-17T21:29:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.818e-06 2024-09-17T21:29:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082928+0000", 2024-09-17T21:29:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.913e-06 2024-09-17T21:29:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.082933+0000", 2024-09-17T21:29:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4159999999999997e-06 2024-09-17T21:29:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10907 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.086331+0000", 2024-09-17T21:29:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012989916000000001, 2024-09-17T21:29:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015274494, 2024-09-17T21:29:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10907 2024-09-17T21:29:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086331+0000", 2024-09-17T21:29:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086331+0000", 2024-09-17T21:29:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086331+0000", 2024-09-17T21:29:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:29:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086333+0000", 2024-09-17T21:29:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8250000000000001e-06 2024-09-17T21:29:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086341+0000", 2024-09-17T21:29:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8879999999999997e-06 2024-09-17T21:29:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086353+0000", 2024-09-17T21:29:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2449999999999999e-05 2024-09-17T21:29:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10981 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.087274+0000", 2024-09-17T21:29:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012046433, 2024-09-17T21:29:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014376945, 2024-09-17T21:29:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10981 2024-09-17T21:29:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087274+0000", 2024-09-17T21:29:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087274+0000", 2024-09-17T21:29:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087275+0000", 2024-09-17T21:29:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0100000000000005e-07 2024-09-17T21:29:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087278+0000", 2024-09-17T21:29:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1990000000000002e-06 2024-09-17T21:29:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087286+0000", 2024-09-17T21:29:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1389999999999995e-06 2024-09-17T21:29:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087292+0000", 2024-09-17T21:29:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.959e-06 2024-09-17T21:29:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11200 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.100025+0000", 2024-09-17T21:29:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4294967295.9992957, 2024-09-17T21:29:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001671411, 2024-09-17T21:29:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11200 2024-09-17T21:29:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100025+0000", 2024-09-17T21:29:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100025+0000", 2024-09-17T21:29:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100026+0000", 2024-09-17T21:29:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-17T21:29:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100028+0000", 2024-09-17T21:29:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5469999999999999e-06 2024-09-17T21:29:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100035+0000", 2024-09-17T21:29:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9589999999999998e-06 2024-09-17T21:29:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100040+0000", 2024-09-17T21:29:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7940000000000002e-06 2024-09-17T21:29:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10908 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.086361+0000", 2024-09-17T21:29:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012959557, 2024-09-17T21:29:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015385984, 2024-09-17T21:29:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10908 2024-09-17T21:29:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086361+0000", 2024-09-17T21:29:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086361+0000", 2024-09-17T21:29:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086362+0000", 2024-09-17T21:29:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3799999999999997e-07 2024-09-17T21:29:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086365+0000", 2024-09-17T21:29:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0699999999999998e-06 2024-09-17T21:29:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086374+0000", 2024-09-17T21:29:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1579999999999994e-06 2024-09-17T21:29:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086379+0000", 2024-09-17T21:29:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:29:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10982 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.087302+0000", 2024-09-17T21:29:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012018236999999999, 2024-09-17T21:29:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014490503, 2024-09-17T21:29:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10982 2024-09-17T21:29:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087302+0000", 2024-09-17T21:29:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087302+0000", 2024-09-17T21:29:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087303+0000", 2024-09-17T21:29:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9399999999999995e-07 2024-09-17T21:29:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087307+0000", 2024-09-17T21:29:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1989999999999999e-06 2024-09-17T21:29:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087315+0000", 2024-09-17T21:29:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0239999999999997e-06 2024-09-17T21:29:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087320+0000", 2024-09-17T21:29:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:29:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11201 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.100047+0000", 2024-09-17T21:29:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4294967295.9992738, 2024-09-17T21:29:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017987610000000001, 2024-09-17T21:29:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11201 2024-09-17T21:29:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100047+0000", 2024-09-17T21:29:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100047+0000", 2024-09-17T21:29:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100048+0000", 2024-09-17T21:29:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.63e-07 2024-09-17T21:29:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100052+0000", 2024-09-17T21:29:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.228e-06 2024-09-17T21:29:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100059+0000", 2024-09-17T21:29:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.691e-06 2024-09-17T21:29:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100064+0000", 2024-09-17T21:29:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8690000000000001e-06 2024-09-17T21:29:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10909 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.086388+0000", 2024-09-17T21:29:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012932747, 2024-09-17T21:29:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015485281, 2024-09-17T21:29:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10909 2024-09-17T21:29:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086388+0000", 2024-09-17T21:29:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086388+0000", 2024-09-17T21:29:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086388+0000", 2024-09-17T21:29:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:29:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086399+0000", 2024-09-17T21:29:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0183999999999999e-05 2024-09-17T21:29:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086406+0000", 2024-09-17T21:29:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0859999999999996e-06 2024-09-17T21:29:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086410+0000", 2024-09-17T21:29:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7720000000000002e-06 2024-09-17T21:29:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10983 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.087328+0000", 2024-09-17T21:29:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.011992467999999999, 2024-09-17T21:29:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014576584, 2024-09-17T21:29:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10983 2024-09-17T21:29:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087328+0000", 2024-09-17T21:29:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087328+0000", 2024-09-17T21:29:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087329+0000", 2024-09-17T21:29:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:29:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087331+0000", 2024-09-17T21:29:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0760000000000001e-06 2024-09-17T21:29:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087337+0000", 2024-09-17T21:29:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4810000000000003e-06 2024-09-17T21:29:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087342+0000", 2024-09-17T21:29:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9919999999999998e-06 2024-09-17T21:29:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11202 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.100072+0000", 2024-09-17T21:29:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4294967295.999249, 2024-09-17T21:29:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018565400000000001, 2024-09-17T21:29:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11202 2024-09-17T21:29:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100072+0000", 2024-09-17T21:29:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100072+0000", 2024-09-17T21:29:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100072+0000", 2024-09-17T21:29:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:29:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100074+0000", 2024-09-17T21:29:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8959999999999999e-06 2024-09-17T21:29:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100081+0000", 2024-09-17T21:29:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5440000000000004e-06 2024-09-17T21:29:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100086+0000", 2024-09-17T21:29:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7849999999999999e-06 2024-09-17T21:29:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10910 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.086418+0000", 2024-09-17T21:29:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012902591, 2024-09-17T21:29:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015536737, 2024-09-17T21:29:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10910 2024-09-17T21:29:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086418+0000", 2024-09-17T21:29:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086418+0000", 2024-09-17T21:29:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086419+0000", 2024-09-17T21:29:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:29:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086421+0000", 2024-09-17T21:29:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.046e-06 2024-09-17T21:29:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086426+0000", 2024-09-17T21:29:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5609999999999998e-06 2024-09-17T21:29:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086433+0000", 2024-09-17T21:29:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3269999999999997e-06 2024-09-17T21:29:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10984 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.087350+0000", 2024-09-17T21:29:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.01197118, 2024-09-17T21:29:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014629207, 2024-09-17T21:29:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10984 2024-09-17T21:29:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087350+0000", 2024-09-17T21:29:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087350+0000", 2024-09-17T21:29:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087350+0000", 2024-09-17T21:29:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-17T21:29:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087352+0000", 2024-09-17T21:29:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.917e-06 2024-09-17T21:29:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087358+0000", 2024-09-17T21:29:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5259999999999999e-06 2024-09-17T21:29:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087364+0000", 2024-09-17T21:29:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5559999999999998e-06 2024-09-17T21:29:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11071 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.091148+0000", 2024-09-17T21:29:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0081725749999999996, 2024-09-17T21:29:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010873542, 2024-09-17T21:29:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11071 2024-09-17T21:29:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091148+0000", 2024-09-17T21:29:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091148+0000", 2024-09-17T21:29:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091149+0000", 2024-09-17T21:29:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5699999999999998e-07 2024-09-17T21:29:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091150+0000", 2024-09-17T21:29:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.905e-06 2024-09-17T21:29:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091157+0000", 2024-09-17T21:29:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6069999999999996e-06 2024-09-17T21:29:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091162+0000", 2024-09-17T21:29:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0159999999999999e-06 2024-09-17T21:29:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11203 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.100092+0000", 2024-09-17T21:29:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4294967295.9992285, 2024-09-17T21:29:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019769119999999999, 2024-09-17T21:29:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11203 2024-09-17T21:29:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100092+0000", 2024-09-17T21:29:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100092+0000", 2024-09-17T21:29:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100093+0000", 2024-09-17T21:29:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:29:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100094+0000", 2024-09-17T21:29:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8619999999999999e-06 2024-09-17T21:29:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100101+0000", 2024-09-17T21:29:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4409999999999998e-06 2024-09-17T21:29:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100106+0000", 2024-09-17T21:29:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:29:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10911 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.086440+0000", 2024-09-17T21:29:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012880252, 2024-09-17T21:29:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015683964000000002, 2024-09-17T21:29:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10911 2024-09-17T21:29:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086440+0000", 2024-09-17T21:29:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086440+0000", 2024-09-17T21:29:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086441+0000", 2024-09-17T21:29:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2800000000000002e-07 2024-09-17T21:29:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086443+0000", 2024-09-17T21:29:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0549999999999998e-06 2024-09-17T21:29:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086448+0000", 2024-09-17T21:29:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5269999999999998e-06 2024-09-17T21:29:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086453+0000", 2024-09-17T21:29:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4020000000000004e-06 2024-09-17T21:29:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10985 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.087371+0000", 2024-09-17T21:29:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.011949909, 2024-09-17T21:29:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014801096999999999, 2024-09-17T21:29:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10985 2024-09-17T21:29:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087371+0000", 2024-09-17T21:29:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087371+0000", 2024-09-17T21:29:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087371+0000", 2024-09-17T21:29:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1900000000000003e-07 2024-09-17T21:29:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087373+0000", 2024-09-17T21:29:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8759999999999999e-06 2024-09-17T21:29:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087380+0000", 2024-09-17T21:29:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.365e-06 2024-09-17T21:29:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087393+0000", 2024-09-17T21:29:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3468e-05 2024-09-17T21:29:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11072 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.091201+0000", 2024-09-17T21:29:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0081199789999999994, 2024-09-17T21:29:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011022290000000001, 2024-09-17T21:29:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11072 2024-09-17T21:29:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091201+0000", 2024-09-17T21:29:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091201+0000", 2024-09-17T21:29:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091202+0000", 2024-09-17T21:29:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6400000000000001e-07 2024-09-17T21:29:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091206+0000", 2024-09-17T21:29:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1269999999999996e-06 2024-09-17T21:29:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091217+0000", 2024-09-17T21:29:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-05 2024-09-17T21:29:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091225+0000", 2024-09-17T21:29:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2690000000000002e-06 2024-09-17T21:29:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11204 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.100112+0000", 2024-09-17T21:29:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4294967295.9992085, 2024-09-17T21:29:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021410370000000001, 2024-09-17T21:29:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11204 2024-09-17T21:29:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100112+0000", 2024-09-17T21:29:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100112+0000", 2024-09-17T21:29:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100113+0000", 2024-09-17T21:29:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2599999999999998e-07 2024-09-17T21:29:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100115+0000", 2024-09-17T21:29:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:29:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100122+0000", 2024-09-17T21:29:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1289999999999998e-06 2024-09-17T21:29:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100127+0000", 2024-09-17T21:29:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1669999999999998e-06 2024-09-17T21:29:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10912 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.086459+0000", 2024-09-17T21:29:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012861622, 2024-09-17T21:29:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015817443, 2024-09-17T21:29:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10912 2024-09-17T21:29:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086459+0000", 2024-09-17T21:29:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086459+0000", 2024-09-17T21:29:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086459+0000", 2024-09-17T21:29:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:29:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086461+0000", 2024-09-17T21:29:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9139999999999998e-06 2024-09-17T21:29:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086467+0000", 2024-09-17T21:29:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9560000000000002e-06 2024-09-17T21:29:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086472+0000", 2024-09-17T21:29:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3610000000000003e-06 2024-09-17T21:29:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10986 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.087401+0000", 2024-09-17T21:29:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.011919729, 2024-09-17T21:29:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014984053000000001, 2024-09-17T21:29:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10986 2024-09-17T21:29:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087401+0000", 2024-09-17T21:29:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087401+0000", 2024-09-17T21:29:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087401+0000", 2024-09-17T21:29:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:29:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087408+0000", 2024-09-17T21:29:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5860000000000001e-06 2024-09-17T21:29:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087415+0000", 2024-09-17T21:29:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9140000000000002e-06 2024-09-17T21:29:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087420+0000", 2024-09-17T21:29:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2569999999999998e-06 2024-09-17T21:29:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11073 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.091237+0000", 2024-09-17T21:29:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0080833620000000002, 2024-09-17T21:29:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011195867999999999, 2024-09-17T21:29:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11073 2024-09-17T21:29:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091237+0000", 2024-09-17T21:29:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091237+0000", 2024-09-17T21:29:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091238+0000", 2024-09-17T21:29:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.51e-07 2024-09-17T21:29:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091241+0000", 2024-09-17T21:29:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9560000000000002e-06 2024-09-17T21:29:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091248+0000", 2024-09-17T21:29:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4170000000000003e-06 2024-09-17T21:29:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091254+0000", 2024-09-17T21:29:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6960000000000002e-06 2024-09-17T21:29:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11205 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.100133+0000", 2024-09-17T21:29:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4294967295.9991875, 2024-09-17T21:29:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023509490000000002, 2024-09-17T21:29:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11205 2024-09-17T21:29:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100133+0000", 2024-09-17T21:29:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100133+0000", 2024-09-17T21:29:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100134+0000", 2024-09-17T21:29:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:29:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100136+0000", 2024-09-17T21:29:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.841e-06 2024-09-17T21:29:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100142+0000", 2024-09-17T21:29:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2790000000000004e-06 2024-09-17T21:29:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100147+0000", 2024-09-17T21:29:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8470000000000003e-06 2024-09-17T21:29:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10913 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.086479+0000", 2024-09-17T21:29:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012841814, 2024-09-17T21:29:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016051884999999998, 2024-09-17T21:29:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10913 2024-09-17T21:29:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086479+0000", 2024-09-17T21:29:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086479+0000", 2024-09-17T21:29:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086479+0000", 2024-09-17T21:29:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:29:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086487+0000", 2024-09-17T21:29:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1089999999999998e-06 2024-09-17T21:29:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086493+0000", 2024-09-17T21:29:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9630000000000003e-06 2024-09-17T21:29:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086498+0000", 2024-09-17T21:29:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6519999999999997e-06 2024-09-17T21:29:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10987 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.087428+0000", 2024-09-17T21:29:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.011892477, 2024-09-17T21:29:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015142652, 2024-09-17T21:29:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10987 2024-09-17T21:29:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087428+0000", 2024-09-17T21:29:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087428+0000", 2024-09-17T21:29:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087429+0000", 2024-09-17T21:29:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-07 2024-09-17T21:29:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087430+0000", 2024-09-17T21:29:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7630000000000001e-06 2024-09-17T21:29:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087437+0000", 2024-09-17T21:29:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5089999999999999e-06 2024-09-17T21:29:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.087442+0000", 2024-09-17T21:29:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0119999999999996e-06 2024-09-17T21:29:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11074 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.091262+0000", 2024-09-17T21:29:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0080590780000000008, 2024-09-17T21:29:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011356049, 2024-09-17T21:29:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11074 2024-09-17T21:29:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091262+0000", 2024-09-17T21:29:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091262+0000", 2024-09-17T21:29:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091262+0000", 2024-09-17T21:29:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.27e-07 2024-09-17T21:29:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091264+0000", 2024-09-17T21:29:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0959999999999999e-06 2024-09-17T21:29:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091271+0000", 2024-09-17T21:29:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:29:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091276+0000", 2024-09-17T21:29:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1850000000000003e-06 2024-09-17T21:29:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11206 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.100153+0000", 2024-09-17T21:29:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4294967295.9991679, 2024-09-17T21:29:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025809399999999999, 2024-09-17T21:29:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11206 2024-09-17T21:29:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100153+0000", 2024-09-17T21:29:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100153+0000", 2024-09-17T21:29:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100153+0000", 2024-09-17T21:29:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.01e-07 2024-09-17T21:29:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100155+0000", 2024-09-17T21:29:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.742e-06 2024-09-17T21:29:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100161+0000", 2024-09-17T21:29:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2369999999999997e-06 2024-09-17T21:29:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100166+0000", 2024-09-17T21:29:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9359999999999998e-06 2024-09-17T21:29:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10914 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.086505+0000", 2024-09-17T21:29:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012815481, 2024-09-17T21:29:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016276948999999999, 2024-09-17T21:29:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10914 2024-09-17T21:29:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086505+0000", 2024-09-17T21:29:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086505+0000", 2024-09-17T21:29:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086506+0000", 2024-09-17T21:29:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:29:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086508+0000", 2024-09-17T21:29:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0719999999999998e-06 2024-09-17T21:29:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086513+0000", 2024-09-17T21:29:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5570000000000003e-06 2024-09-17T21:29:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086518+0000", 2024-09-17T21:29:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3200000000000001e-06 2024-09-17T21:29:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11001 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.090367+0000", 2024-09-17T21:29:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0089541039999999992, 2024-09-17T21:29:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012470774, 2024-09-17T21:29:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11001 2024-09-17T21:29:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090367+0000", 2024-09-17T21:29:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090367+0000", 2024-09-17T21:29:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090367+0000", 2024-09-17T21:29:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000001e-07 2024-09-17T21:29:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090378+0000", 2024-09-17T21:29:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1055e-05 2024-09-17T21:29:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090385+0000", 2024-09-17T21:29:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-06 2024-09-17T21:29:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090389+0000", 2024-09-17T21:29:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3019999999999996e-06 2024-09-17T21:29:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11075 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.091283+0000", 2024-09-17T21:29:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0080378629999999993, 2024-09-17T21:29:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011603657999999999, 2024-09-17T21:29:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11075 2024-09-17T21:29:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091283+0000", 2024-09-17T21:29:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091283+0000", 2024-09-17T21:29:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091283+0000", 2024-09-17T21:29:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:29:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091286+0000", 2024-09-17T21:29:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2539999999999999e-06 2024-09-17T21:29:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091292+0000", 2024-09-17T21:29:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:29:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091297+0000", 2024-09-17T21:29:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.843e-06 2024-09-17T21:29:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11207 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.100186+0000", 2024-09-17T21:29:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4294967295.9991345, 2024-09-17T21:29:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027524870000000001, 2024-09-17T21:29:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11207 2024-09-17T21:29:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100186+0000", 2024-09-17T21:29:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100186+0000", 2024-09-17T21:29:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100187+0000", 2024-09-17T21:29:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:29:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100189+0000", 2024-09-17T21:29:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5359999999999999e-06 2024-09-17T21:29:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100197+0000", 2024-09-17T21:29:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3080000000000002e-06 2024-09-17T21:29:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100212+0000", 2024-09-17T21:29:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5047e-05 2024-09-17T21:29:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10915 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.086532+0000", 2024-09-17T21:29:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012788760999999999, 2024-09-17T21:29:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016453448999999998, 2024-09-17T21:29:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10915 2024-09-17T21:29:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086532+0000", 2024-09-17T21:29:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086532+0000", 2024-09-17T21:29:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086532+0000", 2024-09-17T21:29:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1699999999999999e-07 2024-09-17T21:29:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086534+0000", 2024-09-17T21:29:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3599999999999999e-06 2024-09-17T21:29:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086540+0000", 2024-09-17T21:29:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8329999999999996e-06 2024-09-17T21:29:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086543+0000", 2024-09-17T21:29:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8049999999999998e-06 2024-09-17T21:29:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11002 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.090401+0000", 2024-09-17T21:29:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0089196420000000002, 2024-09-17T21:29:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012632625999999999, 2024-09-17T21:29:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11002 2024-09-17T21:29:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090401+0000", 2024-09-17T21:29:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090401+0000", 2024-09-17T21:29:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090401+0000", 2024-09-17T21:29:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:29:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090403+0000", 2024-09-17T21:29:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:29:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090407+0000", 2024-09-17T21:29:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9709999999999998e-06 2024-09-17T21:29:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090410+0000", 2024-09-17T21:29:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9299999999999999e-06 2024-09-17T21:29:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11076 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.091303+0000", 2024-09-17T21:29:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0080175760000000002, 2024-09-17T21:29:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011776922, 2024-09-17T21:29:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11076 2024-09-17T21:29:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091303+0000", 2024-09-17T21:29:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091303+0000", 2024-09-17T21:29:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091304+0000", 2024-09-17T21:29:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:29:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091316+0000", 2024-09-17T21:29:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2428e-05 2024-09-17T21:29:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091323+0000", 2024-09-17T21:29:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0520000000000004e-06 2024-09-17T21:29:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091333+0000", 2024-09-17T21:29:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6579999999999997e-06 2024-09-17T21:29:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11208 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.100230+0000", 2024-09-17T21:29:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4294967295.9990911, 2024-09-17T21:29:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029046139999999998, 2024-09-17T21:29:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11208 2024-09-17T21:29:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100230+0000", 2024-09-17T21:29:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100230+0000", 2024-09-17T21:29:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100230+0000", 2024-09-17T21:29:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5000000000000003e-07 2024-09-17T21:29:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100233+0000", 2024-09-17T21:29:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5720000000000001e-06 2024-09-17T21:29:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100241+0000", 2024-09-17T21:29:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3520000000000007e-06 2024-09-17T21:29:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100254+0000", 2024-09-17T21:29:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2809000000000001e-05 2024-09-17T21:29:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10916 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.086548+0000", 2024-09-17T21:29:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012773122, 2024-09-17T21:29:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016635550999999998, 2024-09-17T21:29:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10916 2024-09-17T21:29:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086548+0000", 2024-09-17T21:29:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086548+0000", 2024-09-17T21:29:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086548+0000", 2024-09-17T21:29:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6100000000000002e-07 2024-09-17T21:29:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086549+0000", 2024-09-17T21:29:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1209999999999999e-06 2024-09-17T21:29:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086552+0000", 2024-09-17T21:29:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3759999999999999e-06 2024-09-17T21:29:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086560+0000", 2024-09-17T21:29:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2930000000000003e-06 2024-09-17T21:29:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11003 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.090420+0000", 2024-09-17T21:29:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0089006859999999997, 2024-09-17T21:29:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012808559000000001, 2024-09-17T21:29:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11003 2024-09-17T21:29:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090420+0000", 2024-09-17T21:29:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090420+0000", 2024-09-17T21:29:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090420+0000", 2024-09-17T21:29:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:29:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090422+0000", 2024-09-17T21:29:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.595e-06 2024-09-17T21:29:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090426+0000", 2024-09-17T21:29:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9339999999999999e-06 2024-09-17T21:29:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090429+0000", 2024-09-17T21:29:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7480000000000002e-06 2024-09-17T21:29:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11077 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.091340+0000", 2024-09-17T21:29:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0079809270000000005, 2024-09-17T21:29:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011938706, 2024-09-17T21:29:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11077 2024-09-17T21:29:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091340+0000", 2024-09-17T21:29:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091340+0000", 2024-09-17T21:29:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091340+0000", 2024-09-17T21:29:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:29:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091342+0000", 2024-09-17T21:29:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.198e-06 2024-09-17T21:29:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091349+0000", 2024-09-17T21:29:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1689999999999997e-06 2024-09-17T21:29:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091353+0000", 2024-09-17T21:29:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8690000000000003e-06 2024-09-17T21:29:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10917 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.086563+0000", 2024-09-17T21:29:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.012757378999999999, 2024-09-17T21:29:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016762947, 2024-09-17T21:29:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10917 2024-09-17T21:29:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086563+0000", 2024-09-17T21:29:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086563+0000", 2024-09-17T21:29:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086564+0000", 2024-09-17T21:29:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4299999999999999e-07 2024-09-17T21:29:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086565+0000", 2024-09-17T21:29:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.063e-06 2024-09-17T21:29:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086568+0000", 2024-09-17T21:29:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:29:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.086574+0000", 2024-09-17T21:29:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0680000000000002e-06 2024-09-17T21:29:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11004 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.090435+0000", 2024-09-17T21:29:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0088858849999999996, 2024-09-17T21:29:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012940464, 2024-09-17T21:29:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11004 2024-09-17T21:29:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090435+0000", 2024-09-17T21:29:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090435+0000", 2024-09-17T21:29:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090435+0000", 2024-09-17T21:29:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3900000000000001e-07 2024-09-17T21:29:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090437+0000", 2024-09-17T21:29:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5009999999999999e-06 2024-09-17T21:29:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090442+0000", 2024-09-17T21:29:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3329999999999996e-06 2024-09-17T21:29:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090447+0000", 2024-09-17T21:29:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:29:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11078 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.091374+0000", 2024-09-17T21:29:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0079467640000000003, 2024-09-17T21:29:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012048237999999999, 2024-09-17T21:29:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11078 2024-09-17T21:29:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091374+0000", 2024-09-17T21:29:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091374+0000", 2024-09-17T21:29:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091374+0000", 2024-09-17T21:29:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:29:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091377+0000", 2024-09-17T21:29:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0439999999999998e-06 2024-09-17T21:29:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091383+0000", 2024-09-17T21:29:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.516e-06 2024-09-17T21:29:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091388+0000", 2024-09-17T21:29:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6759999999999998e-06 2024-09-17T21:29:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11005 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.090459+0000", 2024-09-17T21:29:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.008861964, 2024-09-17T21:29:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013016148999999999, 2024-09-17T21:29:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11005 2024-09-17T21:29:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090459+0000", 2024-09-17T21:29:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090459+0000", 2024-09-17T21:29:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090459+0000", 2024-09-17T21:29:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8999999999999996e-07 2024-09-17T21:29:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090461+0000", 2024-09-17T21:29:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9580000000000001e-06 2024-09-17T21:29:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090467+0000", 2024-09-17T21:29:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.378e-06 2024-09-17T21:29:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090471+0000", 2024-09-17T21:29:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3540000000000002e-06 2024-09-17T21:29:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11079 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.091395+0000", 2024-09-17T21:29:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0079258799999999997, 2024-09-17T21:29:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012129971999999999, 2024-09-17T21:29:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11079 2024-09-17T21:29:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091395+0000", 2024-09-17T21:29:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091395+0000", 2024-09-17T21:29:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091395+0000", 2024-09-17T21:29:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8500000000000001e-07 2024-09-17T21:29:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091397+0000", 2024-09-17T21:29:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0449999999999999e-06 2024-09-17T21:29:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091403+0000", 2024-09-17T21:29:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0190000000000003e-06 2024-09-17T21:29:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091410+0000", 2024-09-17T21:29:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999999e-06 2024-09-17T21:29:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11006 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.090481+0000", 2024-09-17T21:29:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0088394440000000001, 2024-09-17T21:29:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013096379999999999, 2024-09-17T21:29:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11006 2024-09-17T21:29:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090481+0000", 2024-09-17T21:29:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090481+0000", 2024-09-17T21:29:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090482+0000", 2024-09-17T21:29:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:29:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090484+0000", 2024-09-17T21:29:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.888e-06 2024-09-17T21:29:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090489+0000", 2024-09-17T21:29:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8719999999999999e-06 2024-09-17T21:29:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090497+0000", 2024-09-17T21:29:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:29:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11080 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.091417+0000", 2024-09-17T21:29:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0079039149999999992, 2024-09-17T21:29:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012206711, 2024-09-17T21:29:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11080 2024-09-17T21:29:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091417+0000", 2024-09-17T21:29:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091417+0000", 2024-09-17T21:29:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091417+0000", 2024-09-17T21:29:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1399999999999997e-07 2024-09-17T21:29:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091426+0000", 2024-09-17T21:29:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4200000000000007e-06 2024-09-17T21:29:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091433+0000", 2024-09-17T21:29:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7200000000000006e-06 2024-09-17T21:29:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091439+0000", 2024-09-17T21:29:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4e-06 2024-09-17T21:29:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11007 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.090504+0000", 2024-09-17T21:29:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.008816328, 2024-09-17T21:29:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.01316847, 2024-09-17T21:29:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11007 2024-09-17T21:29:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090504+0000", 2024-09-17T21:29:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090504+0000", 2024-09-17T21:29:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090505+0000", 2024-09-17T21:29:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:29:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090507+0000", 2024-09-17T21:29:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8700000000000001e-06 2024-09-17T21:29:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090513+0000", 2024-09-17T21:29:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4659999999999996e-06 2024-09-17T21:29:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090528+0000", 2024-09-17T21:29:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4452e-05 2024-09-17T21:29:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11081 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.091446+0000", 2024-09-17T21:29:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0078743900000000002, 2024-09-17T21:29:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012272706, 2024-09-17T21:29:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11081 2024-09-17T21:29:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091446+0000", 2024-09-17T21:29:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091446+0000", 2024-09-17T21:29:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091447+0000", 2024-09-17T21:29:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:29:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091449+0000", 2024-09-17T21:29:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1260000000000001e-06 2024-09-17T21:29:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091456+0000", 2024-09-17T21:29:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0949999999999998e-06 2024-09-17T21:29:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091463+0000", 2024-09-17T21:29:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5030000000000002e-06 2024-09-17T21:29:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11168 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.095813+0000", 2024-09-17T21:29:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0035078129999999998, 2024-09-17T21:29:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079540860000000008, 2024-09-17T21:29:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11168 2024-09-17T21:29:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095813+0000", 2024-09-17T21:29:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095813+0000", 2024-09-17T21:29:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095813+0000", 2024-09-17T21:29:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8299999999999997e-07 2024-09-17T21:29:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095816+0000", 2024-09-17T21:29:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.294e-06 2024-09-17T21:29:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095822+0000", 2024-09-17T21:29:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.613e-06 2024-09-17T21:29:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095828+0000", 2024-09-17T21:29:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9270000000000002e-06 2024-09-17T21:29:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10797 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.065443+0000", 2024-09-17T21:29:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.033877313999999999, 2024-09-17T21:29:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038377242999999998, 2024-09-17T21:29:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:29:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10797 2024-09-17T21:29:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065443+0000", 2024-09-17T21:29:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065443+0000", 2024-09-17T21:29:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065444+0000", 2024-09-17T21:29:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7899999999999999e-07 2024-09-17T21:29:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065451+0000", 2024-09-17T21:29:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2200000000000003e-06 2024-09-17T21:29:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065457+0000", 2024-09-17T21:29:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.161e-06 2024-09-17T21:29:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065462+0000", 2024-09-17T21:29:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4109999999999998e-06 2024-09-17T21:29:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099269+0000", 2024-09-17T21:29:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.03380735 2024-09-17T21:29:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099325+0000", 2024-09-17T21:29:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6057000000000002e-05 2024-09-17T21:29:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:29:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099562+0000", 2024-09-17T21:29:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00023666299999999999 2024-09-17T21:29:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11008 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.090535+0000", 2024-09-17T21:29:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0087857549999999993, 2024-09-17T21:29:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013354945, 2024-09-17T21:29:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11008 2024-09-17T21:29:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090535+0000", 2024-09-17T21:29:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090535+0000", 2024-09-17T21:29:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090535+0000", 2024-09-17T21:29:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T21:29:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090538+0000", 2024-09-17T21:29:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1280000000000002e-06 2024-09-17T21:29:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090546+0000", 2024-09-17T21:29:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5229999999999995e-06 2024-09-17T21:29:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090558+0000", 2024-09-17T21:29:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1579e-05 2024-09-17T21:29:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11082 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.091470+0000", 2024-09-17T21:29:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0078509740000000001, 2024-09-17T21:29:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0124682, 2024-09-17T21:29:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11082 2024-09-17T21:29:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091470+0000", 2024-09-17T21:29:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091470+0000", 2024-09-17T21:29:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091470+0000", 2024-09-17T21:29:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:29:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091472+0000", 2024-09-17T21:29:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0880000000000002e-06 2024-09-17T21:29:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091478+0000", 2024-09-17T21:29:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0240000000000003e-06 2024-09-17T21:29:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091484+0000", 2024-09-17T21:29:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4149999999999998e-06 2024-09-17T21:29:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11169 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.095844+0000", 2024-09-17T21:29:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0034766939999999998, 2024-09-17T21:29:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0081450099999999994, 2024-09-17T21:29:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11169 2024-09-17T21:29:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095844+0000", 2024-09-17T21:29:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095844+0000", 2024-09-17T21:29:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095844+0000", 2024-09-17T21:29:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2399999999999999e-07 2024-09-17T21:29:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095847+0000", 2024-09-17T21:29:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4080000000000001e-06 2024-09-17T21:29:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095854+0000", 2024-09-17T21:29:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9609999999999999e-06 2024-09-17T21:29:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095859+0000", 2024-09-17T21:29:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1189999999999996e-06 2024-09-17T21:29:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10799 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:29:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.033852620999999999, 2024-09-17T21:29:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038571144000000002, 2024-09-17T21:29:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:29:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10799 2024-09-17T21:29:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:29:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:29:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065469+0000", 2024-09-17T21:29:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:29:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065477+0000", 2024-09-17T21:29:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1319999999999994e-06 2024-09-17T21:29:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065483+0000", 2024-09-17T21:29:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2040000000000002e-06 2024-09-17T21:29:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065487+0000", 2024-09-17T21:29:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.42e-06 2024-09-17T21:29:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099984+0000", 2024-09-17T21:29:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034496484000000001 2024-09-17T21:29:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100065+0000", 2024-09-17T21:29:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1200999999999997e-05 2024-09-17T21:29:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:29:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100422+0000", 2024-09-17T21:29:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035657700000000001 2024-09-17T21:29:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10876 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.083473+0000", 2024-09-17T21:29:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.015847406000000001, 2024-09-17T21:29:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020631010000000002, 2024-09-17T21:29:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10876 2024-09-17T21:29:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083473+0000", 2024-09-17T21:29:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083473+0000", 2024-09-17T21:29:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083474+0000", 2024-09-17T21:29:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0900000000000002e-07 2024-09-17T21:29:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083476+0000", 2024-09-17T21:29:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:29:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083481+0000", 2024-09-17T21:29:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6690000000000003e-06 2024-09-17T21:29:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083486+0000", 2024-09-17T21:29:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3100000000000002e-06 2024-09-17T21:29:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11009 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.090566+0000", 2024-09-17T21:29:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.008754408, 2024-09-17T21:29:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013585785, 2024-09-17T21:29:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11009 2024-09-17T21:29:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090566+0000", 2024-09-17T21:29:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090566+0000", 2024-09-17T21:29:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090567+0000", 2024-09-17T21:29:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6400000000000003e-07 2024-09-17T21:29:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090569+0000", 2024-09-17T21:29:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.187e-06 2024-09-17T21:29:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090575+0000", 2024-09-17T21:29:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7400000000000001e-06 2024-09-17T21:29:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090580+0000", 2024-09-17T21:29:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2469999999999999e-06 2024-09-17T21:29:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11083 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.091491+0000", 2024-09-17T21:29:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.007830063, 2024-09-17T21:29:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012710279, 2024-09-17T21:29:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11083 2024-09-17T21:29:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091491+0000", 2024-09-17T21:29:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091491+0000", 2024-09-17T21:29:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091491+0000", 2024-09-17T21:29:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.27e-07 2024-09-17T21:29:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091493+0000", 2024-09-17T21:29:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9649999999999998e-06 2024-09-17T21:29:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091499+0000", 2024-09-17T21:29:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9959999999999999e-06 2024-09-17T21:29:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091504+0000", 2024-09-17T21:29:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7439999999999998e-06 2024-09-17T21:29:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11170 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.096508+0000", 2024-09-17T21:29:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0028130989999999999, 2024-09-17T21:29:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077387649999999999, 2024-09-17T21:29:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11170 2024-09-17T21:29:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096508+0000", 2024-09-17T21:29:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096508+0000", 2024-09-17T21:29:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096508+0000", 2024-09-17T21:29:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3499999999999995e-07 2024-09-17T21:29:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096512+0000", 2024-09-17T21:29:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7699999999999999e-06 2024-09-17T21:29:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096521+0000", 2024-09-17T21:29:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4370000000000007e-06 2024-09-17T21:29:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096527+0000", 2024-09-17T21:29:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7889999999999997e-06 2024-09-17T21:29:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10801 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.069714+0000", 2024-09-17T21:29:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.029606349000000001, 2024-09-17T21:29:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034587854000000001, 2024-09-17T21:29:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:29:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10801 2024-09-17T21:29:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.069714+0000", 2024-09-17T21:29:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.069714+0000", 2024-09-17T21:29:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.069716+0000", 2024-09-17T21:29:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.22e-06 2024-09-17T21:29:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.069721+0000", 2024-09-17T21:29:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4770000000000002e-06 2024-09-17T21:29:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.069730+0000", 2024-09-17T21:29:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6459999999999999e-06 2024-09-17T21:29:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.069737+0000", 2024-09-17T21:29:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5120000000000002e-06 2024-09-17T21:29:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100808+0000", 2024-09-17T21:29:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031070341000000001 2024-09-17T21:29:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100888+0000", 2024-09-17T21:29:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0038999999999997e-05 2024-09-17T21:29:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:29:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.101258+0000", 2024-09-17T21:29:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00037037099999999998 2024-09-17T21:29:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104218+0000", 2024-09-17T21:29:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002960369 2024-09-17T21:29:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10877 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.083492+0000", 2024-09-17T21:29:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.015829006999999999, 2024-09-17T21:29:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020874604000000001, 2024-09-17T21:29:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10877 2024-09-17T21:29:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083492+0000", 2024-09-17T21:29:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083492+0000", 2024-09-17T21:29:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083492+0000", 2024-09-17T21:29:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:29:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083499+0000", 2024-09-17T21:29:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6290000000000004e-06 2024-09-17T21:29:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083505+0000", 2024-09-17T21:29:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2249999999999997e-06 2024-09-17T21:29:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083510+0000", 2024-09-17T21:29:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5619999999999997e-06 2024-09-17T21:29:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11010 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.090593+0000", 2024-09-17T21:29:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0087280550000000002, 2024-09-17T21:29:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013820828, 2024-09-17T21:29:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11010 2024-09-17T21:29:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090593+0000", 2024-09-17T21:29:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090593+0000", 2024-09-17T21:29:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090593+0000", 2024-09-17T21:29:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:29:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090595+0000", 2024-09-17T21:29:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1720000000000001e-06 2024-09-17T21:29:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090602+0000", 2024-09-17T21:29:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9460000000000001e-06 2024-09-17T21:29:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090614+0000", 2024-09-17T21:29:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1644999999999999e-05 2024-09-17T21:29:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11084 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.091510+0000", 2024-09-17T21:29:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.007810572, 2024-09-17T21:29:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012948952999999999, 2024-09-17T21:29:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11084 2024-09-17T21:29:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091510+0000", 2024-09-17T21:29:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091510+0000", 2024-09-17T21:29:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091511+0000", 2024-09-17T21:29:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7e-07 2024-09-17T21:29:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091512+0000", 2024-09-17T21:29:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7099999999999999e-06 2024-09-17T21:29:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091518+0000", 2024-09-17T21:29:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7180000000000001e-06 2024-09-17T21:29:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.091522+0000", 2024-09-17T21:29:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4390000000000002e-06 2024-09-17T21:29:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11171 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.096539+0000", 2024-09-17T21:29:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0027819289999999998, 2024-09-17T21:29:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079695470000000004, 2024-09-17T21:29:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11171 2024-09-17T21:29:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096539+0000", 2024-09-17T21:29:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096539+0000", 2024-09-17T21:29:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096539+0000", 2024-09-17T21:29:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2099999999999997e-07 2024-09-17T21:29:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096542+0000", 2024-09-17T21:29:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5550000000000001e-06 2024-09-17T21:29:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096549+0000", 2024-09-17T21:29:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0890000000000002e-06 2024-09-17T21:29:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096555+0000", 2024-09-17T21:29:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.054e-06 2024-09-17T21:29:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10803 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.075002+0000", 2024-09-17T21:29:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.024318920000000001, 2024-09-17T21:29:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.029558164000000001, 2024-09-17T21:29:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:29:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10803 2024-09-17T21:29:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.075002+0000", 2024-09-17T21:29:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.075002+0000", 2024-09-17T21:29:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.075003+0000", 2024-09-17T21:29:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1629999999999999e-06 2024-09-17T21:29:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.075008+0000", 2024-09-17T21:29:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2379999999999997e-06 2024-09-17T21:29:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.075020+0000", 2024-09-17T21:29:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1887e-05 2024-09-17T21:29:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.075028+0000", 2024-09-17T21:29:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.413e-06 2024-09-17T21:29:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.101867+0000", 2024-09-17T21:29:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026839767 2024-09-17T21:29:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.101954+0000", 2024-09-17T21:29:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6828000000000003e-05 2024-09-17T21:29:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:29:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.102323+0000", 2024-09-17T21:29:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00036884199999999998 2024-09-17T21:29:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10878 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.083517+0000", 2024-09-17T21:29:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.01580341, 2024-09-17T21:29:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021106173999999998, 2024-09-17T21:29:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10878 2024-09-17T21:29:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083517+0000", 2024-09-17T21:29:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083517+0000", 2024-09-17T21:29:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083518+0000", 2024-09-17T21:29:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.15e-07 2024-09-17T21:29:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083520+0000", 2024-09-17T21:29:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9080000000000002e-06 2024-09-17T21:29:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083525+0000", 2024-09-17T21:29:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8329999999999999e-06 2024-09-17T21:29:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083530+0000", 2024-09-17T21:29:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:29:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11011 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.090622+0000", 2024-09-17T21:29:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0086991840000000004, 2024-09-17T21:29:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014047408000000001, 2024-09-17T21:29:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11011 2024-09-17T21:29:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090622+0000", 2024-09-17T21:29:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090622+0000", 2024-09-17T21:29:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090622+0000", 2024-09-17T21:29:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4799999999999999e-07 2024-09-17T21:29:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090624+0000", 2024-09-17T21:29:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2510000000000001e-06 2024-09-17T21:29:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090631+0000", 2024-09-17T21:29:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1060000000000002e-06 2024-09-17T21:29:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090636+0000", 2024-09-17T21:29:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9350000000000002e-06 2024-09-17T21:29:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11098 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.095026+0000", 2024-09-17T21:29:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0042949390000000002, 2024-09-17T21:29:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0096926700000000005, 2024-09-17T21:29:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11098 2024-09-17T21:29:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095026+0000", 2024-09-17T21:29:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095026+0000", 2024-09-17T21:29:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095027+0000", 2024-09-17T21:29:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3399999999999997e-07 2024-09-17T21:29:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095031+0000", 2024-09-17T21:29:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3640000000000001e-06 2024-09-17T21:29:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095040+0000", 2024-09-17T21:29:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8829999999999995e-06 2024-09-17T21:29:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095056+0000", 2024-09-17T21:29:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5809000000000001e-05 2024-09-17T21:29:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11172 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.096563+0000", 2024-09-17T21:29:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.002758075, 2024-09-17T21:29:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.008200677, 2024-09-17T21:29:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11172 2024-09-17T21:29:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096563+0000", 2024-09-17T21:29:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096563+0000", 2024-09-17T21:29:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096563+0000", 2024-09-17T21:29:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999998e-07 2024-09-17T21:29:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096565+0000", 2024-09-17T21:29:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-06 2024-09-17T21:29:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096572+0000", 2024-09-17T21:29:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7449999999999998e-06 2024-09-17T21:29:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096577+0000", 2024-09-17T21:29:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1399999999999999e-06 2024-09-17T21:29:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10805 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.078811+0000", 2024-09-17T21:29:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.020509889, 2024-09-17T21:29:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025988925, 2024-09-17T21:29:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:29:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10805 2024-09-17T21:29:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.078811+0000", 2024-09-17T21:29:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.078811+0000", 2024-09-17T21:29:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.078812+0000", 2024-09-17T21:29:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0800000000000004e-07 2024-09-17T21:29:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.078816+0000", 2024-09-17T21:29:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2980000000000002e-06 2024-09-17T21:29:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.078829+0000", 2024-09-17T21:29:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2758999999999999e-05 2024-09-17T21:29:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.078838+0000", 2024-09-17T21:29:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7370000000000001e-06 2024-09-17T21:29:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.103012+0000", 2024-09-17T21:29:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024173626 2024-09-17T21:29:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.103090+0000", 2024-09-17T21:29:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8291e-05 2024-09-17T21:29:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:29:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.103444+0000", 2024-09-17T21:29:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035362600000000001 2024-09-17T21:29:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10879 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.083537+0000", 2024-09-17T21:29:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.015783538999999999, 2024-09-17T21:29:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021319853, 2024-09-17T21:29:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10879 2024-09-17T21:29:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083537+0000", 2024-09-17T21:29:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083537+0000", 2024-09-17T21:29:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083538+0000", 2024-09-17T21:29:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0600000000000001e-07 2024-09-17T21:29:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083539+0000", 2024-09-17T21:29:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8059999999999999e-06 2024-09-17T21:29:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083545+0000", 2024-09-17T21:29:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4079999999999997e-06 2024-09-17T21:29:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083550+0000", 2024-09-17T21:29:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7650000000000001e-06 2024-09-17T21:29:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11012 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.090644+0000", 2024-09-17T21:29:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0086771160000000003, 2024-09-17T21:29:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014348672999999999, 2024-09-17T21:29:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11012 2024-09-17T21:29:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090644+0000", 2024-09-17T21:29:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090644+0000", 2024-09-17T21:29:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090644+0000", 2024-09-17T21:29:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0600000000000001e-07 2024-09-17T21:29:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090652+0000", 2024-09-17T21:29:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7729999999999999e-06 2024-09-17T21:29:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090658+0000", 2024-09-17T21:29:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4579999999999998e-06 2024-09-17T21:29:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090663+0000", 2024-09-17T21:29:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9910000000000002e-06 2024-09-17T21:29:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11099 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.095072+0000", 2024-09-17T21:29:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0042490499999999999, 2024-09-17T21:29:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0099686040000000007, 2024-09-17T21:29:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11099 2024-09-17T21:29:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095072+0000", 2024-09-17T21:29:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095072+0000", 2024-09-17T21:29:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095072+0000", 2024-09-17T21:29:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4000000000000002e-07 2024-09-17T21:29:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095075+0000", 2024-09-17T21:29:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7590000000000002e-06 2024-09-17T21:29:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095081+0000", 2024-09-17T21:29:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5200000000000003e-06 2024-09-17T21:29:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095100+0000", 2024-09-17T21:29:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8561999999999999e-05 2024-09-17T21:29:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11173 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.096584+0000", 2024-09-17T21:29:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0027368269999999998, 2024-09-17T21:29:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0085070040000000003, 2024-09-17T21:29:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11173 2024-09-17T21:29:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096584+0000", 2024-09-17T21:29:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096584+0000", 2024-09-17T21:29:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096584+0000", 2024-09-17T21:29:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:29:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096586+0000", 2024-09-17T21:29:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0590000000000001e-06 2024-09-17T21:29:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096593+0000", 2024-09-17T21:29:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4459999999999998e-06 2024-09-17T21:29:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096598+0000", 2024-09-17T21:29:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.074e-06 2024-09-17T21:29:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10880 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.083556+0000", 2024-09-17T21:29:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.015765125000000001, 2024-09-17T21:29:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021560275, 2024-09-17T21:29:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10880 2024-09-17T21:29:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083556+0000", 2024-09-17T21:29:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083556+0000", 2024-09-17T21:29:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083556+0000", 2024-09-17T21:29:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:29:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083558+0000", 2024-09-17T21:29:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7540000000000001e-06 2024-09-17T21:29:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083563+0000", 2024-09-17T21:29:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:29:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083568+0000", 2024-09-17T21:29:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4549999999999997e-06 2024-09-17T21:29:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11013 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.090670+0000", 2024-09-17T21:29:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0086506269999999993, 2024-09-17T21:29:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014471734, 2024-09-17T21:29:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11013 2024-09-17T21:29:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090670+0000", 2024-09-17T21:29:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090670+0000", 2024-09-17T21:29:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090671+0000", 2024-09-17T21:29:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:29:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090673+0000", 2024-09-17T21:29:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.075e-06 2024-09-17T21:29:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090683+0000", 2024-09-17T21:29:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0489000000000001e-05 2024-09-17T21:29:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090694+0000", 2024-09-17T21:29:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.061e-05 2024-09-17T21:29:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11100 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.095111+0000", 2024-09-17T21:29:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0042100000000000002, 2024-09-17T21:29:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010054963, 2024-09-17T21:29:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11100 2024-09-17T21:29:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095111+0000", 2024-09-17T21:29:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095111+0000", 2024-09-17T21:29:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095111+0000", 2024-09-17T21:29:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6800000000000001e-07 2024-09-17T21:29:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095119+0000", 2024-09-17T21:29:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7740000000000004e-06 2024-09-17T21:29:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095131+0000", 2024-09-17T21:29:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1548e-05 2024-09-17T21:29:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095137+0000", 2024-09-17T21:29:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6189999999999996e-06 2024-09-17T21:29:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11174 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.096605+0000", 2024-09-17T21:29:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0027155980000000001, 2024-09-17T21:29:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0085848750000000005, 2024-09-17T21:29:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11174 2024-09-17T21:29:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096605+0000", 2024-09-17T21:29:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096605+0000", 2024-09-17T21:29:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096606+0000", 2024-09-17T21:29:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4799999999999999e-07 2024-09-17T21:29:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096608+0000", 2024-09-17T21:29:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1830000000000001e-06 2024-09-17T21:29:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096615+0000", 2024-09-17T21:29:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0099999999999998e-06 2024-09-17T21:29:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096620+0000", 2024-09-17T21:29:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9300000000000002e-06 2024-09-17T21:29:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10881 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.083590+0000", 2024-09-17T21:29:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.015730931, 2024-09-17T21:29:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021623546, 2024-09-17T21:29:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10881 2024-09-17T21:29:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083590+0000", 2024-09-17T21:29:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083590+0000", 2024-09-17T21:29:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083590+0000", 2024-09-17T21:29:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:29:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083592+0000", 2024-09-17T21:29:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1770000000000001e-06 2024-09-17T21:29:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083599+0000", 2024-09-17T21:29:49.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.139e-06 2024-09-17T21:29:49.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.083605+0000", 2024-09-17T21:29:49.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2029999999999997e-06 2024-09-17T21:29:49.547 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.547 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.547 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.547 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11014 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.548 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.090724+0000", 2024-09-17T21:29:49.548 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0085963849999999998, 2024-09-17T21:29:49.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014513262000000001, 2024-09-17T21:29:49.548 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.548 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.548 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.548 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.548 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.548 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.548 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11014 2024-09-17T21:29:49.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.548 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090724+0000", 2024-09-17T21:29:49.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090724+0000", 2024-09-17T21:29:49.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090725+0000", 2024-09-17T21:29:49.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:29:49.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090732+0000", 2024-09-17T21:29:49.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1960000000000002e-06 2024-09-17T21:29:49.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090739+0000", 2024-09-17T21:29:49.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2019999999999998e-06 2024-09-17T21:29:49.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.090745+0000", 2024-09-17T21:29:49.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6500000000000001e-06 2024-09-17T21:29:49.551 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.551 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.551 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.551 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11101 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.551 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.095146+0000", 2024-09-17T21:29:49.551 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0041744019999999998, 2024-09-17T21:29:49.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010118655000000001, 2024-09-17T21:29:49.551 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.551 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.551 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.552 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.552 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.552 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.552 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11101 2024-09-17T21:29:49.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.552 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095146+0000", 2024-09-17T21:29:49.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095146+0000", 2024-09-17T21:29:49.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095147+0000", 2024-09-17T21:29:49.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6499999999999999e-07 2024-09-17T21:29:49.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095149+0000", 2024-09-17T21:29:49.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3269999999999999e-06 2024-09-17T21:29:49.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095157+0000", 2024-09-17T21:29:49.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6599999999999995e-06 2024-09-17T21:29:49.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.095162+0000", 2024-09-17T21:29:49.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:29:49.554 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.554 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.554 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.555 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:11175 3.1d 3.f9a5067d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.555 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.096627+0000", 2024-09-17T21:29:49.555 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0026936680000000002, 2024-09-17T21:29:49.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0086631380000000008, 2024-09-17T21:29:49.555 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.555 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.555 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:29:49.555 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.555 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.555 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.555 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 11175 2024-09-17T21:29:49.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.556 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096627+0000", 2024-09-17T21:29:49.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096627+0000", 2024-09-17T21:29:49.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096628+0000", 2024-09-17T21:29:49.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:29:49.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096637+0000", 2024-09-17T21:29:49.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8700000000000004e-06 2024-09-17T21:29:49.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096645+0000", 2024-09-17T21:29:49.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2309999999999999e-06 2024-09-17T21:29:49.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.096650+0000", 2024-09-17T21:29:49.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4369999999999998e-06 2024-09-17T21:29:49.558 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.558 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.558 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.558 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.558 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-17T21:29:49.558 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 129 2024-09-17T21:29:49.558 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:49.559 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:29:49.561 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is unset 2024-09-17T21:29:49.561 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:29:49.561 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:49.557+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:49.660 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:49.658+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:49.757 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:49.757 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:29:49.757 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:29:49.758 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:29:49.758 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:49.761 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:49.758+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:29:49.769 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:29:49.777 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:29:49.777 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:29:49.815 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:29:49.816 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:29:49.842 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:29:49.842 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:29:49.861 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:49.859+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:49.885 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:29:49.885 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:29:49.919 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:29:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:29:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:29:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10799 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:29:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.85089314500000002, 2024-09-17T21:29:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039122745, 2024-09-17T21:29:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10799 2024-09-17T21:29:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:29:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:29:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065469+0000", 2024-09-17T21:29:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:29:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065477+0000", 2024-09-17T21:29:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1319999999999994e-06 2024-09-17T21:29:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065483+0000", 2024-09-17T21:29:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2040000000000002e-06 2024-09-17T21:29:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065487+0000", 2024-09-17T21:29:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.42e-06 2024-09-17T21:29:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099984+0000", 2024-09-17T21:29:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034496484000000001 2024-09-17T21:29:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100065+0000", 2024-09-17T21:29:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1200999999999997e-05 2024-09-17T21:29:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:29:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100422+0000", 2024-09-17T21:29:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035657700000000001 2024-09-17T21:29:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104214+0000", 2024-09-17T21:29:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003792045 2024-09-17T21:29:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104505+0000", 2024-09-17T21:29:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029141399999999999 2024-09-17T21:29:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104556+0000", 2024-09-17T21:29:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1388999999999999e-05 2024-09-17T21:29:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104591+0000", 2024-09-17T21:29:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4455999999999997e-05 2024-09-17T21:29:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12187 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:29:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77680041399999999, 2024-09-17T21:29:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038825750999999999, 2024-09-17T21:29:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12187 2024-09-17T21:29:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:29:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:29:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:29:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:29:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139563+0000", 2024-09-17T21:29:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.624e-06 2024-09-17T21:29:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139569+0000", 2024-09-17T21:29:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1090000000000003e-06 2024-09-17T21:29:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139573+0000", 2024-09-17T21:29:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4429999999999996e-06 2024-09-17T21:29:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178206+0000", 2024-09-17T21:29:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038632376000000003 2024-09-17T21:29:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178282+0000", 2024-09-17T21:29:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6076999999999997e-05 2024-09-17T21:29:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178387+0000", 2024-09-17T21:29:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104682 2024-09-17T21:29:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12188 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:29:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77678223300000004, 2024-09-17T21:29:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039053826999999999, 2024-09-17T21:29:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12188 2024-09-17T21:29:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:29:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:29:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139746+0000", 2024-09-17T21:29:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000167281 2024-09-17T21:29:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139749+0000", 2024-09-17T21:29:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9660000000000001e-06 2024-09-17T21:29:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139757+0000", 2024-09-17T21:29:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8029999999999997e-06 2024-09-17T21:29:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139771+0000", 2024-09-17T21:29:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3541e-05 2024-09-17T21:29:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178415+0000", 2024-09-17T21:29:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038644803999999998 2024-09-17T21:29:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178517+0000", 2024-09-17T21:29:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101298 2024-09-17T21:29:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178633+0000", 2024-09-17T21:29:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116134 2024-09-17T21:29:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12189 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:29:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77658175600000001, 2024-09-17T21:29:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039135991000000002, 2024-09-17T21:29:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12189 2024-09-17T21:29:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:29:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:29:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139780+0000", 2024-09-17T21:29:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:29:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139782+0000", 2024-09-17T21:29:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1950000000000002e-06 2024-09-17T21:29:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139788+0000", 2024-09-17T21:29:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0639999999999999e-06 2024-09-17T21:29:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139792+0000", 2024-09-17T21:29:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1960000000000001e-06 2024-09-17T21:29:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178671+0000", 2024-09-17T21:29:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038878176 2024-09-17T21:29:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178778+0000", 2024-09-17T21:29:49.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107278 2024-09-17T21:29:49.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:49.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178915+0000", 2024-09-17T21:29:49.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013757900000000001 2024-09-17T21:29:49.940 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.940 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.940 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.941 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12190 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.941 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:29:49.941 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77656257299999998, 2024-09-17T21:29:49.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039343781000000001, 2024-09-17T21:29:49.941 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.941 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.941 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:49.941 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.941 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.941 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.941 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12190 2024-09-17T21:29:49.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.941 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:29:49.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:29:49.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:29:49.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:29:49.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139801+0000", 2024-09-17T21:29:49.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.916e-06 2024-09-17T21:29:49.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139807+0000", 2024-09-17T21:29:49.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4540000000000004e-06 2024-09-17T21:29:49.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139812+0000", 2024-09-17T21:29:49.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9729999999999997e-06 2024-09-17T21:29:49.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178960+0000", 2024-09-17T21:29:49.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039147567000000001 2024-09-17T21:29:49.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179051+0000", 2024-09-17T21:29:49.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0754000000000006e-05 2024-09-17T21:29:49.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:49.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179142+0000", 2024-09-17T21:29:49.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1712999999999994e-05 2024-09-17T21:29:49.945 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.945 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.945 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.945 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12191 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.945 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:29:49.945 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77654271500000005, 2024-09-17T21:29:49.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039464336000000003, 2024-09-17T21:29:49.945 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.945 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.946 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:49.946 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.946 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.946 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.946 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12191 2024-09-17T21:29:49.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.946 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:29:49.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:29:49.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:29:49.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:29:49.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139821+0000", 2024-09-17T21:29:49.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1260000000000001e-06 2024-09-17T21:29:49.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139827+0000", 2024-09-17T21:29:49.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T21:29:49.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139840+0000", 2024-09-17T21:29:49.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2607e-05 2024-09-17T21:29:49.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179162+0000", 2024-09-17T21:29:49.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039322863999999999 2024-09-17T21:29:49.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179204+0000", 2024-09-17T21:29:49.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1938999999999999e-05 2024-09-17T21:29:49.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:49.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179283+0000", 2024-09-17T21:29:49.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8547000000000005e-05 2024-09-17T21:29:49.950 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.950 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.950 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.950 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12192 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.950 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:29:49.950 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77651500299999998, 2024-09-17T21:29:49.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039652318999999998, 2024-09-17T21:29:49.950 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.950 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.950 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:49.950 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.951 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.951 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.951 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12192 2024-09-17T21:29:49.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.951 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:29:49.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:29:49.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141458+0000", 2024-09-17T21:29:49.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001612098 2024-09-17T21:29:49.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141469+0000", 2024-09-17T21:29:49.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-17T21:29:49.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141478+0000", 2024-09-17T21:29:49.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T21:29:49.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141485+0000", 2024-09-17T21:29:49.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4760000000000003e-06 2024-09-17T21:29:49.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179317+0000", 2024-09-17T21:29:49.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037831832000000003 2024-09-17T21:29:49.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179395+0000", 2024-09-17T21:29:49.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8349000000000005e-05 2024-09-17T21:29:49.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:49.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179499+0000", 2024-09-17T21:29:49.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103628 2024-09-17T21:29:49.954 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.954 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.955 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12257 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.955 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:29:49.955 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77432619499999999, 2024-09-17T21:29:49.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038862751000000001, 2024-09-17T21:29:49.955 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.955 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.955 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:49.955 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.955 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.955 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.956 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12257 2024-09-17T21:29:49.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.956 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:29:49.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:29:49.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:29:49.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T21:29:49.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142037+0000", 2024-09-17T21:29:49.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.03e-06 2024-09-17T21:29:49.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142043+0000", 2024-09-17T21:29:49.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6319999999999996e-06 2024-09-17T21:29:49.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142052+0000", 2024-09-17T21:29:49.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2709999999999998e-06 2024-09-17T21:29:49.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180794+0000", 2024-09-17T21:29:49.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038741758000000001 2024-09-17T21:29:49.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180843+0000", 2024-09-17T21:29:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9021999999999999e-05 2024-09-17T21:29:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180898+0000", 2024-09-17T21:29:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673000000000003e-05 2024-09-17T21:29:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12258 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:29:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77430267799999997, 2024-09-17T21:29:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038950717000000003, 2024-09-17T21:29:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12258 2024-09-17T21:29:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:29:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:29:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:29:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199999999999999e-07 2024-09-17T21:29:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142061+0000", 2024-09-17T21:29:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8279999999999999e-06 2024-09-17T21:29:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142067+0000", 2024-09-17T21:29:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5600000000000001e-06 2024-09-17T21:29:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142071+0000", 2024-09-17T21:29:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6240000000000001e-06 2024-09-17T21:29:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180921+0000", 2024-09-17T21:29:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038849366000000003 2024-09-17T21:29:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180961+0000", 2024-09-17T21:29:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0219000000000002e-05 2024-09-17T21:29:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181009+0000", 2024-09-17T21:29:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8437999999999999e-05 2024-09-17T21:29:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12259 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:29:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77427543099999996, 2024-09-17T21:29:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039116832999999997, 2024-09-17T21:29:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12259 2024-09-17T21:29:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:29:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:29:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:29:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2200000000000004e-07 2024-09-17T21:29:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142088+0000", 2024-09-17T21:29:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0329999999999998e-06 2024-09-17T21:29:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142095+0000", 2024-09-17T21:29:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4320000000000004e-06 2024-09-17T21:29:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142100+0000", 2024-09-17T21:29:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:29:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181037+0000", 2024-09-17T21:29:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038937601000000002 2024-09-17T21:29:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181114+0000", 2024-09-17T21:29:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6857e-05 2024-09-17T21:29:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181203+0000", 2024-09-17T21:29:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8437999999999996e-05 2024-09-17T21:29:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12260 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:29:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77425514399999995, 2024-09-17T21:29:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039204184000000003, 2024-09-17T21:29:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12260 2024-09-17T21:29:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:29:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:29:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142107+0000", 2024-09-17T21:29:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:29:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142108+0000", 2024-09-17T21:29:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.753e-06 2024-09-17T21:29:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142114+0000", 2024-09-17T21:29:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7660000000000004e-06 2024-09-17T21:29:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142119+0000", 2024-09-17T21:29:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:29:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181216+0000", 2024-09-17T21:29:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039097191000000003 2024-09-17T21:29:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181260+0000", 2024-09-17T21:29:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4731000000000003e-05 2024-09-17T21:29:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181310+0000", 2024-09-17T21:29:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9818000000000003e-05 2024-09-17T21:29:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12261 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:29:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77423682000000005, 2024-09-17T21:29:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039292200999999999, 2024-09-17T21:29:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12261 2024-09-17T21:29:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:29:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:29:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142125+0000", 2024-09-17T21:29:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:29:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142127+0000", 2024-09-17T21:29:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.792e-06 2024-09-17T21:29:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142132+0000", 2024-09-17T21:29:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3979999999999998e-06 2024-09-17T21:29:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142136+0000", 2024-09-17T21:29:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.211e-06 2024-09-17T21:29:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181324+0000", 2024-09-17T21:29:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039187560000000003 2024-09-17T21:29:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181363+0000", 2024-09-17T21:29:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9375000000000002e-05 2024-09-17T21:29:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181417+0000", 2024-09-17T21:29:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3372000000000003e-05 2024-09-17T21:29:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12262 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:29:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.774219027, 2024-09-17T21:29:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039384875, 2024-09-17T21:29:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12262 2024-09-17T21:29:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:29:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:29:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142143+0000", 2024-09-17T21:29:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:29:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142148+0000", 2024-09-17T21:29:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4360000000000001e-06 2024-09-17T21:29:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142154+0000", 2024-09-17T21:29:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8719999999999999e-06 2024-09-17T21:29:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142158+0000", 2024-09-17T21:29:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.476e-06 2024-09-17T21:29:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181436+0000", 2024-09-17T21:29:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039277919000000001 2024-09-17T21:29:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181480+0000", 2024-09-17T21:29:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3649999999999997e-05 2024-09-17T21:29:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181527+0000", 2024-09-17T21:29:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7131e-05 2024-09-17T21:29:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12263 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1079296~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:29:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77419659200000002, 2024-09-17T21:29:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039473339000000003, 2024-09-17T21:29:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12263 2024-09-17T21:29:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:29:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:29:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:29:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:29:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142167+0000", 2024-09-17T21:29:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.81e-06 2024-09-17T21:29:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142172+0000", 2024-09-17T21:29:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:29:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142177+0000", 2024-09-17T21:29:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T21:29:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181553+0000", 2024-09-17T21:29:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039376130000000002 2024-09-17T21:29:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181592+0000", 2024-09-17T21:29:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9153999999999998e-05 2024-09-17T21:29:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181638+0000", 2024-09-17T21:29:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5864e-05 2024-09-17T21:29:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12278 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1095680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:29:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77417861700000001, 2024-09-17T21:29:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039454222999999997, 2024-09-17T21:29:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12278 2024-09-17T21:29:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:29:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:29:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:29:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:29:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142185+0000", 2024-09-17T21:29:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.697e-06 2024-09-17T21:29:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142191+0000", 2024-09-17T21:29:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:29:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142196+0000", 2024-09-17T21:29:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4829999999999999e-06 2024-09-17T21:29:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181548+0000", 2024-09-17T21:29:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039351200000000003 2024-09-17T21:29:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181591+0000", 2024-09-17T21:29:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3492000000000003e-05 2024-09-17T21:29:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181637+0000", 2024-09-17T21:29:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5847000000000003e-05 2024-09-17T21:29:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12279 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:29:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77415532799999998, 2024-09-17T21:29:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039539801999999999, 2024-09-17T21:29:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12279 2024-09-17T21:29:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:29:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:29:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:29:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.89e-07 2024-09-17T21:29:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142208+0000", 2024-09-17T21:29:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7579999999999999e-06 2024-09-17T21:29:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142214+0000", 2024-09-17T21:29:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:29:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142218+0000", 2024-09-17T21:29:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3429999999999998e-06 2024-09-17T21:29:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181653+0000", 2024-09-17T21:29:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039435322000000002 2024-09-17T21:29:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181698+0000", 2024-09-17T21:29:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4978999999999997e-05 2024-09-17T21:29:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181746+0000", 2024-09-17T21:29:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571999999999998e-05 2024-09-17T21:29:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12280 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1124352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:29:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77413780899999995, 2024-09-17T21:29:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039704358000000002, 2024-09-17T21:29:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12280 2024-09-17T21:29:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:29:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:29:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142224+0000", 2024-09-17T21:29:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:29:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142226+0000", 2024-09-17T21:29:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:29:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142231+0000", 2024-09-17T21:29:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:29:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142240+0000", 2024-09-17T21:29:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4590000000000004e-06 2024-09-17T21:29:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181774+0000", 2024-09-17T21:29:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039533833999999997 2024-09-17T21:29:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181859+0000", 2024-09-17T21:29:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4328999999999998e-05 2024-09-17T21:29:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181928+0000", 2024-09-17T21:29:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9139000000000003e-05 2024-09-17T21:29:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12281 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1136640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:29:50.002 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77411490299999997, 2024-09-17T21:29:50.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039787799999999998, 2024-09-17T21:29:50.002 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:50.002 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:50.002 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:50.002 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:50.002 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:50.002 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:50.002 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12281 2024-09-17T21:29:50.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.002 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:50.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:50.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:29:50.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:50.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:50.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:29:50.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:50.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:50.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142309+0000", 2024-09-17T21:29:50.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2323999999999995e-05 2024-09-17T21:29:50.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:50.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142317+0000", 2024-09-17T21:29:50.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1859999999999992e-06 2024-09-17T21:29:50.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:50.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142324+0000", 2024-09-17T21:29:50.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4699999999999996e-06 2024-09-17T21:29:50.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:50.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142330+0000", 2024-09-17T21:29:50.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4650000000000002e-06 2024-09-17T21:29:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181948+0000", 2024-09-17T21:29:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039618056999999998 2024-09-17T21:29:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181986+0000", 2024-09-17T21:29:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8442999999999998e-05 2024-09-17T21:29:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182034+0000", 2024-09-17T21:29:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7855000000000002e-05 2024-09-17T21:29:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12282 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1150976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:29:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77402267400000002, 2024-09-17T21:29:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039871919999999998, 2024-09-17T21:29:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12282 2024-09-17T21:29:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:29:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:29:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:29:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:29:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142341+0000", 2024-09-17T21:29:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.232e-06 2024-09-17T21:29:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142348+0000", 2024-09-17T21:29:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7320000000000001e-06 2024-09-17T21:29:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142353+0000", 2024-09-17T21:29:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:29:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182062+0000", 2024-09-17T21:29:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039709739000000001 2024-09-17T21:29:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182128+0000", 2024-09-17T21:29:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5739000000000002e-05 2024-09-17T21:29:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182210+0000", 2024-09-17T21:29:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:29:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12283 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:29:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77400226299999997, 2024-09-17T21:29:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039896434000000001, 2024-09-17T21:29:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:29:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12283 2024-09-17T21:29:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:29:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:29:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144852+0000", 2024-09-17T21:29:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024934079999999999 2024-09-17T21:29:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144856+0000", 2024-09-17T21:29:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:29:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144864+0000", 2024-09-17T21:29:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0160000000000008e-06 2024-09-17T21:29:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144869+0000", 2024-09-17T21:29:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:29:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182246+0000", 2024-09-17T21:29:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037377207000000003 2024-09-17T21:29:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182255+0000", 2024-09-17T21:29:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1840000000000005e-06 2024-09-17T21:29:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:50.015 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:50.016 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:29:50.016 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:29:50.017 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:29:50.017 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:29:50.018 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:50.014+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:50.018 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:29:50.018 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:29:50.018 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:29:50.019 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:29:50.019 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:29:50.117 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:50.114+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:50.172 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:50.172 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:50.180 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:29:50.224 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:50.222+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:50.284 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:50.284 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:50.293 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:29:50.315 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-17T21:29:50.334 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:50.332+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:50.419 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:50.417+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:50.420 INFO:teuthology.orchestra.run.smithi129.stdout:292 write 0xed19c87 thru 0xed1bc60 (0x1fda bytes) 2024-09-17T21:29:50.423 INFO:teuthology.orchestra.run.smithi129.stdout:295 writesame 0x2af800d thru 0x2afd618 data_size 0x2(0x560c bytes) 2024-09-17T21:29:50.432 INFO:teuthology.orchestra.run.smithi129.stdout:296 read 0xd30b2f1 thru 0xd31a656 (0xf366 bytes) 2024-09-17T21:29:50.432 INFO:teuthology.orchestra.run.smithi129.stdout:298 write 0x17bab68 thru 0x17c3a10 (0x8ea9 bytes) 2024-09-17T21:29:50.446 INFO:teuthology.orchestra.run.smithi129.stdout:299 write 0x8edd2cf thru 0x8eeb891 (0xe5c3 bytes) 2024-09-17T21:29:50.446 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:50.446 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:50.453 INFO:teuthology.orchestra.run.smithi129.stdout:300 writesame 0xe9d7e8a thru 0xe9dfff4 data_size 0x1(0x816b bytes) 2024-09-17T21:29:50.456 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:29:50.471 INFO:teuthology.orchestra.run.smithi129.stdout:302 writesame 0xae068b3 thru 0xae0a954 data_size 0x1(0x40a2 bytes) 2024-09-17T21:29:50.480 INFO:teuthology.orchestra.run.smithi129.stdout:303 write 0x29d8dc2 thru 0x29e772d (0xe96c bytes) 2024-09-17T21:29:50.494 INFO:teuthology.orchestra.run.smithi129.stdout:304 read 0x8a377c1 thru 0x8a3f69b (0x7edb bytes) 2024-09-17T21:29:50.509 INFO:teuthology.orchestra.run.smithi129.stdout:305 trunc from 0xed1bc61 to 0x3084fc9 2024-09-17T21:29:50.520 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:50.518+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:50.532 INFO:teuthology.orchestra.run.smithi129.stdout:306 read 0x1ae2d62 thru 0x1ae8c54 (0x5ef3 bytes) 2024-09-17T21:29:50.533 INFO:teuthology.orchestra.run.smithi129.stdout:307 read 0x2192d3e thru 0x2197462 (0x4725 bytes) 2024-09-17T21:29:50.534 INFO:teuthology.orchestra.run.smithi129.stdout:308 trunc from 0x3084fc9 to 0x2685a0d 2024-09-17T21:29:50.570 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:50.571 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:50.574 INFO:teuthology.orchestra.run.smithi129.stdout:310 trunc from 0x2685a0d to 0xb235dfd 2024-09-17T21:29:50.580 INFO:teuthology.orchestra.run.smithi129.stdout:311 read 0x3a3bf23 thru 0x3a49a33 (0xdb11 bytes) 2024-09-17T21:29:50.581 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:50.581 INFO:teuthology.orchestra.run.smithi129.stdout:313 read 0x889048b thru 0x889f93f (0xf4b5 bytes) 2024-09-17T21:29:50.581 INFO:teuthology.orchestra.run.smithi129.stdout:314 read 0x3647e9b thru 0x364a093 (0x21f9 bytes) 2024-09-17T21:29:50.581 INFO:teuthology.orchestra.run.smithi129.stdout:316 read 0x83729b9 thru 0x8378c10 (0x6258 bytes) 2024-09-17T21:29:50.581 INFO:teuthology.orchestra.run.smithi129.stdout:317 read 0x2b40934 thru 0x2b4eaa6 (0xe173 bytes) 2024-09-17T21:29:50.581 INFO:teuthology.orchestra.run.smithi129.stdout:318 write 0x977e289 thru 0x978c5e1 (0xe359 bytes) 2024-09-17T21:29:50.581 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:29:50.587 INFO:teuthology.orchestra.run.smithi129.stdout:320 punch from 0x7d030f2 to 0x7d106cb, (0xd5d9 bytes) 2024-09-17T21:29:50.587 INFO:teuthology.orchestra.run.smithi129.stdout:322 read 0x14e9808 thru 0x14f1ddc (0x85d5 bytes) 2024-09-17T21:29:50.587 INFO:teuthology.orchestra.run.smithi129.stdout:323 read 0x8dce3bc thru 0x8dd25f8 (0x423d bytes) 2024-09-17T21:29:50.588 INFO:teuthology.orchestra.run.smithi129.stdout:324 write 0x4261bd2 thru 0x4270efb (0xf32a bytes) 2024-09-17T21:29:50.598 INFO:teuthology.orchestra.run.smithi129.stdout:325 writesame 0xce8bb6c thru 0xce9529a data_size 0x61(0x972f bytes) 2024-09-17T21:29:50.604 INFO:teuthology.orchestra.run.smithi129.stdout:327 trunc from 0xce9529b to 0x7d163f7 2024-09-17T21:29:50.618 INFO:teuthology.orchestra.run.smithi129.stdout:331 writesame 0x82b86e9 thru 0x82ba8a1 data_size 0x1(0x21b9 bytes) 2024-09-17T21:29:50.618 INFO:teuthology.orchestra.run.smithi129.stdout:332 write 0x4f4e490 thru 0x4f57f11 (0x9a82 bytes) 2024-09-17T21:29:50.620 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:50.618+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:50.621 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:50.627 INFO:teuthology.orchestra.run.smithi129.stdout:334 write 0x88fa4ed thru 0x8903411 (0x8f25 bytes) 2024-09-17T21:29:50.631 INFO:teuthology.orchestra.run.smithi129.stdout:335 writesame 0x17d3ca0 thru 0x17d6151 data_size 0x16(0x24b2 bytes) 2024-09-17T21:29:50.632 INFO:teuthology.orchestra.run.smithi129.stdout:336 read 0x6a034a2 thru 0x6a0f51f (0xc07e bytes) 2024-09-17T21:29:50.633 INFO:teuthology.orchestra.run.smithi129.stdout:338 punch from 0x33c3450 to 0x33cc182, (0x8d32 bytes) 2024-09-17T21:29:50.633 INFO:teuthology.orchestra.run.smithi129.stdout:339 writesame 0x7c5efeb thru 0x7c65c16 data_size 0x2e(0x6c2c bytes) 2024-09-17T21:29:50.646 INFO:teuthology.orchestra.run.smithi129.stdout:341 write 0xd3a6503 thru 0xd3aaf15 (0x4a13 bytes) 2024-09-17T21:29:50.649 INFO:teuthology.orchestra.run.smithi129.stdout:347 read 0x56a170d thru 0x56a7092 (0x5986 bytes) 2024-09-17T21:29:50.649 INFO:teuthology.orchestra.run.smithi129.stdout:348 read 0xbcc4511 thru 0xbcd2381 (0xde71 bytes) 2024-09-17T21:29:50.649 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:50.649 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:50.665 INFO:teuthology.orchestra.run.smithi129.stdout:351 trunc from 0xd3aaf16 to 0x30021f 2024-09-17T21:29:50.696 INFO:teuthology.orchestra.run.smithi129.stdout:352 read 0x23e8bf thru 0x24a0c5 (0xb807 bytes) 2024-09-17T21:29:50.696 INFO:teuthology.orchestra.run.smithi129.stdout:353 punch from 0x2d1950 to 0x2db46b, (0x9b1b bytes) 2024-09-17T21:29:50.706 INFO:teuthology.orchestra.run.smithi129.stdout:354 write 0x36c9a6e thru 0x36cf1d9 (0x576c bytes) 2024-09-17T21:29:50.709 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:50.709 INFO:teuthology.orchestra.run.smithi129.stdout:359 write 0x88b6a2 thru 0x891c9e (0x65fd bytes) 2024-09-17T21:29:50.710 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:50.710 INFO:teuthology.orchestra.run.smithi129.stdout:361 read 0x22d1b34 thru 0x22dcd8e (0xb25b bytes) 2024-09-17T21:29:50.711 INFO:teuthology.orchestra.run.smithi129.stdout:365 punch from 0x29cac00 to 0x29cdbb9, (0x2fb9 bytes) 2024-09-17T21:29:50.711 INFO:teuthology.orchestra.run.smithi129.stdout:366 read 0x2a50388 thru 0x2a50b4b (0x7c4 bytes) 2024-09-17T21:29:50.711 INFO:teuthology.orchestra.run.smithi129.stdout:368 read 0x2b0e78c thru 0x2b1abb8 (0xc42d bytes) 2024-09-17T21:29:50.711 INFO:teuthology.orchestra.run.smithi129.stdout:369 read 0xd5ecb thru 0xe04a3 (0xa5d9 bytes) 2024-09-17T21:29:50.714 INFO:teuthology.orchestra.run.smithi129.stdout:370 trunc from 0x36cf1da to 0x58f9ed 2024-09-17T21:29:50.721 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:50.719+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:50.726 INFO:teuthology.orchestra.run.smithi129.stdout:371 read 0x55bb07 thru 0x562399 (0x6893 bytes) 2024-09-17T21:29:50.734 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:50.735 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:50.743 INFO:teuthology.orchestra.run.smithi129.stdout:372 trunc from 0x58f9ed to 0xe13167c 2024-09-17T21:29:50.743 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:29:50.751 INFO:teuthology.orchestra.run.smithi129.stdout:375 writesame 0x1b77d14 thru 0x1b7877a data_size 0x1(0xa67 bytes) 2024-09-17T21:29:50.754 INFO:teuthology.orchestra.run.smithi129.stdout:376 punch from 0x30fda5 to 0x31409a, (0x42f5 bytes) 2024-09-17T21:29:50.755 INFO:teuthology.orchestra.run.smithi129.stdout:377 read 0xd4e7ba8 thru 0xd4f73e6 (0xf83f bytes) 2024-09-17T21:29:50.758 INFO:teuthology.orchestra.run.smithi129.stdout:378 trunc from 0xe13167c to 0xa9faa14 2024-09-17T21:29:50.765 INFO:teuthology.orchestra.run.smithi129.stdout:379 write 0x16a12bf thru 0x16ab1a9 (0x9eeb bytes) 2024-09-17T21:29:50.766 INFO:teuthology.orchestra.run.smithi129.stdout:383 read 0x6207666 thru 0x620ad4b (0x36e6 bytes) 2024-09-17T21:29:50.767 INFO:teuthology.orchestra.run.smithi129.stdout:385 writesame 0x460c2e5 thru 0x4612255 data_size 0x1(0x5f71 bytes) 2024-09-17T21:29:50.778 INFO:teuthology.orchestra.run.smithi129.stdout:386 read 0x286a36a thru 0x286d5df (0x3276 bytes) 2024-09-17T21:29:50.778 INFO:teuthology.orchestra.run.smithi129.stdout:387 read 0xb422de thru 0xb4b2cb (0x8fee bytes) 2024-09-17T21:29:50.778 INFO:teuthology.orchestra.run.smithi129.stdout:388 write 0x9a603a thru 0x9b5c5d (0xfc24 bytes) 2024-09-17T21:29:50.780 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:50.782 INFO:teuthology.orchestra.run.smithi129.stdout:390 trunc from 0xa9faa14 to 0x8ecf619 2024-09-17T21:29:50.789 INFO:teuthology.orchestra.run.smithi129.stdout:391 read 0x819368a thru 0x819e5e5 (0xaf5c bytes) 2024-09-17T21:29:50.789 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:50.789 INFO:teuthology.orchestra.run.smithi129.stdout:394 read 0x45522b9 thru 0x455317f (0xec7 bytes) 2024-09-17T21:29:50.790 INFO:teuthology.orchestra.run.smithi129.stdout:396 writesame 0x529d3f8 thru 0x52a8523 data_size 0x5c(0xb12c bytes) 2024-09-17T21:29:50.794 INFO:teuthology.orchestra.run.smithi129.stdout:397 write 0x848bbaf thru 0x848c7df (0xc31 bytes) 2024-09-17T21:29:50.799 INFO:teuthology.orchestra.run.smithi129.stdout:398 trunc from 0x8ecf619 to 0xb8d30de 2024-09-17T21:29:50.806 INFO:teuthology.orchestra.run.smithi129.stdout:400 write 0x9e2d82d thru 0x9e2f565 (0x1d39 bytes) 2024-09-17T21:29:50.809 INFO:teuthology.orchestra.run.smithi129.stdout:402 read 0xa54fcce thru 0xa5523e4 (0x2717 bytes) 2024-09-17T21:29:50.809 INFO:teuthology.orchestra.run.smithi129.stdout:403 write 0x6d76575 thru 0x6d794f8 (0x2f84 bytes) 2024-09-17T21:29:50.812 INFO:teuthology.orchestra.run.smithi129.stdout:404 read 0x9770f55 thru 0x977bddd (0xae89 bytes) 2024-09-17T21:29:50.813 INFO:teuthology.orchestra.run.smithi129.stdout:405 writesame 0x253c448 thru 0x254940b data_size 0x2(0xcfc4 bytes) 2024-09-17T21:29:50.823 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:50.821+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:50.823 INFO:teuthology.orchestra.run.smithi129.stdout:407 read 0x7da9023 thru 0x7db691d (0xd8fb bytes) 2024-09-17T21:29:50.823 INFO:teuthology.orchestra.run.smithi129.stdout:408 read 0x4d60048 thru 0x4d6f6ee (0xf6a7 bytes) 2024-09-17T21:29:50.824 INFO:teuthology.orchestra.run.smithi129.stdout:409 read 0x304f15c thru 0x30576a6 (0x854b bytes) 2024-09-17T21:29:50.824 INFO:teuthology.orchestra.run.smithi129.stdout:411 write 0x487c357 thru 0x4882385 (0x602f bytes) 2024-09-17T21:29:50.834 INFO:teuthology.orchestra.run.smithi129.stdout:413 write 0x9cdcd7c thru 0x9ce721e (0xa4a3 bytes) 2024-09-17T21:29:50.839 INFO:teuthology.orchestra.run.smithi129.stdout:414 read 0x6257ce3 thru 0x6264502 (0xc820 bytes) 2024-09-17T21:29:50.839 INFO:teuthology.orchestra.run.smithi129.stdout:415 read 0xa1b15da thru 0xa1bd815 (0xc23c bytes) 2024-09-17T21:29:50.839 INFO:teuthology.orchestra.run.smithi129.stdout:416 read 0x6130ed3 thru 0x613c409 (0xb537 bytes) 2024-09-17T21:29:50.839 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:50.839 INFO:teuthology.orchestra.run.smithi129.stdout:418 punch from 0x2957c80 to 0x2961320, (0x96a0 bytes) 2024-09-17T21:29:50.849 INFO:teuthology.orchestra.run.smithi129.stdout:419 trunc from 0xb8d30de to 0x2e510e8 2024-09-17T21:29:50.855 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:50.855 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:50.865 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:29:50.873 INFO:teuthology.orchestra.run.smithi129.stdout:420 trunc from 0x2e510e8 to 0x5ae08fb 2024-09-17T21:29:50.879 INFO:teuthology.orchestra.run.smithi129.stdout:423 read 0x1f09e5 thru 0x1fa641 (0x9c5d bytes) 2024-09-17T21:29:50.895 INFO:teuthology.orchestra.run.smithi129.stdout:424 write 0xdd0aa19 thru 0xdd12715 (0x7cfd bytes) 2024-09-17T21:29:50.898 INFO:teuthology.orchestra.run.smithi129.stdout:426 writesame 0xaf4e8c2 thru 0xaf52005 data_size 0x24(0x3744 bytes) 2024-09-17T21:29:50.901 INFO:teuthology.orchestra.run.smithi129.stdout:427 read 0xbe22e3e thru 0xbe29465 (0x6628 bytes) 2024-09-17T21:29:50.901 INFO:teuthology.orchestra.run.smithi129.stdout:428 read 0xee3af0 thru 0xeefacf (0xbfe0 bytes) 2024-09-17T21:29:50.902 INFO:teuthology.orchestra.run.smithi129.stdout:432 read 0xbfdba7 thru 0xc00d5d (0x31b7 bytes) 2024-09-17T21:29:50.902 INFO:teuthology.orchestra.run.smithi129.stdout:433 write 0x2897b73 thru 0x2898e07 (0x1295 bytes) 2024-09-17T21:29:50.907 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:50.907 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:50.907 INFO:teuthology.orchestra.run.smithi129.stdout:437 read 0x79e6434 thru 0x79f2823 (0xc3f0 bytes) 2024-09-17T21:29:50.907 INFO:teuthology.orchestra.run.smithi129.stdout:439 read 0x32e58d7 thru 0x32efa2c (0xa156 bytes) 2024-09-17T21:29:50.907 INFO:teuthology.orchestra.run.smithi129.stdout:442 writesame 0x7756850 thru 0x775b8c1 data_size 0x7(0x5072 bytes) 2024-09-17T21:29:50.916 INFO:teuthology.orchestra.run.smithi129.stdout:443 write 0xe2edd84 thru 0xe2f65c6 (0x8843 bytes) 2024-09-17T21:29:50.919 INFO:teuthology.orchestra.run.smithi129.stdout:444 write 0x501ff0d thru 0x5028bb7 (0x8cab bytes) 2024-09-17T21:29:50.923 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:50.921+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:29:50.927 INFO:teuthology.orchestra.run.smithi129.stdout:445 trunc from 0xe2f65c7 to 0xc5ef149 2024-09-17T21:29:50.944 INFO:teuthology.orchestra.run.smithi129.stdout:446 write 0xd9ffbcc thru 0xda01138 (0x156d bytes) 2024-09-17T21:29:50.950 INFO:teuthology.orchestra.run.smithi129.stdout:447 read 0xc4f5d12 thru 0xc4fa20b (0x44fa bytes) 2024-09-17T21:29:50.950 INFO:teuthology.orchestra.run.smithi129.stdout:448 read 0x7ec13c6 thru 0x7ec8314 (0x6f4f bytes) 2024-09-17T21:29:50.950 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:50.950 INFO:teuthology.orchestra.run.smithi129.stdout:450 write 0xb057579 thru 0xb06618c (0xec14 bytes) 2024-09-17T21:29:50.956 INFO:teuthology.orchestra.run.smithi129.stdout:451 trunc from 0xda01139 to 0xe14e5ab 2024-09-17T21:29:50.963 INFO:teuthology.orchestra.run.smithi129.stdout:452 write 0x3a7e75c thru 0x3a85721 (0x6fc6 bytes) 2024-09-17T21:29:50.968 INFO:teuthology.orchestra.run.smithi129.stdout:453 punch from 0x1d5c816 to 0x1d683f4, (0xbbde bytes) 2024-09-17T21:29:50.968 INFO:teuthology.orchestra.run.smithi129.stdout:454 write 0xaf15e39 thru 0xaf2426c (0xe434 bytes) 2024-09-17T21:29:50.972 INFO:teuthology.orchestra.run.smithi129.stdout:455 write 0x3879028 thru 0x38805a2 (0x757b bytes) 2024-09-17T21:29:50.984 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:50.984 INFO:teuthology.orchestra.run.smithi129.stdout:459 write 0xd658a70 thru 0xd659609 (0xb9a bytes) 2024-09-17T21:29:50.986 INFO:teuthology.orchestra.run.smithi129.stdout:460 read 0xa3dfb04 thru 0xa3ee065 (0xe562 bytes) 2024-09-17T21:29:50.987 INFO:teuthology.orchestra.run.smithi129.stdout:461 write 0x5777f84 thru 0x577f17f (0x71fc bytes) 2024-09-17T21:29:50.988 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:50.988 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:50.992 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:50.992 INFO:teuthology.orchestra.run.smithi129.stdout:463 read 0xbde912d thru 0xbde98b9 (0x78d bytes) 2024-09-17T21:29:50.992 INFO:teuthology.orchestra.run.smithi129.stdout:464 read 0x67be186 thru 0x67c685a (0x86d5 bytes) 2024-09-17T21:29:50.993 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:50.993 INFO:teuthology.orchestra.run.smithi129.stdout:467 writesame 0x1722e59 thru 0x1723860 data_size 0xc(0xa08 bytes) 2024-09-17T21:29:50.994 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:50.994 INFO:teuthology.orchestra.run.smithi129.stdout:469 read 0x94b47d thru 0x95aead (0xfa31 bytes) 2024-09-17T21:29:50.994 INFO:teuthology.orchestra.run.smithi129.stdout:470 writesame 0x30bf80 thru 0x30fa3a data_size 0x5(0x3abb bytes) 2024-09-17T21:29:50.996 INFO:teuthology.orchestra.run.smithi129.stdout:471 writesame 0x14eb3d1 thru 0x14f1340 data_size 0x18(0x5f70 bytes) 2024-09-17T21:29:50.998 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:29:50.998 INFO:teuthology.orchestra.run.smithi129.stdout:472 punch from 0x46bec7b to 0x46c4132, (0x54b7 bytes) 2024-09-17T21:29:50.998 INFO:teuthology.orchestra.run.smithi129.stdout:473 read 0x2e39911 thru 0x2e3a0d1 (0x7c1 bytes) 2024-09-17T21:29:50.998 INFO:teuthology.orchestra.run.smithi129.stdout:474 writesame 0x7711b20 thru 0x771abf1 data_size 0x25(0x90d2 bytes) 2024-09-17T21:29:51.001 INFO:teuthology.orchestra.run.smithi129.stdout:475 writesame 0x85cdf68 thru 0x85dc809 data_size 0xb(0xe8a2 bytes) 2024-09-17T21:29:51.005 INFO:teuthology.orchestra.run.smithi129.stdout:476 write 0x3903113 thru 0x390b03c (0x7f2a bytes) 2024-09-17T21:29:51.010 INFO:teuthology.orchestra.run.smithi129.stdout:478 trunc from 0xe14e5ab to 0xb0e2b84 2024-09-17T21:29:51.024 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:51.022+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:51.029 INFO:teuthology.orchestra.run.smithi129.stdout:480 writesame 0x6e6da79 thru 0x6e6edda data_size 0x3(0x1362 bytes) 2024-09-17T21:29:51.032 INFO:teuthology.orchestra.run.smithi129.stdout:481 writesame 0x8ebcc7f thru 0x8ec4c99 data_size 0x7(0x801b bytes) 2024-09-17T21:29:51.036 INFO:teuthology.orchestra.run.smithi129.stdout:482 writesame 0x1abde8b thru 0x1ac06d9 data_size 0x1(0x284f bytes) 2024-09-17T21:29:51.040 INFO:teuthology.orchestra.run.smithi129.stdout:484 clone 4 order 20 su 131072 sc 5 2024-09-17T21:29:51.106 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:51.106 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:51.115 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:29:51.125 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:51.123+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:51.226 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:51.224+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:51.257 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:51.258 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:51.266 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:29:51.350 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:51.348+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:51.409 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:51.410 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:51.418 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:29:51.460 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:51.457+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:51.528 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:51.526+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:51.562 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:29:51.569 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:51.569 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:51.575 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone3 from 0xb0e2b84 (overlap 0x30021f) to 0x0 2024-09-17T21:29:51.579 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:29:51.629 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:51.627+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:51.633 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-17T21:29:51.730 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:51.727+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:51.747 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:51.747 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:51.757 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:29:51.841 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:51.839+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:29:51.902 INFO:teuthology.orchestra.run.smithi129.stdout:485 punch from 0x3ccca96 to 0x3cd6532, (0x9a9c bytes) 2024-09-17T21:29:51.905 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:51.905 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:51.919 INFO:teuthology.orchestra.run.smithi129.stdout:487 trunc from 0xb0e2b84 to 0xdfc01 2024-09-17T21:29:51.922 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:29:51.948 INFO:teuthology.orchestra.run.smithi129.stdout:489 trunc from 0xdfc01 to 0x9bb4d20 2024-09-17T21:29:51.949 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:51.946+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:51.955 INFO:teuthology.orchestra.run.smithi129.stdout:491 punch from 0x998e3da to 0x9998ce8, (0xa90e bytes) 2024-09-17T21:29:51.955 INFO:teuthology.orchestra.run.smithi129.stdout:493 read 0x8523665 thru 0x852e401 (0xad9d bytes) 2024-09-17T21:29:51.955 INFO:teuthology.orchestra.run.smithi129.stdout:494 write 0x1b668de thru 0x1b6e9b4 (0x80d7 bytes) 2024-09-17T21:29:51.958 INFO:teuthology.orchestra.run.smithi129.stdout:495 read 0x91e55a9 thru 0x91e7020 (0x1a78 bytes) 2024-09-17T21:29:51.958 INFO:teuthology.orchestra.run.smithi129.stdout:497 writesame 0x5ca8fab thru 0x5cb16da data_size 0x70(0x8730 bytes) 2024-09-17T21:29:51.961 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:51.971 INFO:teuthology.orchestra.run.smithi129.stdout:501 write 0xcde2758 thru 0xcdebe5d (0x9706 bytes) 2024-09-17T21:29:51.976 INFO:teuthology.orchestra.run.smithi129.stdout:502 punch from 0xca517fb to 0xca5b46f, (0x9c74 bytes) 2024-09-17T21:29:51.977 INFO:teuthology.orchestra.run.smithi129.stdout:503 write 0x3c53107 thru 0x3c5c2a7 (0x91a1 bytes) 2024-09-17T21:29:51.982 INFO:teuthology.orchestra.run.smithi129.stdout:504 write 0x6a99a2f thru 0x6a9a7d2 (0xda4 bytes) 2024-09-17T21:29:51.999 INFO:teuthology.orchestra.run.smithi129.stdout:505 trunc from 0xcdebe5e to 0xe99978 2024-09-17T21:29:52.022 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:29:52.023 INFO:teuthology.orchestra.run.smithi129.stdout:506 write 0x7d5f303 thru 0x7d608a8 (0x15a6 bytes) 2024-09-17T21:29:52.030 INFO:teuthology.orchestra.run.smithi129.stdout:509 punch from 0x7819966 to 0x7825b27, (0xc1c1 bytes) 2024-09-17T21:29:52.030 INFO:teuthology.orchestra.run.smithi129.stdout:511 read 0x33c35ca thru 0x33cd6c0 (0xa0f7 bytes) 2024-09-17T21:29:52.030 INFO:teuthology.orchestra.run.smithi129.stdout:512 read 0x3d54d7a thru 0x3d5d292 (0x8519 bytes) 2024-09-17T21:29:52.030 INFO:teuthology.orchestra.run.smithi129.stdout:513 read 0x5209749 thru 0x520bff4 (0x28ac bytes) 2024-09-17T21:29:52.030 INFO:teuthology.orchestra.run.smithi129.stdout:514 read 0xc9a224 thru 0xca50ca (0xaea7 bytes) 2024-09-17T21:29:52.032 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:52.030+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:52.035 INFO:teuthology.orchestra.run.smithi129.stdout:515 write 0x87e408f thru 0x87e4fff (0xf71 bytes) 2024-09-17T21:29:52.038 INFO:teuthology.orchestra.run.smithi129.stdout:516 read 0xe6d650 thru 0xe78481 (0xae32 bytes) 2024-09-17T21:29:52.049 INFO:teuthology.orchestra.run.smithi129.stdout:517 write 0xcfd7511 thru 0xcfe016b (0x8c5b bytes) 2024-09-17T21:29:52.053 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.053 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.053 INFO:teuthology.orchestra.run.smithi129.stdout:521 read 0x3c44744 thru 0x3c44fec (0x8a9 bytes) 2024-09-17T21:29:52.053 INFO:teuthology.orchestra.run.smithi129.stdout:522 read 0x80be4b3 thru 0x80c0290 (0x1dde bytes) 2024-09-17T21:29:52.053 INFO:teuthology.orchestra.run.smithi129.stdout:523 read 0x281895 thru 0x28b7fc (0x9f68 bytes) 2024-09-17T21:29:52.054 INFO:teuthology.orchestra.run.smithi129.stdout:524 trunc from 0xcfe016c to 0xc62eb8c 2024-09-17T21:29:52.064 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:52.064 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:52.067 INFO:teuthology.orchestra.run.smithi129.stdout:525 write 0xb9f638a thru 0xb9fdc8c (0x7903 bytes) 2024-09-17T21:29:52.070 INFO:teuthology.orchestra.run.smithi129.stdout:526 punch from 0x234d46a to 0x234ea9b, (0x1631 bytes) 2024-09-17T21:29:52.070 INFO:teuthology.orchestra.run.smithi129.stdout:527 punch from 0x20aacd8 to 0x20b1208, (0x6530 bytes) 2024-09-17T21:29:52.070 INFO:teuthology.orchestra.run.smithi129.stdout:528 clone 5 order 23 su 2097152 sc 7 2024-09-17T21:29:52.074 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:29:52.133 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:52.131+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:52.148 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:29:52.149 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:29:52.166 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:29:52.166 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:29:52.179 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T21:29:52.179 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:29:52.179 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T21:29:52.180 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T21:29:52.183 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:29:52.183 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:29:52.200 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:29:52.200 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:29:52.217 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:29:52.217 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:29:52.224 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:29:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:29:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:29:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4269.0:1 2.3 e21/20 2:c4c92e5a:::rbd_trash:head v 21'1, mlcod=0'0)", 2024-09-17T21:29:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:32.973144+0000", 2024-09-17T21:29:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.248668928000001, 2024-09-17T21:29:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00080378800000000005, 2024-09-17T21:29:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973144+0000", 2024-09-17T21:29:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973144+0000", 2024-09-17T21:29:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973144+0000", 2024-09-17T21:29:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7599999999999998e-07 2024-09-17T21:29:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973159+0000", 2024-09-17T21:29:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4997999999999999e-05 2024-09-17T21:29:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973168+0000", 2024-09-17T21:29:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0380000000000005e-06 2024-09-17T21:29:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973176+0000", 2024-09-17T21:29:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0450000000000001e-06 2024-09-17T21:29:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973225+0000", 2024-09-17T21:29:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8899000000000001e-05 2024-09-17T21:29:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973248+0000", 2024-09-17T21:29:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3113e-05 2024-09-17T21:29:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973908+0000", 2024-09-17T21:29:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066012799999999995 2024-09-17T21:29:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:32.973948+0000", 2024-09-17T21:29:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9190999999999998e-05 2024-09-17T21:29:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2790 3.0 e29/25 3:02f4e340:::rbd_data.11202c1b6cb4.0000000000000028:head v 29'3, mlcod=29'1)", 2024-09-17T21:29:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.270914+0000", 2024-09-17T21:29:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.9508985150000004, 2024-09-17T21:29:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000928888, 2024-09-17T21:29:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270914+0000", 2024-09-17T21:29:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270914+0000", 2024-09-17T21:29:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270915+0000", 2024-09-17T21:29:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.172e-06 2024-09-17T21:29:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270930+0000", 2024-09-17T21:29:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4629e-05 2024-09-17T21:29:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270947+0000", 2024-09-17T21:29:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7204999999999999e-05 2024-09-17T21:29:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.270964+0000", 2024-09-17T21:29:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6569999999999999e-05 2024-09-17T21:29:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.271034+0000", 2024-09-17T21:29:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0226000000000002e-05 2024-09-17T21:29:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.271058+0000", 2024-09-17T21:29:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3808e-05 2024-09-17T21:29:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.271808+0000", 2024-09-17T21:29:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00075016199999999996 2024-09-17T21:29:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.271843+0000", 2024-09-17T21:29:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5116000000000001e-05 2024-09-17T21:29:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2843 3.0 e30/25 3:04479396:::rbd_id.image_client.0-clone2:head v 30'4, mlcod=29'2)", 2024-09-17T21:29:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.530358+0000", 2024-09-17T21:29:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6914543950000001, 2024-09-17T21:29:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066505500000000001, 2024-09-17T21:29:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.530358+0000", 2024-09-17T21:29:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.530358+0000", 2024-09-17T21:29:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.530358+0000", 2024-09-17T21:29:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4700000000000001e-07 2024-09-17T21:29:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.530362+0000", 2024-09-17T21:29:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.546e-06 2024-09-17T21:29:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.530374+0000", 2024-09-17T21:29:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.201e-05 2024-09-17T21:29:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.530380+0000", 2024-09-17T21:29:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1879999999999999e-06 2024-09-17T21:29:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.530441+0000", 2024-09-17T21:29:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0427999999999998e-05 2024-09-17T21:29:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.530464+0000", 2024-09-17T21:29:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2935999999999999e-05 2024-09-17T21:29:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.530990+0000", 2024-09-17T21:29:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00052651300000000005 2024-09-17T21:29:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.531023+0000", 2024-09-17T21:29:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3287000000000002e-05 2024-09-17T21:29:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:2892 3.0 e30/25 3:02f4e340:::rbd_data.11202c1b6cb4.0000000000000028:head v 30'6, mlcod=29'3)", 2024-09-17T21:29:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.562368+0000", 2024-09-17T21:29:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6594450109999999, 2024-09-17T21:29:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000923331, 2024-09-17T21:29:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.562368+0000", 2024-09-17T21:29:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.562368+0000", 2024-09-17T21:29:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.562369+0000", 2024-09-17T21:29:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.344e-06 2024-09-17T21:29:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.562385+0000", 2024-09-17T21:29:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5815999999999999e-05 2024-09-17T21:29:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.562400+0000", 2024-09-17T21:29:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.488e-05 2024-09-17T21:29:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.562406+0000", 2024-09-17T21:29:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3149999999999997e-06 2024-09-17T21:29:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.562462+0000", 2024-09-17T21:29:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6271999999999999e-05 2024-09-17T21:29:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.562486+0000", 2024-09-17T21:29:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3909000000000001e-05 2024-09-17T21:29:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.563258+0000", 2024-09-17T21:29:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00077164499999999995 2024-09-17T21:29:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.563291+0000", 2024-09-17T21:29:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3149999999999999e-05 2024-09-17T21:29:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:3342 3.9 e30/25 3:918b0f69:::rbd_data.11207744cc2c.00000000000000a7:head v 30'4, mlcod=28'2)", 2024-09-17T21:29:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.881471+0000", 2024-09-17T21:29:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.3403417329999998, 2024-09-17T21:29:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00069977799999999997, 2024-09-17T21:29:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.881471+0000", 2024-09-17T21:29:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.881471+0000", 2024-09-17T21:29:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.881471+0000", 2024-09-17T21:29:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7000000000000001e-07 2024-09-17T21:29:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.881483+0000", 2024-09-17T21:29:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2140000000000001e-05 2024-09-17T21:29:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.881492+0000", 2024-09-17T21:29:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4700000000000002e-06 2024-09-17T21:29:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.881501+0000", 2024-09-17T21:29:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7909999999999993e-06 2024-09-17T21:29:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.881550+0000", 2024-09-17T21:29:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9462000000000002e-05 2024-09-17T21:29:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.881567+0000", 2024-09-17T21:29:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6558e-05 2024-09-17T21:29:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.882126+0000", 2024-09-17T21:29:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055904399999999997 2024-09-17T21:29:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.882171+0000", 2024-09-17T21:29:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5043000000000002e-05 2024-09-17T21:29:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:4374 3.0 e30/25 3:01f4aeef:::rbd_data.11207744cc2c.0000000000000230:head v 30'7, mlcod=30'4)", 2024-09-17T21:29:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.950450+0000", 2024-09-17T21:29:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2713623189999996, 2024-09-17T21:29:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00074742999999999999, 2024-09-17T21:29:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.950450+0000", 2024-09-17T21:29:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.950450+0000", 2024-09-17T21:29:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.950451+0000", 2024-09-17T21:29:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1179999999999999e-06 2024-09-17T21:29:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.950464+0000", 2024-09-17T21:29:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2767e-05 2024-09-17T21:29:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.950479+0000", 2024-09-17T21:29:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4865000000000001e-05 2024-09-17T21:29:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.950488+0000", 2024-09-17T21:29:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4689999999999997e-06 2024-09-17T21:29:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.950540+0000", 2024-09-17T21:29:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2370000000000002e-05 2024-09-17T21:29:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.950557+0000", 2024-09-17T21:29:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6552999999999999e-05 2024-09-17T21:29:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.951157+0000", 2024-09-17T21:29:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00060008900000000005 2024-09-17T21:29:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.951198+0000", 2024-09-17T21:29:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1199000000000003e-05 2024-09-17T21:29:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:4799 3.9 e30/25 3:972a9815:::rbd_data.11207744cc2c.0000000000000094:head v 30'5, mlcod=28'3)", 2024-09-17T21:29:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:47.975939+0000", 2024-09-17T21:29:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2458736530000003, 2024-09-17T21:29:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00079038399999999999, 2024-09-17T21:29:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.975939+0000", 2024-09-17T21:29:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.975939+0000", 2024-09-17T21:29:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.975941+0000", 2024-09-17T21:29:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5370000000000001e-06 2024-09-17T21:29:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.975949+0000", 2024-09-17T21:29:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8599999999999999e-06 2024-09-17T21:29:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.975967+0000", 2024-09-17T21:29:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7643e-05 2024-09-17T21:29:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.975976+0000", 2024-09-17T21:29:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1279999999999997e-06 2024-09-17T21:29:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.976038+0000", 2024-09-17T21:29:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2013000000000004e-05 2024-09-17T21:29:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.976061+0000", 2024-09-17T21:29:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2524e-05 2024-09-17T21:29:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.976692+0000", 2024-09-17T21:29:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000630942 2024-09-17T21:29:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:47.976729+0000", 2024-09-17T21:29:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7737000000000001e-05 2024-09-17T21:29:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:14839 3.9 e31/25 3:96c42965:::rbd_data.11207744cc2c.0000000000000062:head v 31'6, mlcod=30'4)", 2024-09-17T21:29:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.297020+0000", 2024-09-17T21:29:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.9247924530000002, 2024-09-17T21:29:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0014376460000000001, 2024-09-17T21:29:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297020+0000", 2024-09-17T21:29:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297020+0000", 2024-09-17T21:29:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297021+0000", 2024-09-17T21:29:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2300000000000001e-06 2024-09-17T21:29:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297217+0000", 2024-09-17T21:29:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019523500000000001 2024-09-17T21:29:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297233+0000", 2024-09-17T21:29:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6055000000000001e-05 2024-09-17T21:29:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297242+0000", 2024-09-17T21:29:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0939999999999997e-06 2024-09-17T21:29:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297294+0000", 2024-09-17T21:29:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2456000000000001e-05 2024-09-17T21:29:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297308+0000", 2024-09-17T21:29:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4051e-05 2024-09-17T21:29:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.298407+0000", 2024-09-17T21:29:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001098829 2024-09-17T21:29:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.298458+0000", 2024-09-17T21:29:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0695999999999997e-05 2024-09-17T21:29:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:14841 3.9 e31/25 3:96c42965:::rbd_data.11207744cc2c.0000000000000062:head v 31'8, mlcod=30'4)", 2024-09-17T21:29:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.297265+0000", 2024-09-17T21:29:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.9245473190000002, 2024-09-17T21:29:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0016100859999999999, 2024-09-17T21:29:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297265+0000", 2024-09-17T21:29:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297265+0000", 2024-09-17T21:29:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297266+0000", 2024-09-17T21:29:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7e-07 2024-09-17T21:29:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297278+0000", 2024-09-17T21:29:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2259000000000001e-05 2024-09-17T21:29:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297285+0000", 2024-09-17T21:29:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1049999999999997e-06 2024-09-17T21:29:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297291+0000", 2024-09-17T21:29:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3029999999999999e-06 2024-09-17T21:29:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297823+0000", 2024-09-17T21:29:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053223799999999998 2024-09-17T21:29:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297847+0000", 2024-09-17T21:29:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4063e-05 2024-09-17T21:29:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.298833+0000", 2024-09-17T21:29:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00098568500000000003 2024-09-17T21:29:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.298875+0000", 2024-09-17T21:29:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2963000000000002e-05 2024-09-17T21:29:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:20796 3.4 e32/25 3:219b1041:::rbd_data.11207744cc2c.00000000000000fc:head v 32'5, mlcod=29'3)", 2024-09-17T21:29:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.644732+0000", 2024-09-17T21:29:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.577080311, 2024-09-17T21:29:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002629045, 2024-09-17T21:29:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.644732+0000", 2024-09-17T21:29:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.644732+0000", 2024-09-17T21:29:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.644733+0000", 2024-09-17T21:29:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.27e-07 2024-09-17T21:29:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.644757+0000", 2024-09-17T21:29:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4097999999999999e-05 2024-09-17T21:29:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.644770+0000", 2024-09-17T21:29:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3275000000000001e-05 2024-09-17T21:29:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.644779+0000", 2024-09-17T21:29:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8950000000000004e-06 2024-09-17T21:29:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.644844+0000", 2024-09-17T21:29:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5087999999999998e-05 2024-09-17T21:29:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.644878+0000", 2024-09-17T21:29:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4214999999999997e-05 2024-09-17T21:29:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.647315+0000", 2024-09-17T21:29:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0024362149999999998 2024-09-17T21:29:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.647361+0000", 2024-09-17T21:29:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6931999999999997e-05 2024-09-17T21:29:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:20797 3.4 e32/25 3:219b1041:::rbd_data.11207744cc2c.00000000000000fc:head v 32'7, mlcod=29'3)", 2024-09-17T21:29:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.646448+0000", 2024-09-17T21:29:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.5753642189999999, 2024-09-17T21:29:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0013113619999999999, 2024-09-17T21:29:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.646448+0000", 2024-09-17T21:29:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.646448+0000", 2024-09-17T21:29:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.646449+0000", 2024-09-17T21:29:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.44e-07 2024-09-17T21:29:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.646455+0000", 2024-09-17T21:29:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3589999999999996e-06 2024-09-17T21:29:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.646464+0000", 2024-09-17T21:29:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6500000000000002e-06 2024-09-17T21:29:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.646471+0000", 2024-09-17T21:29:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5150000000000003e-06 2024-09-17T21:29:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.646513+0000", 2024-09-17T21:29:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2348999999999997e-05 2024-09-17T21:29:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.646531+0000", 2024-09-17T21:29:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8249000000000001e-05 2024-09-17T21:29:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.647725+0000", 2024-09-17T21:29:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0011933410000000001 2024-09-17T21:29:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.647760+0000", 2024-09-17T21:29:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5255e-05 2024-09-17T21:29:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:20970 3.9 e32/25 3:9272546e:::rbd_data.11207744cc2c.00000000000000fe:head v 32'9, mlcod=31'6)", 2024-09-17T21:29:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.653977+0000", 2024-09-17T21:29:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.567835761, 2024-09-17T21:29:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001730353, 2024-09-17T21:29:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.653977+0000", 2024-09-17T21:29:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.653977+0000", 2024-09-17T21:29:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.653977+0000", 2024-09-17T21:29:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:29:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.654039+0000", 2024-09-17T21:29:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1467e-05 2024-09-17T21:29:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.654047+0000", 2024-09-17T21:29:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1259999999999998e-06 2024-09-17T21:29:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.654056+0000", 2024-09-17T21:29:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8999999999999995e-06 2024-09-17T21:29:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.654107+0000", 2024-09-17T21:29:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1561000000000003e-05 2024-09-17T21:29:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.654131+0000", 2024-09-17T21:29:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3516999999999999e-05 2024-09-17T21:29:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.655647+0000", 2024-09-17T21:29:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015161720000000001 2024-09-17T21:29:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.655707+0000", 2024-09-17T21:29:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0238999999999997e-05 2024-09-17T21:29:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:20971 3.9 e32/25 3:9272546e:::rbd_data.11207744cc2c.00000000000000fe:head v 32'11, mlcod=31'6)", 2024-09-17T21:29:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.655655+0000", 2024-09-17T21:29:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.5661581880000002, 2024-09-17T21:29:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010148309999999999, 2024-09-17T21:29:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.655655+0000", 2024-09-17T21:29:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.655655+0000", 2024-09-17T21:29:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.655655+0000", 2024-09-17T21:29:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0999999999999998e-07 2024-09-17T21:29:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.655671+0000", 2024-09-17T21:29:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5531000000000001e-05 2024-09-17T21:29:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.655682+0000", 2024-09-17T21:29:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1501000000000001e-05 2024-09-17T21:29:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.655689+0000", 2024-09-17T21:29:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:29:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.655731+0000", 2024-09-17T21:29:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2082999999999997e-05 2024-09-17T21:29:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.655747+0000", 2024-09-17T21:29:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6341000000000002e-05 2024-09-17T21:29:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:29:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.656640+0000", 2024-09-17T21:29:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00089324399999999996 2024-09-17T21:29:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.656669+0000", 2024-09-17T21:29:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9105999999999999e-05 2024-09-17T21:29:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.290 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:52.292 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:29:52.292 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:29:52.292 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:29:52.294 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:29:52.295 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:29:52.295 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:29:52.296 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:29:52.297 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:52.289+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:52.392 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:52.390+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:52.394 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is set 2024-09-17T21:29:52.443 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:52.443 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:52.443 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:29:52.443 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:52.453 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:29:52.475 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone4 from 0xc62eb8c (overlap 0xdfc01) to 0x2cc00 2024-09-17T21:29:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:29:52.491 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 34, 2024-09-17T21:29:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "fsid": "f6d3a80c-8a1f-4716-b25d-e97b28890c0d", 2024-09-17T21:29:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "created": "2024-09-17T21:29:12.606637+0000", 2024-09-17T21:29:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "modified": "2024-09-17T21:29:52.392086+0000", 2024-09-17T21:29:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "last_up_change": "2024-09-17T21:29:24.913767+0000", 2024-09-17T21:29:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "last_in_change": "2024-09-17T21:29:15.180249+0000", 2024-09-17T21:29:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T21:29:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_num": 5801984, 2024-09-17T21:29:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_set": [ 2024-09-17T21:29:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "noscrub", 2024-09-17T21:29:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "pglog_hardlimit", 2024-09-17T21:29:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "purged_snapdirs", 2024-09-17T21:29:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_deletes", 2024-09-17T21:29:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "sortbitwise" 2024-09-17T21:29:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:29:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_version": 5, 2024-09-17T21:29:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T21:29:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T21:29:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T21:29:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_snapshot": "", 2024-09-17T21:29:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_max": 3, 2024-09-17T21:29:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "max_osd": 8, 2024-09-17T21:29:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "require_min_compat_client": "luminous", 2024-09-17T21:29:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "min_compat_client": "jewel", 2024-09-17T21:29:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "require_osd_release": "squid", 2024-09-17T21:29:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_crimson": false, 2024-09-17T21:29:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "pools": [ 2024-09-17T21:29:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 1, 2024-09-17T21:29:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": ".mgr", 2024-09-17T21:29:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "create_time": "2024-09-17T21:29:26.876763+0000", 2024-09-17T21:29:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-17T21:29:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_names": "hashpspool", 2024-09-17T21:29:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "type": 1, 2024-09-17T21:29:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 2, 2024-09-17T21:29:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": 1, 2024-09-17T21:29:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_rule": 0, 2024-09-17T21:29:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:29:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:29:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:29:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:29:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "is_stretch_pool": false, 2024-09-17T21:29:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "object_hash": 2, 2024-09-17T21:29:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:29:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": 1, 2024-09-17T21:29:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num": 1, 2024-09-17T21:29:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num_target": 1, 2024-09-17T21:29:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_target": 1, 2024-09-17T21:29:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_pending": 1, 2024-09-17T21:29:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "last_pg_merge_meta": { 2024-09-17T21:29:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "source_pgid": "0.0", 2024-09-17T21:29:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "ready_epoch": 0, 2024-09-17T21:29:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_started": 0, 2024-09-17T21:29:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_clean": 0, 2024-09-17T21:29:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "source_version": "0'0", 2024-09-17T21:29:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "target_version": "0'0" 2024-09-17T21:29:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "last_change": "19", 2024-09-17T21:29:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend": "0", 2024-09-17T21:29:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:29:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:29:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "auid": 0, 2024-09-17T21:29:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:29:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_seq": 0, 2024-09-17T21:29:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_epoch": 0, 2024-09-17T21:29:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_snaps": [], 2024-09-17T21:29:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps": "[]", 2024-09-17T21:29:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_bytes": 0, 2024-09-17T21:29:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_objects": 0, 2024-09-17T21:29:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "tiers": [], 2024-09-17T21:29:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "tier_of": -1, 2024-09-17T21:29:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "read_tier": -1, 2024-09-17T21:29:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "write_tier": -1, 2024-09-17T21:29:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_mode": "none", 2024-09-17T21:29:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_bytes": 0, 2024-09-17T21:29:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_objects": 0, 2024-09-17T21:29:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:29:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:29:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:29:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_flush_age": 0, 2024-09-17T21:29:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_evict_age": 0, 2024-09-17T21:29:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profile": "", 2024-09-17T21:29:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_params": { 2024-09-17T21:29:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "none" 2024-09-17T21:29:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_period": 0, 2024-09-17T21:29:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_count": 0, 2024-09-17T21:29:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "use_gmt_hitset": true, 2024-09-17T21:29:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:29:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:29:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:29:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:29:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "grade_table": [], 2024-09-17T21:29:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "stripe_width": 0, 2024-09-17T21:29:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "expected_num_objects": 0, 2024-09-17T21:29:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "fast_read": false, 2024-09-17T21:29:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "options": { 2024-09-17T21:29:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_max": 32, 2024-09-17T21:29:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_min": 1 2024-09-17T21:29:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "application_metadata": { 2024-09-17T21:29:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr": {} 2024-09-17T21:29:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "read_balance": { 2024-09-17T21:29:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "score_type": "Fair distribution", 2024-09-17T21:29:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "score_acting": 8, 2024-09-17T21:29:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "score_stable": 8, 2024-09-17T21:29:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "optimal_score": 0.25, 2024-09-17T21:29:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_acting": 2, 2024-09-17T21:29:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_stable": 2, 2024-09-17T21:29:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:29:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity": 1, 2024-09-17T21:29:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:29:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 2, 2024-09-17T21:29:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": "rbd", 2024-09-17T21:29:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "create_time": "2024-09-17T21:29:31.078466+0000", 2024-09-17T21:29:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 8193, 2024-09-17T21:29:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:29:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "type": 1, 2024-09-17T21:29:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 2, 2024-09-17T21:29:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": 1, 2024-09-17T21:29:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_rule": 0, 2024-09-17T21:29:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:29:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:29:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:29:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:29:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "is_stretch_pool": false, 2024-09-17T21:29:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "object_hash": 2, 2024-09-17T21:29:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:29:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": 8, 2024-09-17T21:29:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num": 8, 2024-09-17T21:29:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num_target": 8, 2024-09-17T21:29:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_target": 8, 2024-09-17T21:29:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_pending": 8, 2024-09-17T21:29:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "last_pg_merge_meta": { 2024-09-17T21:29:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "source_pgid": "0.0", 2024-09-17T21:29:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "ready_epoch": 0, 2024-09-17T21:29:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_started": 0, 2024-09-17T21:29:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_clean": 0, 2024-09-17T21:29:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "source_version": "0'0", 2024-09-17T21:29:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "target_version": "0'0" 2024-09-17T21:29:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "last_change": "23", 2024-09-17T21:29:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend": "0", 2024-09-17T21:29:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:29:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:29:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "auid": 0, 2024-09-17T21:29:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:29:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_seq": 2, 2024-09-17T21:29:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_epoch": 23, 2024-09-17T21:29:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_snaps": [], 2024-09-17T21:29:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps": "[]", 2024-09-17T21:29:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_bytes": 0, 2024-09-17T21:29:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_objects": 0, 2024-09-17T21:29:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "tiers": [], 2024-09-17T21:29:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "tier_of": -1, 2024-09-17T21:29:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "read_tier": -1, 2024-09-17T21:29:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "write_tier": -1, 2024-09-17T21:29:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_mode": "none", 2024-09-17T21:29:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_bytes": 0, 2024-09-17T21:29:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_objects": 0, 2024-09-17T21:29:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:29:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:29:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:29:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_flush_age": 0, 2024-09-17T21:29:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_evict_age": 0, 2024-09-17T21:29:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profile": "", 2024-09-17T21:29:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_params": { 2024-09-17T21:29:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "none" 2024-09-17T21:29:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_period": 0, 2024-09-17T21:29:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_count": 0, 2024-09-17T21:29:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "use_gmt_hitset": true, 2024-09-17T21:29:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:29:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:29:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:29:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:29:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "grade_table": [], 2024-09-17T21:29:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "stripe_width": 0, 2024-09-17T21:29:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "expected_num_objects": 0, 2024-09-17T21:29:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "fast_read": false, 2024-09-17T21:29:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "options": {}, 2024-09-17T21:29:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "application_metadata": { 2024-09-17T21:29:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd": {} 2024-09-17T21:29:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "read_balance": { 2024-09-17T21:29:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "score_type": "Fair distribution", 2024-09-17T21:29:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "score_acting": 1.9900000095367432, 2024-09-17T21:29:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "score_stable": 1.9900000095367432, 2024-09-17T21:29:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "optimal_score": 0.87999999523162842, 2024-09-17T21:29:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_acting": 1.75, 2024-09-17T21:29:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_stable": 1.75, 2024-09-17T21:29:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:29:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity": 1, 2024-09-17T21:29:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:29:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 3, 2024-09-17T21:29:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": "pool_client.0", 2024-09-17T21:29:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "create_time": "2024-09-17T21:29:39.892418+0000", 2024-09-17T21:29:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 8193, 2024-09-17T21:29:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:29:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "type": 1, 2024-09-17T21:29:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 2, 2024-09-17T21:29:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": 1, 2024-09-17T21:29:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_rule": 0, 2024-09-17T21:29:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:29:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:29:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:29:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:29:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "is_stretch_pool": false, 2024-09-17T21:29:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "object_hash": 2, 2024-09-17T21:29:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:29:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": 32, 2024-09-17T21:29:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num": 32, 2024-09-17T21:29:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num_target": 32, 2024-09-17T21:29:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_target": 32, 2024-09-17T21:29:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_pending": 32, 2024-09-17T21:29:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "last_pg_merge_meta": { 2024-09-17T21:29:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "source_pgid": "0.0", 2024-09-17T21:29:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "ready_epoch": 0, 2024-09-17T21:29:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_started": 0, 2024-09-17T21:29:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_clean": 0, 2024-09-17T21:29:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "source_version": "0'0", 2024-09-17T21:29:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "target_version": "0'0" 2024-09-17T21:29:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "last_change": "34", 2024-09-17T21:29:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend": "0", 2024-09-17T21:29:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:29:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:29:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "auid": 0, 2024-09-17T21:29:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:29:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_seq": 7, 2024-09-17T21:29:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_epoch": 34, 2024-09-17T21:29:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_snaps": [], 2024-09-17T21:29:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps": "[]", 2024-09-17T21:29:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_bytes": 0, 2024-09-17T21:29:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_objects": 0, 2024-09-17T21:29:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "tiers": [], 2024-09-17T21:29:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "tier_of": -1, 2024-09-17T21:29:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "read_tier": -1, 2024-09-17T21:29:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "write_tier": -1, 2024-09-17T21:29:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_mode": "none", 2024-09-17T21:29:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_bytes": 0, 2024-09-17T21:29:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_objects": 0, 2024-09-17T21:29:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:29:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:29:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:29:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_flush_age": 0, 2024-09-17T21:29:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_evict_age": 0, 2024-09-17T21:29:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profile": "", 2024-09-17T21:29:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_params": { 2024-09-17T21:29:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "none" 2024-09-17T21:29:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_period": 0, 2024-09-17T21:29:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_count": 0, 2024-09-17T21:29:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "use_gmt_hitset": true, 2024-09-17T21:29:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:29:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:29:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:29:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:29:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "grade_table": [], 2024-09-17T21:29:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "stripe_width": 0, 2024-09-17T21:29:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "expected_num_objects": 0, 2024-09-17T21:29:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "fast_read": false, 2024-09-17T21:29:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "options": {}, 2024-09-17T21:29:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "application_metadata": { 2024-09-17T21:29:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd": {} 2024-09-17T21:29:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "read_balance": { 2024-09-17T21:29:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "score_type": "Fair distribution", 2024-09-17T21:29:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "score_acting": 1.5, 2024-09-17T21:29:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "score_stable": 1.5, 2024-09-17T21:29:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "optimal_score": 1, 2024-09-17T21:29:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_acting": 1.5, 2024-09-17T21:29:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_stable": 1.5, 2024-09-17T21:29:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:29:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity": 1, 2024-09-17T21:29:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:29:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:29:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "osds": [ 2024-09-17T21:29:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 0, 2024-09-17T21:29:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "ad067b51-9402-4f69-a0f0-08fbf068dcfc", 2024-09-17T21:29:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:29:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:29:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:29:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:29:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:29:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:29:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 13, 2024-09-17T21:29:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 25, 2024-09-17T21:29:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:29:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:29:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:29:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6800", 2024-09-17T21:29:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:29:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6801", 2024-09-17T21:29:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:29:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:29:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6802", 2024-09-17T21:29:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:29:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6803", 2024-09-17T21:29:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:29:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:29:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6806", 2024-09-17T21:29:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:29:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6807", 2024-09-17T21:29:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:29:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:29:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6804", 2024-09-17T21:29:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:29:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6805", 2024-09-17T21:29:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:29:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6801/3635367421", 2024-09-17T21:29:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6803/3635367421", 2024-09-17T21:29:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6807/3635367421", 2024-09-17T21:29:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6805/3635367421", 2024-09-17T21:29:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:29:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:29:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:29:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 1, 2024-09-17T21:29:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "30385fc0-ef5f-45f5-a3ec-37e890d4826e", 2024-09-17T21:29:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:29:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:29:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:29:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:29:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:29:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:29:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:29:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 25, 2024-09-17T21:29:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:29:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:29:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:29:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6818", 2024-09-17T21:29:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 176704438 2024-09-17T21:29:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6819", 2024-09-17T21:29:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 176704438 2024-09-17T21:29:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:29:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6820", 2024-09-17T21:29:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 176704438 2024-09-17T21:29:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6821", 2024-09-17T21:29:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 176704438 2024-09-17T21:29:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:29:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6824", 2024-09-17T21:29:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 176704438 2024-09-17T21:29:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6825", 2024-09-17T21:29:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 176704438 2024-09-17T21:29:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:29:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6822", 2024-09-17T21:29:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 176704438 2024-09-17T21:29:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6823", 2024-09-17T21:29:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 176704438 2024-09-17T21:29:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6819/176704438", 2024-09-17T21:29:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6821/176704438", 2024-09-17T21:29:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6825/176704438", 2024-09-17T21:29:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6823/176704438", 2024-09-17T21:29:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:29:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:29:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:29:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 2, 2024-09-17T21:29:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "76be9b2b-b63f-402a-9ab5-2564bd671ac3", 2024-09-17T21:29:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:29:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:29:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:29:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:29:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:29:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:29:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:29:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 20, 2024-09-17T21:29:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:29:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:29:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:29:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6808", 2024-09-17T21:29:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:29:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6809", 2024-09-17T21:29:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:29:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:29:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6810", 2024-09-17T21:29:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:29:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6811", 2024-09-17T21:29:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:29:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:29:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6814", 2024-09-17T21:29:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:29:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6815", 2024-09-17T21:29:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:29:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:29:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6812", 2024-09-17T21:29:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:29:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6813", 2024-09-17T21:29:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:29:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6809/1790706405", 2024-09-17T21:29:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6811/1790706405", 2024-09-17T21:29:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6815/1790706405", 2024-09-17T21:29:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6813/1790706405", 2024-09-17T21:29:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:29:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:29:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:29:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 3, 2024-09-17T21:29:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "d6c34e3b-1ac6-415c-9c56-38ae77a5c9b0", 2024-09-17T21:29:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:29:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:29:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:29:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:29:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:29:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:29:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:29:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 25, 2024-09-17T21:29:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:29:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:29:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:29:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6826", 2024-09-17T21:29:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:29:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6827", 2024-09-17T21:29:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:29:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:29:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6828", 2024-09-17T21:29:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:29:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6829", 2024-09-17T21:29:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:29:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:29:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6832", 2024-09-17T21:29:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:29:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6833", 2024-09-17T21:29:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:29:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:29:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6830", 2024-09-17T21:29:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:29:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6831", 2024-09-17T21:29:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:29:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6827/393361453", 2024-09-17T21:29:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6829/393361453", 2024-09-17T21:29:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6833/393361453", 2024-09-17T21:29:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6831/393361453", 2024-09-17T21:29:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:29:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:29:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:29:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 4, 2024-09-17T21:29:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "7d722e8c-c497-4f0c-9f61-d9826f8c93d8", 2024-09-17T21:29:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:29:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:29:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:29:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:29:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:29:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:29:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:29:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 25, 2024-09-17T21:29:52.539 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:29:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:29:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:29:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6808", 2024-09-17T21:29:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:29:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6809", 2024-09-17T21:29:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:29:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:29:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6810", 2024-09-17T21:29:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:29:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6811", 2024-09-17T21:29:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:29:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:29:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6814", 2024-09-17T21:29:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:29:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6815", 2024-09-17T21:29:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:29:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:29:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6812", 2024-09-17T21:29:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:29:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6813", 2024-09-17T21:29:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:29:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6809/2337546183", 2024-09-17T21:29:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6811/2337546183", 2024-09-17T21:29:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6815/2337546183", 2024-09-17T21:29:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6813/2337546183", 2024-09-17T21:29:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:29:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:29:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:29:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 5, 2024-09-17T21:29:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "954f801b-3ac9-410a-a375-eb1c533744de", 2024-09-17T21:29:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:29:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:29:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:29:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:29:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:29:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:29:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:29:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 25, 2024-09-17T21:29:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:29:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:29:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:29:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6816", 2024-09-17T21:29:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:29:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6817", 2024-09-17T21:29:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:29:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:29:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6818", 2024-09-17T21:29:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:29:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6819", 2024-09-17T21:29:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:29:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:29:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6822", 2024-09-17T21:29:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:29:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6823", 2024-09-17T21:29:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:29:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:29:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6820", 2024-09-17T21:29:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:29:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6821", 2024-09-17T21:29:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:29:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6817/1510320351", 2024-09-17T21:29:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6819/1510320351", 2024-09-17T21:29:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6823/1510320351", 2024-09-17T21:29:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6821/1510320351", 2024-09-17T21:29:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:29:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:29:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:29:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 6, 2024-09-17T21:29:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "21837515-65a2-4f21-96d6-67cfc4589358", 2024-09-17T21:29:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:29:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:29:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:29:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:29:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:29:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:29:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:29:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 25, 2024-09-17T21:29:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:29:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:29:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:29:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6824", 2024-09-17T21:29:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:29:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6825", 2024-09-17T21:29:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:29:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:29:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6826", 2024-09-17T21:29:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:29:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6827", 2024-09-17T21:29:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:29:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:29:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6830", 2024-09-17T21:29:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:29:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6831", 2024-09-17T21:29:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:29:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:29:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6828", 2024-09-17T21:29:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:29:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6829", 2024-09-17T21:29:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:29:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6825/471043930", 2024-09-17T21:29:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6827/471043930", 2024-09-17T21:29:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6831/471043930", 2024-09-17T21:29:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6829/471043930", 2024-09-17T21:29:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:29:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:29:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:29:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 7, 2024-09-17T21:29:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "67af7eea-8803-425d-ae50-30a6ab91a711", 2024-09-17T21:29:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:29:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:29:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:29:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:29:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:29:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:29:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:29:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 25, 2024-09-17T21:29:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:29:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:29:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:29:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6800", 2024-09-17T21:29:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:29:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6801", 2024-09-17T21:29:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:29:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:29:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6802", 2024-09-17T21:29:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:29:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6803", 2024-09-17T21:29:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:29:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:29:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6806", 2024-09-17T21:29:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:29:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6807", 2024-09-17T21:29:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:29:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:29:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:29:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:29:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6804", 2024-09-17T21:29:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:29:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:29:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6805", 2024-09-17T21:29:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:29:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6801/1770619177", 2024-09-17T21:29:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6803/1770619177", 2024-09-17T21:29:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6807/1770619177", 2024-09-17T21:29:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6805/1770619177", 2024-09-17T21:29:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:29:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:29:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:29:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:29:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_xinfo": [ 2024-09-17T21:29:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 0, 2024-09-17T21:29:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:29:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:29:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:29:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:29:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:29:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:21.445879+0000", 2024-09-17T21:29:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:29:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 1, 2024-09-17T21:29:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:29:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:29:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:29:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:29:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:29:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.240967+0000", 2024-09-17T21:29:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:29:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 2, 2024-09-17T21:29:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:29:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:29:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:29:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:29:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:29:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.039097+0000", 2024-09-17T21:29:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:29:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 3, 2024-09-17T21:29:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:29:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:29:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:29:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:29:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:29:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.372014+0000", 2024-09-17T21:29:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:29:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 4, 2024-09-17T21:29:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:29:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:29:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:29:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:29:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:29:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.049543+0000", 2024-09-17T21:29:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:29:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 5, 2024-09-17T21:29:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:29:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:29:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:29:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:29:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:29:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.009384+0000", 2024-09-17T21:29:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:29:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 6, 2024-09-17T21:29:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:29:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:29:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:29:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:29:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:29:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.048598+0000", 2024-09-17T21:29:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:29:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 7, 2024-09-17T21:29:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:29:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:29:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:29:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:29:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:29:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.046345+0000", 2024-09-17T21:29:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:29:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:29:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_upmap": [], 2024-09-17T21:29:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_upmap_items": [], 2024-09-17T21:29:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_upmap_primaries": [], 2024-09-17T21:29:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_temp": [], 2024-09-17T21:29:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_temp": [], 2024-09-17T21:29:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "blocklist": {}, 2024-09-17T21:29:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "range_blocklist": {}, 2024-09-17T21:29:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profiles": { 2024-09-17T21:29:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "default": { 2024-09-17T21:29:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "crush-failure-domain": "osd", 2024-09-17T21:29:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "k": "2", 2024-09-17T21:29:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "m": "1", 2024-09-17T21:29:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "plugin": "jerasure", 2024-09-17T21:29:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "technique": "reed_sol_van" 2024-09-17T21:29:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps_queue": [], 2024-09-17T21:29:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "new_removed_snaps": [], 2024-09-17T21:29:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "new_purged_snaps": [ 2024-09-17T21:29:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 3, 2024-09-17T21:29:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "snaps": [ 2024-09-17T21:29:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "begin": 2, 2024-09-17T21:29:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "length": 1 2024-09-17T21:29:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:29:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_node_flags": {}, 2024-09-17T21:29:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "device_class_flags": {}, 2024-09-17T21:29:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_mode": { 2024-09-17T21:29:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_mode_enabled": false, 2024-09-17T21:29:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_bucket_count": 0, 2024-09-17T21:29:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "degraded_stretch_mode": 0, 2024-09-17T21:29:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "recovering_stretch_mode": 0, 2024-09-17T21:29:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_mode_bucket": 0 2024-09-17T21:29:52.572 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.572 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:52.574 INFO:tasks.thrashosds.thrasher:j is {'epoch': 34, 'fsid': 'f6d3a80c-8a1f-4716-b25d-e97b28890c0d', 'created': '2024-09-17T21:29:12.606637+0000', 'modified': '2024-09-17T21:29:52.392086+0000', 'last_up_change': '2024-09-17T21:29:24.913767+0000', 'last_in_change': '2024-09-17T21:29:15.180249+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T21:29:26.876763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T21:29:31.078466+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T21:29:39.892418+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 7, 'snap_epoch': 34, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'ad067b51-9402-4f69-a0f0-08fbf068dcfc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6800', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6801', 'nonce': 3635367421}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6802', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6803', 'nonce': 3635367421}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6806', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6807', 'nonce': 3635367421}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6804', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6805', 'nonce': 3635367421}]}, 'public_addr': '172.21.15.129:6801/3635367421', 'cluster_addr': '172.21.15.129:6803/3635367421', 'heartbeat_back_addr': '172.21.15.129:6807/3635367421', 'heartbeat_front_addr': '172.21.15.129:6805/3635367421', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '30385fc0-ef5f-45f5-a3ec-37e890d4826e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6818', 'nonce': 176704438}, {'type': 'v1', 'addr': '172.21.15.129:6819', 'nonce': 176704438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6820', 'nonce': 176704438}, {'type': 'v1', 'addr': '172.21.15.129:6821', 'nonce': 176704438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6824', 'nonce': 176704438}, {'type': 'v1', 'addr': '172.21.15.129:6825', 'nonce': 176704438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6822', 'nonce': 176704438}, {'type': 'v1', 'addr': '172.21.15.129:6823', 'nonce': 176704438}]}, 'public_addr': '172.21.15.129:6819/176704438', 'cluster_addr': '172.21.15.129:6821/176704438', 'heartbeat_back_addr': '172.21.15.129:6825/176704438', 'heartbeat_front_addr': '172.21.15.129:6823/176704438', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '76be9b2b-b63f-402a-9ab5-2564bd671ac3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6808', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6809', 'nonce': 1790706405}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6810', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6811', 'nonce': 1790706405}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6814', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6815', 'nonce': 1790706405}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6812', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6813', 'nonce': 1790706405}]}, 'public_addr': '172.21.15.129:6809/1790706405', 'cluster_addr': '172.21.15.129:6811/1790706405', 'heartbeat_back_addr': '172.21.15.129:6815/1790706405', 'heartbeat_front_addr': '172.21.15.129:6813/1790706405', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'd6c34e3b-1ac6-415c-9c56-38ae77a5c9b0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6826', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6827', 'nonce': 393361453}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6828', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6829', 'nonce': 393361453}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6832', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6833', 'nonce': 393361453}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6830', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6831', 'nonce': 393361453}]}, 'public_addr': '172.21.15.129:6827/393361453', 'cluster_addr': '172.21.15.129:6829/393361453', 'heartbeat_back_addr': '172.21.15.129:6833/393361453', 'heartbeat_front_addr': '172.21.15.129:6831/393361453', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '7d722e8c-c497-4f0c-9f61-d9826f8c93d8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6808', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6809', 'nonce': 2337546183}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6810', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6811', 'nonce': 2337546183}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6814', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6815', 'nonce': 2337546183}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6812', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6813', 'nonce': 2337546183}]}, 'public_addr': '172.21.15.171:6809/2337546183', 'cluster_addr': '172.21.15.171:6811/2337546183', 'heartbeat_back_addr': '172.21.15.171:6815/2337546183', 'heartbeat_front_addr': '172.21.15.171:6813/2337546183', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '954f801b-3ac9-410a-a375-eb1c533744de', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6816', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6817', 'nonce': 1510320351}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6818', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6819', 'nonce': 1510320351}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6822', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6823', 'nonce': 1510320351}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6820', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6821', 'nonce': 1510320351}]}, 'public_addr': '172.21.15.171:6817/1510320351', 'cluster_addr': '172.21.15.171:6819/1510320351', 'heartbeat_back_addr': '172.21.15.171:6823/1510320351', 'heartbeat_front_addr': '172.21.15.171:6821/1510320351', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '21837515-65a2-4f21-96d6-67cfc4589358', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6824', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6825', 'nonce': 471043930}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6826', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6827', 'nonce': 471043930}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6830', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6831', 'nonce': 471043930}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6828', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6829', 'nonce': 471043930}]}, 'public_addr': '172.21.15.171:6825/471043930', 'cluster_addr': '172.21.15.171:6827/471043930', 'heartbeat_back_addr': '172.21.15.171:6831/471043930', 'heartbeat_front_addr': '172.21.15.171:6829/471043930', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '67af7eea-8803-425d-ae50-30a6ab91a711', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6800', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6801', 'nonce': 1770619177}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6802', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6803', 'nonce': 1770619177}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6806', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6807', 'nonce': 1770619177}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6804', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6805', 'nonce': 1770619177}]}, 'public_addr': '172.21.15.171:6801/1770619177', 'cluster_addr': '172.21.15.171:6803/1770619177', 'heartbeat_back_addr': '172.21.15.171:6807/1770619177', 'heartbeat_front_addr': '172.21.15.171:6805/1770619177', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:21.445879+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.240967+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.039097+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.372014+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.049543+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.009384+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.048598+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.046345+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T21:29:52.574 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-17T21:29:52.575 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-17T21:29:52.575 INFO:teuthology.orchestra.run.smithi129.stdout:529 write 0xcd785f5 thru 0xcd7e5dd (0x5fe9 bytes) 2024-09-17T21:29:52.575 INFO:teuthology.orchestra.run.smithi129.stdout:531 read 0x96edb53 thru 0x96f2782 (0x4c30 bytes) 2024-09-17T21:29:52.575 INFO:teuthology.orchestra.run.smithi129.stdout:533 read 0x307fd6d thru 0x30802ac (0x540 bytes) 2024-09-17T21:29:52.576 INFO:teuthology.orchestra.run.smithi129.stdout:534 write 0x29506f8 thru 0x296013b (0xfa44 bytes) 2024-09-17T21:29:52.576 INFO:teuthology.orchestra.run.smithi129.stdout:535 punch from 0x9d19cde to 0x9d20fbd, (0x72df bytes) 2024-09-17T21:29:52.576 INFO:teuthology.orchestra.run.smithi129.stdout:536 read 0x362b25e thru 0x3636baa (0xb94d bytes) 2024-09-17T21:29:52.576 INFO:teuthology.orchestra.run.smithi129.stdout:537 write 0x4bdcc03 thru 0x4bde4c0 (0x18be bytes) 2024-09-17T21:29:52.576 INFO:teuthology.orchestra.run.smithi129.stdout:538 read 0xb04b210 thru 0xb04e623 (0x3414 bytes) 2024-09-17T21:29:52.576 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.576 INFO:teuthology.orchestra.run.smithi129.stdout:540 read 0x672daeb thru 0x672fec0 (0x23d6 bytes) 2024-09-17T21:29:52.576 INFO:teuthology.orchestra.run.smithi129.stdout:542 punch from 0xad22bfa to 0xad2b229, (0x862f bytes) 2024-09-17T21:29:52.576 INFO:teuthology.orchestra.run.smithi129.stdout:544 read 0x432ae7b thru 0x4332082 (0x7208 bytes) 2024-09-17T21:29:52.576 INFO:teuthology.orchestra.run.smithi129.stdout:545 read 0x9de2789 thru 0x9deb0d0 (0x8948 bytes) 2024-09-17T21:29:52.576 INFO:teuthology.orchestra.run.smithi129.stdout:546 write 0x69eccfa thru 0x69f6e6c (0xa173 bytes) 2024-09-17T21:29:52.576 INFO:teuthology.orchestra.run.smithi129.stdout:549 write 0x16fc2bb thru 0x170a2d6 (0xe01c bytes) 2024-09-17T21:29:52.576 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.577 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:52.571+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:52.578 INFO:teuthology.orchestra.run.smithi129.stdout:551 write 0xdf03239 thru 0xdf0cdae (0x9b76 bytes) 2024-09-17T21:29:52.580 INFO:teuthology.orchestra.run.smithi129.stdout:554 write 0x3da7941 thru 0x3dac81c (0x4edc bytes) 2024-09-17T21:29:52.582 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.582 INFO:teuthology.orchestra.run.smithi129.stdout:556 read 0xba8f980 thru 0xba9af07 (0xb588 bytes) 2024-09-17T21:29:52.582 INFO:teuthology.orchestra.run.smithi129.stdout:557 write 0x5a2832f thru 0x5a3062b (0x82fd bytes) 2024-09-17T21:29:52.584 INFO:teuthology.orchestra.run.smithi129.stdout:558 read 0x47993c0 thru 0x47a463c (0xb27d bytes) 2024-09-17T21:29:52.584 INFO:teuthology.orchestra.run.smithi129.stdout:559 read 0xcbea1fe thru 0xcbef68c (0x548f bytes) 2024-09-17T21:29:52.584 INFO:teuthology.orchestra.run.smithi129.stdout:561 write 0xc3a34fb thru 0xc3a9f41 (0x6a47 bytes) 2024-09-17T21:29:52.586 INFO:teuthology.orchestra.run.smithi129.stdout:568 write 0x22f9609 thru 0x23061ea (0xcbe2 bytes) 2024-09-17T21:29:52.591 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:52.591 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:52.591 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:29:52.591 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:29:52.591 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:52.592 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.592 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.592 INFO:teuthology.orchestra.run.smithi129.stdout:572 write 0xb0b1c5c thru 0xb0b96b0 (0x7a55 bytes) 2024-09-17T21:29:52.594 INFO:teuthology.orchestra.run.smithi129.stdout:574 write 0x725bae1 thru 0x72683a7 (0xc8c7 bytes) 2024-09-17T21:29:52.597 INFO:teuthology.orchestra.run.smithi129.stdout:575 write 0x6b3ec02 thru 0x6b4dc63 (0xf062 bytes) 2024-09-17T21:29:52.601 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:29:52.601 INFO:teuthology.orchestra.run.smithi129.stdout:576 read 0x8bb26f7 thru 0x8bbf169 (0xca73 bytes) 2024-09-17T21:29:52.602 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.602 INFO:teuthology.orchestra.run.smithi129.stdout:581 read 0x57e7253 thru 0x57e9343 (0x20f1 bytes) 2024-09-17T21:29:52.602 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.602 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.602 INFO:teuthology.orchestra.run.smithi129.stdout:588 read 0xbca0b72 thru 0xbca3a37 (0x2ec6 bytes) 2024-09-17T21:29:52.603 INFO:teuthology.orchestra.run.smithi129.stdout:590 punch from 0xceadeb3 to 0xceb1341, (0x348e bytes) 2024-09-17T21:29:52.603 INFO:teuthology.orchestra.run.smithi129.stdout:591 write 0x92a21c5 thru 0x92b1d5d (0xfb99 bytes) 2024-09-17T21:29:52.605 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.605 INFO:teuthology.orchestra.run.smithi129.stdout:594 punch from 0xa92ee02 to 0xa9381f5, (0x93f3 bytes) 2024-09-17T21:29:52.605 INFO:teuthology.orchestra.run.smithi129.stdout:595 read 0x30275d2 thru 0x302a83d (0x326c bytes) 2024-09-17T21:29:52.605 INFO:teuthology.orchestra.run.smithi129.stdout:596 trunc from 0xdf0cdaf to 0xd826a8b 2024-09-17T21:29:52.618 INFO:teuthology.orchestra.run.smithi129.stdout:597 write 0x1d0b891 thru 0x1d1080e (0x4f7e bytes) 2024-09-17T21:29:52.621 INFO:teuthology.orchestra.run.smithi129.stdout:598 write 0x2cd7101 thru 0x2cd780e (0x70e bytes) 2024-09-17T21:29:52.625 INFO:teuthology.orchestra.run.smithi129.stdout:599 writesame 0xc59d3b2 thru 0xc5a22d5 data_size 0xa(0x4f24 bytes) 2024-09-17T21:29:52.628 INFO:teuthology.orchestra.run.smithi129.stdout:600 write 0x4b634d8 thru 0x4b655f0 (0x2119 bytes) 2024-09-17T21:29:52.630 INFO:teuthology.orchestra.run.smithi129.stdout:601 write 0x2d7854b thru 0x2d80783 (0x8239 bytes) 2024-09-17T21:29:52.631 INFO:teuthology.orchestra.run.smithi129.stdout:602 writesame 0x7eeb151 thru 0x7ef6f58 data_size 0xc(0xbe08 bytes) 2024-09-17T21:29:52.636 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.636 INFO:teuthology.orchestra.run.smithi129.stdout:604 punch from 0xa6b53ea to 0xa6bbac1, (0x66d7 bytes) 2024-09-17T21:29:52.636 INFO:teuthology.orchestra.run.smithi129.stdout:605 write 0xd1fcd67 thru 0xd20ac39 (0xded3 bytes) 2024-09-17T21:29:52.637 INFO:teuthology.orchestra.run.smithi129.stdout:607 write 0x27d5157 thru 0x27d669d (0x1547 bytes) 2024-09-17T21:29:52.640 INFO:teuthology.orchestra.run.smithi129.stdout:608 writesame 0x38d9dd0 thru 0x38e9b1c data_size 0x83(0xfd4d bytes) 2024-09-17T21:29:52.643 INFO:teuthology.orchestra.run.smithi129.stdout:609 punch from 0xc632896 to 0xc63f413, (0xcb7d bytes) 2024-09-17T21:29:52.649 INFO:teuthology.orchestra.run.smithi129.stdout:610 write 0xe6148db thru 0xe620df4 (0xc51a bytes) 2024-09-17T21:29:52.652 INFO:teuthology.orchestra.run.smithi129.stdout:612 read 0x2812276 thru 0x28191ce (0x6f59 bytes) 2024-09-17T21:29:52.653 INFO:teuthology.orchestra.run.smithi129.stdout:613 writesame 0xd0e4944 thru 0xd0f03cd data_size 0x3f(0xba8a bytes) 2024-09-17T21:29:52.654 INFO:teuthology.orchestra.run.smithi129.stdout:614 punch from 0x6063d3c to 0x606f7e7, (0xbaab bytes) 2024-09-17T21:29:52.655 INFO:teuthology.orchestra.run.smithi129.stdout:615 read 0x55d82ef thru 0x55e0e96 (0x8ba8 bytes) 2024-09-17T21:29:52.655 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.655 INFO:teuthology.orchestra.run.smithi129.stdout:619 write 0xb6c5b92 thru 0xb6c95fa (0x3a69 bytes) 2024-09-17T21:29:52.656 INFO:teuthology.orchestra.run.smithi129.stdout:621 write 0xcd955c9 thru 0xcd95bd7 (0x60f bytes) 2024-09-17T21:29:52.663 INFO:teuthology.orchestra.run.smithi129.stdout:622 trunc from 0xe620df5 to 0x99f5f23 2024-09-17T21:29:52.673 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:52.671+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:52.681 INFO:teuthology.orchestra.run.smithi129.stdout:623 read 0x7b39919 thru 0x7b40581 (0x6c69 bytes) 2024-09-17T21:29:52.681 INFO:teuthology.orchestra.run.smithi129.stdout:625 write 0x885f8a0 thru 0x8866008 (0x6769 bytes) 2024-09-17T21:29:52.683 INFO:teuthology.orchestra.run.smithi129.stdout:626 punch from 0x68c24ce to 0x68c2a2e, (0x560 bytes) 2024-09-17T21:29:52.683 INFO:teuthology.orchestra.run.smithi129.stdout:627 punch from 0x3f3d881 to 0x3f42d29, (0x54a8 bytes) 2024-09-17T21:29:52.684 INFO:teuthology.orchestra.run.smithi129.stdout:628 write 0x411ccb thru 0x419d42 (0x8078 bytes) 2024-09-17T21:29:52.687 INFO:teuthology.orchestra.run.smithi129.stdout:629 write 0x51d7548 thru 0x51e17cb (0xa284 bytes) 2024-09-17T21:29:52.690 INFO:teuthology.orchestra.run.smithi129.stdout:631 writesame 0x1c63d9 thru 0x1d01bc data_size 0x5e(0x9de4 bytes) 2024-09-17T21:29:52.692 INFO:teuthology.orchestra.run.smithi129.stdout:632 punch from 0x27ccd2f to 0x27d8e22, (0xc0f3 bytes) 2024-09-17T21:29:52.692 INFO:teuthology.orchestra.run.smithi129.stdout:633 read 0x5c1d534 thru 0x5c1de48 (0x915 bytes) 2024-09-17T21:29:52.692 INFO:teuthology.orchestra.run.smithi129.stdout:634 punch from 0x8b7e23f to 0x8b82d6a, (0x4b2b bytes) 2024-09-17T21:29:52.692 INFO:teuthology.orchestra.run.smithi129.stdout:636 punch from 0x6d5d0c2 to 0x6d68727, (0xb665 bytes) 2024-09-17T21:29:52.692 INFO:teuthology.orchestra.run.smithi129.stdout:637 read 0x6c1f380 thru 0x6c23e85 (0x4b06 bytes) 2024-09-17T21:29:52.692 INFO:teuthology.orchestra.run.smithi129.stdout:638 read 0x7a94e68 thru 0x7a98f1f (0x40b8 bytes) 2024-09-17T21:29:52.693 INFO:teuthology.orchestra.run.smithi129.stdout:639 punch from 0x16b1930 to 0x16bb413, (0x9ae3 bytes) 2024-09-17T21:29:52.699 INFO:teuthology.orchestra.run.smithi129.stdout:640 write 0xb37314f thru 0xb3788e6 (0x5798 bytes) 2024-09-17T21:29:52.702 INFO:teuthology.orchestra.run.smithi129.stdout:642 punch from 0x2bc2c69 to 0x2bc8867, (0x5bfe bytes) 2024-09-17T21:29:52.703 INFO:teuthology.orchestra.run.smithi129.stdout:643 read 0x114c357 thru 0x1150f3a (0x4be4 bytes) 2024-09-17T21:29:52.703 INFO:teuthology.orchestra.run.smithi129.stdout:644 punch from 0x7cdfc01 to 0x7cee8bc, (0xecbb bytes) 2024-09-17T21:29:52.703 INFO:teuthology.orchestra.run.smithi129.stdout:645 writesame 0xe61314 thru 0xe67e53 data_size 0x8f(0x6b40 bytes) 2024-09-17T21:29:52.711 INFO:teuthology.orchestra.run.smithi129.stdout:646 writesame 0xc576128 thru 0xc579901 data_size 0x3(0x37da bytes) 2024-09-17T21:29:52.711 INFO:teuthology.orchestra.run.smithi129.stdout:649 read 0x77ad32c thru 0x77b8d23 (0xb9f8 bytes) 2024-09-17T21:29:52.714 INFO:teuthology.orchestra.run.smithi129.stdout:650 trunc from 0xc579902 to 0x9cfc31b 2024-09-17T21:29:52.723 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:52.723 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:29:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:29:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:29:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26531 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.849021+0000", 2024-09-17T21:29:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.8717801010000001, 2024-09-17T21:29:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024653859, 2024-09-17T21:29:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26531 2024-09-17T21:29:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849021+0000", 2024-09-17T21:29:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849021+0000", 2024-09-17T21:29:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849021+0000", 2024-09-17T21:29:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6800000000000001e-07 2024-09-17T21:29:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849023+0000", 2024-09-17T21:29:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8789999999999999e-06 2024-09-17T21:29:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849029+0000", 2024-09-17T21:29:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1059999999999996e-06 2024-09-17T21:29:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849034+0000", 2024-09-17T21:29:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:29:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.873511+0000", 2024-09-17T21:29:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024476617999999999 2024-09-17T21:29:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.873592+0000", 2024-09-17T21:29:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.064e-05 2024-09-17T21:29:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.873675+0000", 2024-09-17T21:29:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3289999999999997e-05 2024-09-17T21:29:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26532 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.849040+0000", 2024-09-17T21:29:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.8717607119999999, 2024-09-17T21:29:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024831517000000001, 2024-09-17T21:29:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26532 2024-09-17T21:29:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849040+0000", 2024-09-17T21:29:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849040+0000", 2024-09-17T21:29:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849041+0000", 2024-09-17T21:29:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6300000000000001e-07 2024-09-17T21:29:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849043+0000", 2024-09-17T21:29:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:29:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849049+0000", 2024-09-17T21:29:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9529999999999996e-06 2024-09-17T21:29:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849053+0000", 2024-09-17T21:29:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5569999999999997e-06 2024-09-17T21:29:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.873702+0000", 2024-09-17T21:29:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024648798999999999 2024-09-17T21:29:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.873790+0000", 2024-09-17T21:29:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7899999999999995e-05 2024-09-17T21:29:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.873872+0000", 2024-09-17T21:29:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2033000000000006e-05 2024-09-17T21:29:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26626 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 387072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853605+0000", 2024-09-17T21:29:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.8671955740000001, 2024-09-17T21:29:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024510626000000001, 2024-09-17T21:29:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26626 2024-09-17T21:29:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853605+0000", 2024-09-17T21:29:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853605+0000", 2024-09-17T21:29:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853665+0000", 2024-09-17T21:29:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9477e-05 2024-09-17T21:29:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853668+0000", 2024-09-17T21:29:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3139999999999999e-06 2024-09-17T21:29:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853676+0000", 2024-09-17T21:29:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7389999999999999e-06 2024-09-17T21:29:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853682+0000", 2024-09-17T21:29:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8710000000000002e-06 2024-09-17T21:29:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.877964+0000", 2024-09-17T21:29:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024282142999999999 2024-09-17T21:29:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878038+0000", 2024-09-17T21:29:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3882999999999997e-05 2024-09-17T21:29:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878116+0000", 2024-09-17T21:29:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8199000000000001e-05 2024-09-17T21:29:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26627 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 399360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853691+0000", 2024-09-17T21:29:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.867110002, 2024-09-17T21:29:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024605452999999999, 2024-09-17T21:29:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26627 2024-09-17T21:29:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853691+0000", 2024-09-17T21:29:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853691+0000", 2024-09-17T21:29:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853692+0000", 2024-09-17T21:29:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7300000000000001e-07 2024-09-17T21:29:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853694+0000", 2024-09-17T21:29:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3120000000000001e-06 2024-09-17T21:29:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853700+0000", 2024-09-17T21:29:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1940000000000003e-06 2024-09-17T21:29:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853705+0000", 2024-09-17T21:29:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8980000000000004e-06 2024-09-17T21:29:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878143+0000", 2024-09-17T21:29:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024437708999999998 2024-09-17T21:29:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878218+0000", 2024-09-17T21:29:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5377999999999996e-05 2024-09-17T21:29:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878297+0000", 2024-09-17T21:29:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8489e-05 2024-09-17T21:29:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26628 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 413696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853711+0000", 2024-09-17T21:29:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.8670899080000001, 2024-09-17T21:29:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024762052999999999, 2024-09-17T21:29:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26628 2024-09-17T21:29:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853711+0000", 2024-09-17T21:29:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853711+0000", 2024-09-17T21:29:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853712+0000", 2024-09-17T21:29:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.15e-07 2024-09-17T21:29:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853713+0000", 2024-09-17T21:29:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.79e-06 2024-09-17T21:29:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853719+0000", 2024-09-17T21:29:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0469999999999999e-06 2024-09-17T21:29:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853724+0000", 2024-09-17T21:29:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.69e-06 2024-09-17T21:29:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878324+0000", 2024-09-17T21:29:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024599875 2024-09-17T21:29:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878396+0000", 2024-09-17T21:29:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2486999999999999e-05 2024-09-17T21:29:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878473+0000", 2024-09-17T21:29:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6749000000000007e-05 2024-09-17T21:29:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26629 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853730+0000", 2024-09-17T21:29:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.8670707790000001, 2024-09-17T21:29:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024923506000000002, 2024-09-17T21:29:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26629 2024-09-17T21:29:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853730+0000", 2024-09-17T21:29:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853730+0000", 2024-09-17T21:29:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853731+0000", 2024-09-17T21:29:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:29:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853733+0000", 2024-09-17T21:29:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:29:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853739+0000", 2024-09-17T21:29:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0159999999999997e-06 2024-09-17T21:29:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853744+0000", 2024-09-17T21:29:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0019999999999997e-06 2024-09-17T21:29:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878501+0000", 2024-09-17T21:29:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024757112000000001 2024-09-17T21:29:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878573+0000", 2024-09-17T21:29:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2006000000000001e-05 2024-09-17T21:29:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878654+0000", 2024-09-17T21:29:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0734000000000003e-05 2024-09-17T21:29:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26630 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853750+0000", 2024-09-17T21:29:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.86705129, 2024-09-17T21:29:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025078738999999999, 2024-09-17T21:29:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26630 2024-09-17T21:29:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853750+0000", 2024-09-17T21:29:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853750+0000", 2024-09-17T21:29:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853750+0000", 2024-09-17T21:29:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:29:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853752+0000", 2024-09-17T21:29:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8700000000000001e-06 2024-09-17T21:29:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853758+0000", 2024-09-17T21:29:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1340000000000001e-06 2024-09-17T21:29:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853763+0000", 2024-09-17T21:29:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9819999999999999e-06 2024-09-17T21:29:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878681+0000", 2024-09-17T21:29:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024917661000000001 2024-09-17T21:29:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878752+0000", 2024-09-17T21:29:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1132999999999998e-05 2024-09-17T21:29:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878829+0000", 2024-09-17T21:29:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6638999999999996e-05 2024-09-17T21:29:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26645 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 266240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853769+0000", 2024-09-17T21:29:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.8670321159999999, 2024-09-17T21:29:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025237900000000001, 2024-09-17T21:29:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26645 2024-09-17T21:29:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853769+0000", 2024-09-17T21:29:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853769+0000", 2024-09-17T21:29:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853769+0000", 2024-09-17T21:29:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2599999999999998e-07 2024-09-17T21:29:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853771+0000", 2024-09-17T21:29:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9609999999999999e-06 2024-09-17T21:29:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853777+0000", 2024-09-17T21:29:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3630000000000002e-06 2024-09-17T21:29:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853781+0000", 2024-09-17T21:29:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7779999999999998e-06 2024-09-17T21:29:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878855+0000", 2024-09-17T21:29:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025073649999999999 2024-09-17T21:29:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878927+0000", 2024-09-17T21:29:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1500000000000003e-05 2024-09-17T21:29:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879007+0000", 2024-09-17T21:29:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0321999999999994e-05 2024-09-17T21:29:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26646 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 280576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853787+0000", 2024-09-17T21:29:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.8670138839999999, 2024-09-17T21:29:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025570283999999999, 2024-09-17T21:29:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26646 2024-09-17T21:29:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853787+0000", 2024-09-17T21:29:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853787+0000", 2024-09-17T21:29:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853788+0000", 2024-09-17T21:29:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:29:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853793+0000", 2024-09-17T21:29:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0900000000000004e-06 2024-09-17T21:29:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853798+0000", 2024-09-17T21:29:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8479999999999997e-06 2024-09-17T21:29:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853803+0000", 2024-09-17T21:29:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7620000000000003e-06 2024-09-17T21:29:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879035+0000", 2024-09-17T21:29:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025231535999999999 2024-09-17T21:29:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879107+0000", 2024-09-17T21:29:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2154000000000001e-05 2024-09-17T21:29:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879357+0000", 2024-09-17T21:29:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025055800000000001 2024-09-17T21:29:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26647 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 292864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853809+0000", 2024-09-17T21:29:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.86699182, 2024-09-17T21:29:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025774357000000001, 2024-09-17T21:29:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26647 2024-09-17T21:29:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853809+0000", 2024-09-17T21:29:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853809+0000", 2024-09-17T21:29:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853810+0000", 2024-09-17T21:29:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7500000000000001e-07 2024-09-17T21:29:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853811+0000", 2024-09-17T21:29:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7749999999999999e-06 2024-09-17T21:29:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853818+0000", 2024-09-17T21:29:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7519999999999999e-06 2024-09-17T21:29:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853823+0000", 2024-09-17T21:29:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2399999999999998e-06 2024-09-17T21:29:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879387+0000", 2024-09-17T21:29:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02556394 2024-09-17T21:29:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879462+0000", 2024-09-17T21:29:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4697000000000007e-05 2024-09-17T21:29:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879584+0000", 2024-09-17T21:29:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012157799999999999 2024-09-17T21:29:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26648 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 307200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853862+0000", 2024-09-17T21:29:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.8669393599999999, 2024-09-17T21:29:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025888738000000001, 2024-09-17T21:29:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26648 2024-09-17T21:29:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853862+0000", 2024-09-17T21:29:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853862+0000", 2024-09-17T21:29:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853862+0000", 2024-09-17T21:29:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1799999999999995e-07 2024-09-17T21:29:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853865+0000", 2024-09-17T21:29:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5950000000000001e-06 2024-09-17T21:29:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853872+0000", 2024-09-17T21:29:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1749999999999999e-06 2024-09-17T21:29:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853877+0000", 2024-09-17T21:29:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:29:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879609+0000", 2024-09-17T21:29:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025731970999999999 2024-09-17T21:29:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879674+0000", 2024-09-17T21:29:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4958999999999999e-05 2024-09-17T21:29:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879750+0000", 2024-09-17T21:29:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6327000000000003e-05 2024-09-17T21:29:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26649 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 321536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853886+0000", 2024-09-17T21:29:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.866915267, 2024-09-17T21:29:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026043475999999999, 2024-09-17T21:29:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26649 2024-09-17T21:29:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853886+0000", 2024-09-17T21:29:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853886+0000", 2024-09-17T21:29:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853886+0000", 2024-09-17T21:29:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:29:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853888+0000", 2024-09-17T21:29:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0789999999999999e-06 2024-09-17T21:29:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853894+0000", 2024-09-17T21:29:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0209999999999996e-06 2024-09-17T21:29:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853900+0000", 2024-09-17T21:29:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2220000000000001e-06 2024-09-17T21:29:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879779+0000", 2024-09-17T21:29:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025879049000000001 2024-09-17T21:29:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879852+0000", 2024-09-17T21:29:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3244999999999994e-05 2024-09-17T21:29:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879929+0000", 2024-09-17T21:29:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7472999999999995e-05 2024-09-17T21:29:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26650 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 333824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853906+0000", 2024-09-17T21:29:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.8668951499999999, 2024-09-17T21:29:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026202728000000002, 2024-09-17T21:29:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26650 2024-09-17T21:29:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853906+0000", 2024-09-17T21:29:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853906+0000", 2024-09-17T21:29:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853906+0000", 2024-09-17T21:29:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:29:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853908+0000", 2024-09-17T21:29:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0030000000000001e-06 2024-09-17T21:29:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853914+0000", 2024-09-17T21:29:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3759999999999999e-06 2024-09-17T21:29:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853919+0000", 2024-09-17T21:29:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9139999999999999e-06 2024-09-17T21:29:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879956+0000", 2024-09-17T21:29:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026037523999999999 2024-09-17T21:29:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880027+0000", 2024-09-17T21:29:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0939000000000006e-05 2024-09-17T21:29:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880109+0000", 2024-09-17T21:29:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1594000000000005e-05 2024-09-17T21:29:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26651 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853925+0000", 2024-09-17T21:29:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.8668763410000002, 2024-09-17T21:29:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026365942, 2024-09-17T21:29:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26651 2024-09-17T21:29:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853925+0000", 2024-09-17T21:29:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853925+0000", 2024-09-17T21:29:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853925+0000", 2024-09-17T21:29:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4799999999999999e-07 2024-09-17T21:29:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853927+0000", 2024-09-17T21:29:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9250000000000002e-06 2024-09-17T21:29:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853932+0000", 2024-09-17T21:29:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3489999999999999e-06 2024-09-17T21:29:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853937+0000", 2024-09-17T21:29:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4769999999999997e-06 2024-09-17T21:29:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880136+0000", 2024-09-17T21:29:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026199618000000001 2024-09-17T21:29:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880213+0000", 2024-09-17T21:29:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6594999999999994e-05 2024-09-17T21:29:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880291+0000", 2024-09-17T21:29:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.763e-05 2024-09-17T21:29:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26652 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853943+0000", 2024-09-17T21:29:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.86685788, 2024-09-17T21:29:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026527075000000001, 2024-09-17T21:29:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26652 2024-09-17T21:29:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853943+0000", 2024-09-17T21:29:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853943+0000", 2024-09-17T21:29:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853944+0000", 2024-09-17T21:29:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2599999999999998e-07 2024-09-17T21:29:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853946+0000", 2024-09-17T21:29:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9970000000000001e-06 2024-09-17T21:29:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853953+0000", 2024-09-17T21:29:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4080000000000001e-06 2024-09-17T21:29:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853958+0000", 2024-09-17T21:29:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7280000000000003e-06 2024-09-17T21:29:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880319+0000", 2024-09-17T21:29:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026360999 2024-09-17T21:29:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880391+0000", 2024-09-17T21:29:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2418999999999999e-05 2024-09-17T21:29:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880470+0000", 2024-09-17T21:29:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9197999999999996e-05 2024-09-17T21:29:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26653 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 374784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853964+0000", 2024-09-17T21:29:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.8668371829999999, 2024-09-17T21:29:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026678573000000001, 2024-09-17T21:29:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26653 2024-09-17T21:29:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853964+0000", 2024-09-17T21:29:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853964+0000", 2024-09-17T21:29:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853964+0000", 2024-09-17T21:29:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:29:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853966+0000", 2024-09-17T21:29:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.888e-06 2024-09-17T21:29:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853972+0000", 2024-09-17T21:29:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T21:29:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853976+0000", 2024-09-17T21:29:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7060000000000003e-06 2024-09-17T21:29:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880499+0000", 2024-09-17T21:29:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026523105000000002 2024-09-17T21:29:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880570+0000", 2024-09-17T21:29:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0207000000000001e-05 2024-09-17T21:29:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880642+0000", 2024-09-17T21:29:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2886000000000007e-05 2024-09-17T21:29:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26654 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 389120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853982+0000", 2024-09-17T21:29:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.8668191190000001, 2024-09-17T21:29:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026832110999999999, 2024-09-17T21:29:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26654 2024-09-17T21:29:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853982+0000", 2024-09-17T21:29:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853982+0000", 2024-09-17T21:29:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853982+0000", 2024-09-17T21:29:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8099999999999998e-07 2024-09-17T21:29:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853984+0000", 2024-09-17T21:29:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7460000000000001e-06 2024-09-17T21:29:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853990+0000", 2024-09-17T21:29:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1179999999999997e-06 2024-09-17T21:29:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853995+0000", 2024-09-17T21:29:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4789999999999998e-06 2024-09-17T21:29:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880670+0000", 2024-09-17T21:29:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026675299 2024-09-17T21:29:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880736+0000", 2024-09-17T21:29:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6476000000000005e-05 2024-09-17T21:29:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880814+0000", 2024-09-17T21:29:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7612000000000002e-05 2024-09-17T21:29:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26655 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 403456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.854001+0000", 2024-09-17T21:29:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.86680044, 2024-09-17T21:29:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026991347999999998, 2024-09-17T21:29:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26655 2024-09-17T21:29:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854001+0000", 2024-09-17T21:29:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854001+0000", 2024-09-17T21:29:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854001+0000", 2024-09-17T21:29:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.89e-07 2024-09-17T21:29:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854003+0000", 2024-09-17T21:29:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.826e-06 2024-09-17T21:29:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854009+0000", 2024-09-17T21:29:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7150000000000003e-06 2024-09-17T21:29:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854014+0000", 2024-09-17T21:29:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1109999999999999e-06 2024-09-17T21:29:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880842+0000", 2024-09-17T21:29:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026828161999999999 2024-09-17T21:29:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880912+0000", 2024-09-17T21:29:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0067000000000005e-05 2024-09-17T21:29:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880992+0000", 2024-09-17T21:29:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0078000000000001e-05 2024-09-17T21:29:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26656 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 415744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.854020+0000", 2024-09-17T21:29:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.8667809809999998, 2024-09-17T21:29:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027156040999999999, 2024-09-17T21:29:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26656 2024-09-17T21:29:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854020+0000", 2024-09-17T21:29:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854020+0000", 2024-09-17T21:29:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854020+0000", 2024-09-17T21:29:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:29:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854023+0000", 2024-09-17T21:29:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2139999999999999e-06 2024-09-17T21:29:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854028+0000", 2024-09-17T21:29:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7570000000000001e-06 2024-09-17T21:29:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854033+0000", 2024-09-17T21:29:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3409999999999996e-06 2024-09-17T21:29:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.881018+0000", 2024-09-17T21:29:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026985529000000001 2024-09-17T21:29:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.881090+0000", 2024-09-17T21:29:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1946000000000005e-05 2024-09-17T21:29:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.881176+0000", 2024-09-17T21:29:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5886999999999999e-05 2024-09-17T21:29:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26657 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.854123+0000", 2024-09-17T21:29:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.8666776939999998, 2024-09-17T21:29:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027224600000000002, 2024-09-17T21:29:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26657 2024-09-17T21:29:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854123+0000", 2024-09-17T21:29:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854123+0000", 2024-09-17T21:29:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854124+0000", 2024-09-17T21:29:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:29:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854127+0000", 2024-09-17T21:29:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0750000000000002e-06 2024-09-17T21:29:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854134+0000", 2024-09-17T21:29:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4630000000000004e-06 2024-09-17T21:29:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854141+0000", 2024-09-17T21:29:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:29:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.881204+0000", 2024-09-17T21:29:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027062685999999999 2024-09-17T21:29:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.881273+0000", 2024-09-17T21:29:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8841e-05 2024-09-17T21:29:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.881348+0000", 2024-09-17T21:29:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5265999999999994e-05 2024-09-17T21:29:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:52.816 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:52.817 INFO:teuthology.orchestra.run.smithi129.stdout:651 trunc from 0x9cfc31b to 0x642f984 2024-09-17T21:29:52.818 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:29:52.820 INFO:teuthology.orchestra.run.smithi129.stdout:654 punch from 0x1fa582d to 0x1fac37b, (0x6b4e bytes) 2024-09-17T21:29:52.821 INFO:teuthology.orchestra.run.smithi129.stdout:655 write 0x8d44b5a thru 0x8d4991d (0x4dc4 bytes) 2024-09-17T21:29:52.821 INFO:teuthology.orchestra.run.smithi129.stdout:656 write 0x5f470dd thru 0x5f54c65 (0xdb89 bytes) 2024-09-17T21:29:52.821 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.821 INFO:teuthology.orchestra.run.smithi129.stdout:658 write 0x449342 thru 0x45283e (0x94fd bytes) 2024-09-17T21:29:52.821 INFO:teuthology.orchestra.run.smithi129.stdout:659 punch from 0x6d5b822 to 0x6d68126, (0xc904 bytes) 2024-09-17T21:29:52.821 INFO:teuthology.orchestra.run.smithi129.stdout:660 write 0xc18f14 thru 0xc24ece (0xbfbb bytes) 2024-09-17T21:29:52.821 INFO:teuthology.orchestra.run.smithi129.stdout:661 write 0x20a6aa0 thru 0x20a7c6b (0x11cc bytes) 2024-09-17T21:29:52.821 INFO:teuthology.orchestra.run.smithi129.stdout:662 writesame 0xa76f3d8 thru 0xa776789 data_size 0x2(0x73b2 bytes) 2024-09-17T21:29:52.821 INFO:teuthology.orchestra.run.smithi129.stdout:664 trunc from 0xa77678a to 0x155bad3 2024-09-17T21:29:52.821 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.821 INFO:teuthology.orchestra.run.smithi129.stdout:666 read 0xfebb84 thru 0xff8d19 (0xd196 bytes) 2024-09-17T21:29:52.821 INFO:teuthology.orchestra.run.smithi129.stdout:667 punch from 0x6f912 to 0x7ef53, (0xf641 bytes) 2024-09-17T21:29:52.821 INFO:teuthology.orchestra.run.smithi129.stdout:668 writesame 0x395f018 thru 0x3965e28 data_size 0x1(0x6e11 bytes) 2024-09-17T21:29:52.822 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.822 INFO:teuthology.orchestra.run.smithi129.stdout:671 read 0xfce475 thru 0xfd9243 (0xadcf bytes) 2024-09-17T21:29:52.822 INFO:teuthology.orchestra.run.smithi129.stdout:672 read 0x3123537 thru 0x3132564 (0xf02e bytes) 2024-09-17T21:29:52.822 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.822 INFO:teuthology.orchestra.run.smithi129.stdout:674 flatten 2024-09-17T21:29:52.822 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:52.815+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:52.831 INFO:teuthology.orchestra.run.smithi129.stdout:676 write 0x5b52c0c thru 0x5b60a86 (0xde7b bytes) 2024-09-17T21:29:52.847 INFO:teuthology.orchestra.run.smithi129.stdout:678 write 0xc08756a thru 0xc096169 (0xec00 bytes) 2024-09-17T21:29:52.849 INFO:teuthology.orchestra.run.smithi129.stdout:679 write 0x255e062 thru 0x2569ff4 (0xbf93 bytes) 2024-09-17T21:29:52.862 INFO:teuthology.orchestra.run.smithi129.stdout:680 trunc from 0xc09616a to 0x30d2af3 2024-09-17T21:29:52.877 INFO:teuthology.orchestra.run.smithi129.stdout:681 read 0x2c6001 thru 0x2c8080 (0x2080 bytes) 2024-09-17T21:29:52.890 INFO:teuthology.orchestra.run.smithi129.stdout:682 write 0x8cd5d66 thru 0x8ce3c10 (0xdeab bytes) 2024-09-17T21:29:52.896 INFO:teuthology.orchestra.run.smithi129.stdout:683 trunc from 0x8ce3c11 to 0xb6b6c85 2024-09-17T21:29:52.903 INFO:teuthology.orchestra.run.smithi129.stdout:684 punch from 0xb52bf64 to 0xb53447e, (0x851a bytes) 2024-09-17T21:29:52.904 INFO:teuthology.orchestra.run.smithi129.stdout:685 read 0x800918e thru 0x800c3bf (0x3232 bytes) 2024-09-17T21:29:52.908 INFO:teuthology.orchestra.run.smithi129.stdout:686 trunc from 0xb6b6c85 to 0x7fb0c02 2024-09-17T21:29:52.915 INFO:teuthology.orchestra.run.smithi129.stdout:687 read 0x619c590 thru 0x61a6cce (0xa73f bytes) 2024-09-17T21:29:52.918 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:52.916+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:52.922 INFO:teuthology.orchestra.run.smithi129.stdout:688 write 0x961f33b thru 0x9627b07 (0x87cd bytes) 2024-09-17T21:29:52.925 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.925 INFO:teuthology.orchestra.run.smithi129.stdout:691 read 0x8dc425c thru 0x8dcecf9 (0xaa9e bytes) 2024-09-17T21:29:52.925 INFO:teuthology.orchestra.run.smithi129.stdout:693 read 0x6da6f41 thru 0x6db23e9 (0xb4a9 bytes) 2024-09-17T21:29:52.926 INFO:teuthology.orchestra.run.smithi129.stdout:694 punch from 0x6577ab4 to 0x657f8ee, (0x7e3a bytes) 2024-09-17T21:29:52.926 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:52.926 INFO:teuthology.orchestra.run.smithi129.stdout:696 punch from 0x368b8fc to 0x368fad8, (0x41dc bytes) 2024-09-17T21:29:52.926 INFO:teuthology.orchestra.run.smithi129.stdout:697 clone 6 order 24 su 4194304 sc 14 2024-09-17T21:29:52.929 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:52.929 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:29:52.929 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:29:52.929 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:52.937 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:29:53.020 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:53.017+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:53.041 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:53.041 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:29:53.041 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:29:53.041 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:29:53.041 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:53.049 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:29:53.121 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:53.119+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:53.155 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:53.155 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:29:53.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:29:53.155 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:29:53.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.155 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21284 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2050048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.155 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.673220+0000", 2024-09-17T21:29:53.155 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.478759282, 2024-09-17T21:29:53.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021205326, 2024-09-17T21:29:53.155 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.156 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.156 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.156 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.156 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.156 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.156 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21284 2024-09-17T21:29:53.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.156 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673220+0000", 2024-09-17T21:29:53.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673220+0000", 2024-09-17T21:29:53.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673220+0000", 2024-09-17T21:29:53.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:29:53.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673221+0000", 2024-09-17T21:29:53.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9399999999999993e-07 2024-09-17T21:29:53.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673224+0000", 2024-09-17T21:29:53.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9270000000000001e-06 2024-09-17T21:29:53.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673226+0000", 2024-09-17T21:29:53.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6910000000000002e-06 2024-09-17T21:29:53.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.694286+0000", 2024-09-17T21:29:53.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021059198000000001 2024-09-17T21:29:53.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.694359+0000", 2024-09-17T21:29:53.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3515000000000003e-05 2024-09-17T21:29:53.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.694425+0000", 2024-09-17T21:29:53.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5789000000000003e-05 2024-09-17T21:29:53.160 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.160 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.160 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.161 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21285 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2062336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.161 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.673229+0000", 2024-09-17T21:29:53.161 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.4787493500000002, 2024-09-17T21:29:53.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021361049, 2024-09-17T21:29:53.161 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.161 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.161 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.161 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.161 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.161 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.161 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21285 2024-09-17T21:29:53.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.162 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673229+0000", 2024-09-17T21:29:53.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673229+0000", 2024-09-17T21:29:53.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673230+0000", 2024-09-17T21:29:53.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:29:53.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673231+0000", 2024-09-17T21:29:53.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0049999999999999e-06 2024-09-17T21:29:53.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673234+0000", 2024-09-17T21:29:53.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0350000000000002e-06 2024-09-17T21:29:53.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673236+0000", 2024-09-17T21:29:53.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5399999999999998e-06 2024-09-17T21:29:53.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.694452+0000", 2024-09-17T21:29:53.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021215482000000001 2024-09-17T21:29:53.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.694526+0000", 2024-09-17T21:29:53.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3567999999999998e-05 2024-09-17T21:29:53.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.694591+0000", 2024-09-17T21:29:53.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5036999999999994e-05 2024-09-17T21:29:53.165 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.165 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.165 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.165 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21286 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2076672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.165 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.673240+0000", 2024-09-17T21:29:53.165 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.4787392000000001, 2024-09-17T21:29:53.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021632719000000002, 2024-09-17T21:29:53.165 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.165 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.166 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.166 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.166 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.166 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.166 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21286 2024-09-17T21:29:53.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.166 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673240+0000", 2024-09-17T21:29:53.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673240+0000", 2024-09-17T21:29:53.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673240+0000", 2024-09-17T21:29:53.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1100000000000002e-07 2024-09-17T21:29:53.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673241+0000", 2024-09-17T21:29:53.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.063e-06 2024-09-17T21:29:53.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673245+0000", 2024-09-17T21:29:53.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.985e-06 2024-09-17T21:29:53.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673250+0000", 2024-09-17T21:29:53.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7589999999999997e-06 2024-09-17T21:29:53.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.694618+0000", 2024-09-17T21:29:53.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021368101 2024-09-17T21:29:53.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.694801+0000", 2024-09-17T21:29:53.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000183034 2024-09-17T21:29:53.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.694872+0000", 2024-09-17T21:29:53.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1465999999999996e-05 2024-09-17T21:29:53.170 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.170 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.170 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.170 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21287 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2091008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.170 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.673256+0000", 2024-09-17T21:29:53.170 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.4787231670000001, 2024-09-17T21:29:53.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021785536000000001, 2024-09-17T21:29:53.170 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.171 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.171 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.171 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.171 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.171 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.171 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21287 2024-09-17T21:29:53.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.171 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673256+0000", 2024-09-17T21:29:53.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673256+0000", 2024-09-17T21:29:53.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673256+0000", 2024-09-17T21:29:53.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5700000000000002e-07 2024-09-17T21:29:53.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673258+0000", 2024-09-17T21:29:53.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:29:53.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673264+0000", 2024-09-17T21:29:53.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4729999999999999e-06 2024-09-17T21:29:53.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.673268+0000", 2024-09-17T21:29:53.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4490000000000001e-06 2024-09-17T21:29:53.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.694901+0000", 2024-09-17T21:29:53.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.02163321 2024-09-17T21:29:53.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.694979+0000", 2024-09-17T21:29:53.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7150999999999994e-05 2024-09-17T21:29:53.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.695041+0000", 2024-09-17T21:29:53.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2586e-05 2024-09-17T21:29:53.174 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.174 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.174 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.174 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21308 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 1996800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.175 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.674901+0000", 2024-09-17T21:29:53.175 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.4770782470000001, 2024-09-17T21:29:53.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021307514, 2024-09-17T21:29:53.175 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.175 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.175 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.175 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.175 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.175 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.175 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21308 2024-09-17T21:29:53.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.175 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674901+0000", 2024-09-17T21:29:53.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674901+0000", 2024-09-17T21:29:53.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674901+0000", 2024-09-17T21:29:53.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:29:53.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674902+0000", 2024-09-17T21:29:53.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0729999999999999e-06 2024-09-17T21:29:53.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674905+0000", 2024-09-17T21:29:53.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-06 2024-09-17T21:29:53.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674908+0000", 2024-09-17T21:29:53.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0429999999999999e-06 2024-09-17T21:29:53.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696064+0000", 2024-09-17T21:29:53.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021155964999999999 2024-09-17T21:29:53.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696138+0000", 2024-09-17T21:29:53.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4578000000000003e-05 2024-09-17T21:29:53.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696208+0000", 2024-09-17T21:29:53.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9671000000000004e-05 2024-09-17T21:29:53.178 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.178 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.178 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.179 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21309 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2011136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.179 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.674911+0000", 2024-09-17T21:29:53.179 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.4770677320000001, 2024-09-17T21:29:53.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021410991000000001, 2024-09-17T21:29:53.179 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.179 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.179 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.179 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.179 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.179 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.180 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21309 2024-09-17T21:29:53.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.180 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674911+0000", 2024-09-17T21:29:53.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674911+0000", 2024-09-17T21:29:53.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674911+0000", 2024-09-17T21:29:53.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.28e-07 2024-09-17T21:29:53.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674912+0000", 2024-09-17T21:29:53.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.127e-06 2024-09-17T21:29:53.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674916+0000", 2024-09-17T21:29:53.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0359999999999998e-06 2024-09-17T21:29:53.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674918+0000", 2024-09-17T21:29:53.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.492e-06 2024-09-17T21:29:53.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696233+0000", 2024-09-17T21:29:53.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021315099000000001 2024-09-17T21:29:53.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696281+0000", 2024-09-17T21:29:53.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8309000000000001e-05 2024-09-17T21:29:53.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696322+0000", 2024-09-17T21:29:53.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.07e-05 2024-09-17T21:29:53.183 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.183 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.183 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.183 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21310 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2025472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.183 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.674921+0000", 2024-09-17T21:29:53.183 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.4770576599999998, 2024-09-17T21:29:53.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021512078, 2024-09-17T21:29:53.184 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.184 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.184 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.184 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.184 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.184 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.184 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21310 2024-09-17T21:29:53.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.184 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674921+0000", 2024-09-17T21:29:53.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674921+0000", 2024-09-17T21:29:53.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674921+0000", 2024-09-17T21:29:53.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:29:53.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674922+0000", 2024-09-17T21:29:53.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.06e-06 2024-09-17T21:29:53.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674926+0000", 2024-09-17T21:29:53.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0879999999999999e-06 2024-09-17T21:29:53.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674928+0000", 2024-09-17T21:29:53.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4700000000000001e-06 2024-09-17T21:29:53.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696348+0000", 2024-09-17T21:29:53.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021419915000000001 2024-09-17T21:29:53.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696392+0000", 2024-09-17T21:29:53.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3761999999999999e-05 2024-09-17T21:29:53.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696433+0000", 2024-09-17T21:29:53.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1570999999999999e-05 2024-09-17T21:29:53.187 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.187 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.187 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.187 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21311 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2037760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.187 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.674931+0000", 2024-09-17T21:29:53.188 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.4770476129999999, 2024-09-17T21:29:53.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021603878, 2024-09-17T21:29:53.188 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.188 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.188 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.188 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.188 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.188 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.188 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21311 2024-09-17T21:29:53.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.188 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674931+0000", 2024-09-17T21:29:53.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674931+0000", 2024-09-17T21:29:53.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674931+0000", 2024-09-17T21:29:53.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:29:53.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674932+0000", 2024-09-17T21:29:53.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0419999999999999e-06 2024-09-17T21:29:53.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674935+0000", 2024-09-17T21:29:53.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.999e-06 2024-09-17T21:29:53.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674938+0000", 2024-09-17T21:29:53.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.48e-06 2024-09-17T21:29:53.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696458+0000", 2024-09-17T21:29:53.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021520134999999999 2024-09-17T21:29:53.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696497+0000", 2024-09-17T21:29:53.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8676e-05 2024-09-17T21:29:53.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696535+0000", 2024-09-17T21:29:53.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8333e-05 2024-09-17T21:29:53.192 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.192 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.192 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.192 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21312 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.192 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.674958+0000", 2024-09-17T21:29:53.192 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.4770205710000002, 2024-09-17T21:29:53.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021682422, 2024-09-17T21:29:53.192 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.192 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.192 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.193 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.193 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.193 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.193 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21312 2024-09-17T21:29:53.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.193 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674958+0000", 2024-09-17T21:29:53.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674958+0000", 2024-09-17T21:29:53.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674959+0000", 2024-09-17T21:29:53.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.77e-07 2024-09-17T21:29:53.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674960+0000", 2024-09-17T21:29:53.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:29:53.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674964+0000", 2024-09-17T21:29:53.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7450000000000001e-06 2024-09-17T21:29:53.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674967+0000", 2024-09-17T21:29:53.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0450000000000001e-06 2024-09-17T21:29:53.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696562+0000", 2024-09-17T21:29:53.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021594941999999999 2024-09-17T21:29:53.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696600+0000", 2024-09-17T21:29:53.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8683999999999998e-05 2024-09-17T21:29:53.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696641+0000", 2024-09-17T21:29:53.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0308000000000001e-05 2024-09-17T21:29:53.196 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.196 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.196 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.197 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21313 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2066432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.197 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.674973+0000", 2024-09-17T21:29:53.197 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.4770059949999998, 2024-09-17T21:29:53.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021768973, 2024-09-17T21:29:53.197 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.197 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.197 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.197 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.197 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.197 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.197 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21313 2024-09-17T21:29:53.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.198 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674973+0000", 2024-09-17T21:29:53.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674973+0000", 2024-09-17T21:29:53.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674973+0000", 2024-09-17T21:29:53.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:29:53.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674975+0000", 2024-09-17T21:29:53.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.389e-06 2024-09-17T21:29:53.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674980+0000", 2024-09-17T21:29:53.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1580000000000004e-06 2024-09-17T21:29:53.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674984+0000", 2024-09-17T21:29:53.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.284e-06 2024-09-17T21:29:53.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696665+0000", 2024-09-17T21:29:53.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021681207000000001 2024-09-17T21:29:53.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696703+0000", 2024-09-17T21:29:53.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7599999999999999e-05 2024-09-17T21:29:53.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696742+0000", 2024-09-17T21:29:53.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8930000000000002e-05 2024-09-17T21:29:53.201 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.201 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.201 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.201 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21314 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2078720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.201 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.674990+0000", 2024-09-17T21:29:53.201 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.476988505, 2024-09-17T21:29:53.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021861559999999999, 2024-09-17T21:29:53.202 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.202 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.202 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.202 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.202 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.202 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.202 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21314 2024-09-17T21:29:53.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.202 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674990+0000", 2024-09-17T21:29:53.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674990+0000", 2024-09-17T21:29:53.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674991+0000", 2024-09-17T21:29:53.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4299999999999998e-07 2024-09-17T21:29:53.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674993+0000", 2024-09-17T21:29:53.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8080000000000001e-06 2024-09-17T21:29:53.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.674998+0000", 2024-09-17T21:29:53.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.84e-06 2024-09-17T21:29:53.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.675003+0000", 2024-09-17T21:29:53.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.459e-06 2024-09-17T21:29:53.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696766+0000", 2024-09-17T21:29:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021762624000000001 2024-09-17T21:29:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696806+0000", 2024-09-17T21:29:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0299999999999997e-05 2024-09-17T21:29:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696852+0000", 2024-09-17T21:29:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6085999999999999e-05 2024-09-17T21:29:53.205 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21315 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2093056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.675012+0000", 2024-09-17T21:29:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.476967063, 2024-09-17T21:29:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021939075999999998, 2024-09-17T21:29:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.206 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21315 2024-09-17T21:29:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.675012+0000", 2024-09-17T21:29:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.675012+0000", 2024-09-17T21:29:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.675012+0000", 2024-09-17T21:29:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6100000000000002e-07 2024-09-17T21:29:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.675014+0000", 2024-09-17T21:29:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0320000000000002e-06 2024-09-17T21:29:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.675020+0000", 2024-09-17T21:29:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.976e-06 2024-09-17T21:29:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.675025+0000", 2024-09-17T21:29:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:29:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696875+0000", 2024-09-17T21:29:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021849967000000001 2024-09-17T21:29:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696913+0000", 2024-09-17T21:29:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8294999999999998e-05 2024-09-17T21:29:53.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.696951+0000", 2024-09-17T21:29:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7764e-05 2024-09-17T21:29:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21336 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2000896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676550+0000", 2024-09-17T21:29:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.4754286429999999, 2024-09-17T21:29:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021269804, 2024-09-17T21:29:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.210 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21336 2024-09-17T21:29:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676550+0000", 2024-09-17T21:29:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676550+0000", 2024-09-17T21:29:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676551+0000", 2024-09-17T21:29:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5400000000000002e-07 2024-09-17T21:29:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676552+0000", 2024-09-17T21:29:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9379999999999999e-06 2024-09-17T21:29:53.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676558+0000", 2024-09-17T21:29:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7400000000000001e-06 2024-09-17T21:29:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676563+0000", 2024-09-17T21:29:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.33e-06 2024-09-17T21:29:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.697684+0000", 2024-09-17T21:29:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021121860999999999 2024-09-17T21:29:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.697756+0000", 2024-09-17T21:29:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1240000000000002e-05 2024-09-17T21:29:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.697820+0000", 2024-09-17T21:29:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4340999999999999e-05 2024-09-17T21:29:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.214 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21337 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2013184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676569+0000", 2024-09-17T21:29:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.4754096319999999, 2024-09-17T21:29:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021364847999999999, 2024-09-17T21:29:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21337 2024-09-17T21:29:53.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676569+0000", 2024-09-17T21:29:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676569+0000", 2024-09-17T21:29:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676570+0000", 2024-09-17T21:29:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:29:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676571+0000", 2024-09-17T21:29:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7999999999999999e-06 2024-09-17T21:29:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676577+0000", 2024-09-17T21:29:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0410000000000003e-06 2024-09-17T21:29:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676583+0000", 2024-09-17T21:29:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:29:53.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.697855+0000", 2024-09-17T21:29:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021271742 2024-09-17T21:29:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.697895+0000", 2024-09-17T21:29:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9725999999999999e-05 2024-09-17T21:29:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.697934+0000", 2024-09-17T21:29:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9301999999999997e-05 2024-09-17T21:29:53.218 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21338 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2027520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676612+0000", 2024-09-17T21:29:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.4753668640000002, 2024-09-17T21:29:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021447500000000001, 2024-09-17T21:29:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.219 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.220 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.220 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.220 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21338 2024-09-17T21:29:53.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.220 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676612+0000", 2024-09-17T21:29:53.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676612+0000", 2024-09-17T21:29:53.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676613+0000", 2024-09-17T21:29:53.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8100000000000003e-07 2024-09-17T21:29:53.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676614+0000", 2024-09-17T21:29:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8700000000000001e-06 2024-09-17T21:29:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676619+0000", 2024-09-17T21:29:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1649999999999999e-06 2024-09-17T21:29:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676622+0000", 2024-09-17T21:29:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.253e-06 2024-09-17T21:29:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.697961+0000", 2024-09-17T21:29:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021339130000000001 2024-09-17T21:29:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698005+0000", 2024-09-17T21:29:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3888999999999999e-05 2024-09-17T21:29:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698059+0000", 2024-09-17T21:29:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4611999999999998e-05 2024-09-17T21:29:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.223 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21339 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2041856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676627+0000", 2024-09-17T21:29:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.4753523049999999, 2024-09-17T21:29:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021551851, 2024-09-17T21:29:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21339 2024-09-17T21:29:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.224 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676627+0000", 2024-09-17T21:29:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676627+0000", 2024-09-17T21:29:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676627+0000", 2024-09-17T21:29:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:29:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676628+0000", 2024-09-17T21:29:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.093e-06 2024-09-17T21:29:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676631+0000", 2024-09-17T21:29:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0369999999999999e-06 2024-09-17T21:29:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676634+0000", 2024-09-17T21:29:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6369999999999999e-06 2024-09-17T21:29:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698078+0000", 2024-09-17T21:29:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021444787999999999 2024-09-17T21:29:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698137+0000", 2024-09-17T21:29:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8607999999999999e-05 2024-09-17T21:29:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698178+0000", 2024-09-17T21:29:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1344000000000003e-05 2024-09-17T21:29:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21340 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2054144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676637+0000", 2024-09-17T21:29:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.4753419389999998, 2024-09-17T21:29:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.02167465, 2024-09-17T21:29:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.228 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21340 2024-09-17T21:29:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676637+0000", 2024-09-17T21:29:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676637+0000", 2024-09-17T21:29:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676637+0000", 2024-09-17T21:29:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:29:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676638+0000", 2024-09-17T21:29:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0160000000000001e-06 2024-09-17T21:29:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676641+0000", 2024-09-17T21:29:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0060000000000001e-06 2024-09-17T21:29:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676644+0000", 2024-09-17T21:29:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6089999999999999e-06 2024-09-17T21:29:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698203+0000", 2024-09-17T21:29:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021559526999999998 2024-09-17T21:29:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698246+0000", 2024-09-17T21:29:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2518000000000002e-05 2024-09-17T21:29:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698312+0000", 2024-09-17T21:29:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5759999999999994e-05 2024-09-17T21:29:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21341 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2068480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676648+0000", 2024-09-17T21:29:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.4753309149999998, 2024-09-17T21:29:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021871430000000001, 2024-09-17T21:29:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.232 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21341 2024-09-17T21:29:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676648+0000", 2024-09-17T21:29:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676648+0000", 2024-09-17T21:29:53.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676648+0000", 2024-09-17T21:29:53.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:29:53.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676649+0000", 2024-09-17T21:29:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.062e-06 2024-09-17T21:29:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676652+0000", 2024-09-17T21:29:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0520000000000002e-06 2024-09-17T21:29:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676655+0000", 2024-09-17T21:29:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5380000000000001e-06 2024-09-17T21:29:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698355+0000", 2024-09-17T21:29:53.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021699969 2024-09-17T21:29:53.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698441+0000", 2024-09-17T21:29:53.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6341000000000006e-05 2024-09-17T21:29:53.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698519+0000", 2024-09-17T21:29:53.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8256000000000004e-05 2024-09-17T21:29:53.237 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.237 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.237 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.237 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21342 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2082816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.237 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:29:53.237 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.475321101, 2024-09-17T21:29:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022083523000000001, 2024-09-17T21:29:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21342 2024-09-17T21:29:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:29:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:29:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:29:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:29:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676659+0000", 2024-09-17T21:29:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0950000000000001e-06 2024-09-17T21:29:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676662+0000", 2024-09-17T21:29:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.971e-06 2024-09-17T21:29:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676665+0000", 2024-09-17T21:29:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6189999999999998e-06 2024-09-17T21:29:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698568+0000", 2024-09-17T21:29:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021903549000000001 2024-09-17T21:29:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698668+0000", 2024-09-17T21:29:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9778999999999995e-05 2024-09-17T21:29:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698741+0000", 2024-09-17T21:29:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000006e-05 2024-09-17T21:29:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21343 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2095104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:29:53.241 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:29:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.4753111629999998, 2024-09-17T21:29:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022504024000000001, 2024-09-17T21:29:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21343 2024-09-17T21:29:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:29:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:29:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:29:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:29:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676669+0000", 2024-09-17T21:29:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.003e-06 2024-09-17T21:29:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676672+0000", 2024-09-17T21:29:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9280000000000002e-06 2024-09-17T21:29:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676674+0000", 2024-09-17T21:29:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5469999999999999e-06 2024-09-17T21:29:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699015+0000", 2024-09-17T21:29:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022340563000000001 2024-09-17T21:29:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699102+0000", 2024-09-17T21:29:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7425999999999999e-05 2024-09-17T21:29:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699172+0000", 2024-09-17T21:29:53.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9346999999999999e-05 2024-09-17T21:29:53.246 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.246 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.246 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.246 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.246 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.246 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:53.247 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:29:53.248 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:53.245+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:29:53.348 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:53.345+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:29:53.391 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:29:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:29:53.391 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:53.399 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:29:53.449 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:53.447+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:53.502 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:53.502 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:29:53.502 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:29:53.502 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:29:53.502 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:53.510 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:29:53.551 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:53.549+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:53.651 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:53.648+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:53.656 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:53.657 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:29:53.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:29:53.657 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:29:53.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.657 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10799 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.657 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:29:53.657 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5885055870000002, 2024-09-17T21:29:53.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039122745, 2024-09-17T21:29:53.657 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.657 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.657 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:53.657 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.658 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.658 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.658 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10799 2024-09-17T21:29:53.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.658 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:29:53.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:29:53.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065469+0000", 2024-09-17T21:29:53.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:29:53.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065477+0000", 2024-09-17T21:29:53.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1319999999999994e-06 2024-09-17T21:29:53.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065483+0000", 2024-09-17T21:29:53.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2040000000000002e-06 2024-09-17T21:29:53.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065487+0000", 2024-09-17T21:29:53.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.42e-06 2024-09-17T21:29:53.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099984+0000", 2024-09-17T21:29:53.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034496484000000001 2024-09-17T21:29:53.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100065+0000", 2024-09-17T21:29:53.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1200999999999997e-05 2024-09-17T21:29:53.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:29:53.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100422+0000", 2024-09-17T21:29:53.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035657700000000001 2024-09-17T21:29:53.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:53.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104214+0000", 2024-09-17T21:29:53.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003792045 2024-09-17T21:29:53.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:53.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104505+0000", 2024-09-17T21:29:53.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029141399999999999 2024-09-17T21:29:53.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:53.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104556+0000", 2024-09-17T21:29:53.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1388999999999999e-05 2024-09-17T21:29:53.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104591+0000", 2024-09-17T21:29:53.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4455999999999997e-05 2024-09-17T21:29:53.663 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.663 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.663 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.663 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12187 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.663 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:29:53.663 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5144128559999999, 2024-09-17T21:29:53.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038825750999999999, 2024-09-17T21:29:53.663 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.663 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.663 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.664 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12187 2024-09-17T21:29:53.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.664 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:29:53.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:29:53.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:29:53.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:29:53.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139563+0000", 2024-09-17T21:29:53.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.624e-06 2024-09-17T21:29:53.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139569+0000", 2024-09-17T21:29:53.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1090000000000003e-06 2024-09-17T21:29:53.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139573+0000", 2024-09-17T21:29:53.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4429999999999996e-06 2024-09-17T21:29:53.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178206+0000", 2024-09-17T21:29:53.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038632376000000003 2024-09-17T21:29:53.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178282+0000", 2024-09-17T21:29:53.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6076999999999997e-05 2024-09-17T21:29:53.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178387+0000", 2024-09-17T21:29:53.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104682 2024-09-17T21:29:53.667 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.667 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.667 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.668 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12188 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.668 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:29:53.668 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5143946750000001, 2024-09-17T21:29:53.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039053826999999999, 2024-09-17T21:29:53.668 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.668 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.668 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.668 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12188 2024-09-17T21:29:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:29:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:29:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139746+0000", 2024-09-17T21:29:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000167281 2024-09-17T21:29:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139749+0000", 2024-09-17T21:29:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9660000000000001e-06 2024-09-17T21:29:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139757+0000", 2024-09-17T21:29:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8029999999999997e-06 2024-09-17T21:29:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139771+0000", 2024-09-17T21:29:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3541e-05 2024-09-17T21:29:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178415+0000", 2024-09-17T21:29:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038644803999999998 2024-09-17T21:29:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178517+0000", 2024-09-17T21:29:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101298 2024-09-17T21:29:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178633+0000", 2024-09-17T21:29:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116134 2024-09-17T21:29:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12189 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:29:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5141941980000002, 2024-09-17T21:29:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039135991000000002, 2024-09-17T21:29:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12189 2024-09-17T21:29:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:29:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:29:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139780+0000", 2024-09-17T21:29:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:29:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139782+0000", 2024-09-17T21:29:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1950000000000002e-06 2024-09-17T21:29:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139788+0000", 2024-09-17T21:29:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0639999999999999e-06 2024-09-17T21:29:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139792+0000", 2024-09-17T21:29:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1960000000000001e-06 2024-09-17T21:29:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178671+0000", 2024-09-17T21:29:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038878176 2024-09-17T21:29:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178778+0000", 2024-09-17T21:29:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107278 2024-09-17T21:29:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178915+0000", 2024-09-17T21:29:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013757900000000001 2024-09-17T21:29:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12190 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:29:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5141750150000002, 2024-09-17T21:29:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039343781000000001, 2024-09-17T21:29:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12190 2024-09-17T21:29:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:29:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:29:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:29:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:29:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139801+0000", 2024-09-17T21:29:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.916e-06 2024-09-17T21:29:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139807+0000", 2024-09-17T21:29:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4540000000000004e-06 2024-09-17T21:29:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139812+0000", 2024-09-17T21:29:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9729999999999997e-06 2024-09-17T21:29:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178960+0000", 2024-09-17T21:29:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039147567000000001 2024-09-17T21:29:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179051+0000", 2024-09-17T21:29:53.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0754000000000006e-05 2024-09-17T21:29:53.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179142+0000", 2024-09-17T21:29:53.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1712999999999994e-05 2024-09-17T21:29:53.680 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.680 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.680 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.680 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12191 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.681 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:29:53.681 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5141551570000003, 2024-09-17T21:29:53.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039464336000000003, 2024-09-17T21:29:53.681 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.681 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.681 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.681 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.681 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.681 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.681 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12191 2024-09-17T21:29:53.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.681 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:29:53.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:29:53.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:29:53.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:29:53.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139821+0000", 2024-09-17T21:29:53.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1260000000000001e-06 2024-09-17T21:29:53.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139827+0000", 2024-09-17T21:29:53.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T21:29:53.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139840+0000", 2024-09-17T21:29:53.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2607e-05 2024-09-17T21:29:53.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179162+0000", 2024-09-17T21:29:53.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039322863999999999 2024-09-17T21:29:53.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179204+0000", 2024-09-17T21:29:53.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1938999999999999e-05 2024-09-17T21:29:53.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179283+0000", 2024-09-17T21:29:53.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8547000000000005e-05 2024-09-17T21:29:53.684 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.684 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.684 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.684 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12192 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.685 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:29:53.685 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5141274449999997, 2024-09-17T21:29:53.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039652318999999998, 2024-09-17T21:29:53.685 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.685 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.685 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.685 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.685 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.685 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.685 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12192 2024-09-17T21:29:53.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.685 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:29:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:29:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141458+0000", 2024-09-17T21:29:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001612098 2024-09-17T21:29:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141469+0000", 2024-09-17T21:29:53.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-17T21:29:53.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141478+0000", 2024-09-17T21:29:53.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T21:29:53.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141485+0000", 2024-09-17T21:29:53.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4760000000000003e-06 2024-09-17T21:29:53.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179317+0000", 2024-09-17T21:29:53.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037831832000000003 2024-09-17T21:29:53.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179395+0000", 2024-09-17T21:29:53.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8349000000000005e-05 2024-09-17T21:29:53.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179499+0000", 2024-09-17T21:29:53.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103628 2024-09-17T21:29:53.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.689 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.689 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12257 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.689 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:29:53.689 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5119386370000001, 2024-09-17T21:29:53.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038862751000000001, 2024-09-17T21:29:53.689 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.689 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.689 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.689 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.689 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.689 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.690 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12257 2024-09-17T21:29:53.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.690 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:29:53.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:29:53.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:29:53.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T21:29:53.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142037+0000", 2024-09-17T21:29:53.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.03e-06 2024-09-17T21:29:53.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142043+0000", 2024-09-17T21:29:53.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6319999999999996e-06 2024-09-17T21:29:53.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142052+0000", 2024-09-17T21:29:53.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2709999999999998e-06 2024-09-17T21:29:53.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180794+0000", 2024-09-17T21:29:53.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038741758000000001 2024-09-17T21:29:53.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180843+0000", 2024-09-17T21:29:53.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9021999999999999e-05 2024-09-17T21:29:53.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180898+0000", 2024-09-17T21:29:53.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673000000000003e-05 2024-09-17T21:29:53.693 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.693 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.693 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.693 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12258 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.693 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:29:53.693 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5119151200000003, 2024-09-17T21:29:53.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038950717000000003, 2024-09-17T21:29:53.693 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.693 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.693 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.693 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.693 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.694 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12258 2024-09-17T21:29:53.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.694 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:29:53.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:29:53.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:29:53.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199999999999999e-07 2024-09-17T21:29:53.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142061+0000", 2024-09-17T21:29:53.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8279999999999999e-06 2024-09-17T21:29:53.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142067+0000", 2024-09-17T21:29:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5600000000000001e-06 2024-09-17T21:29:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142071+0000", 2024-09-17T21:29:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6240000000000001e-06 2024-09-17T21:29:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180921+0000", 2024-09-17T21:29:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038849366000000003 2024-09-17T21:29:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180961+0000", 2024-09-17T21:29:53.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0219000000000002e-05 2024-09-17T21:29:53.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181009+0000", 2024-09-17T21:29:53.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8437999999999999e-05 2024-09-17T21:29:53.697 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.697 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.697 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.697 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12259 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.697 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:29:53.697 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.511887873, 2024-09-17T21:29:53.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039116832999999997, 2024-09-17T21:29:53.698 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.698 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.698 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.698 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.698 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.698 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.698 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12259 2024-09-17T21:29:53.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.698 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:29:53.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:29:53.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:29:53.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2200000000000004e-07 2024-09-17T21:29:53.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142088+0000", 2024-09-17T21:29:53.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0329999999999998e-06 2024-09-17T21:29:53.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142095+0000", 2024-09-17T21:29:53.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4320000000000004e-06 2024-09-17T21:29:53.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142100+0000", 2024-09-17T21:29:53.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:29:53.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181037+0000", 2024-09-17T21:29:53.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038937601000000002 2024-09-17T21:29:53.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181114+0000", 2024-09-17T21:29:53.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6857e-05 2024-09-17T21:29:53.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181203+0000", 2024-09-17T21:29:53.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8437999999999996e-05 2024-09-17T21:29:53.701 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.702 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.702 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.702 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12260 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.702 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:29:53.702 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5118675860000002, 2024-09-17T21:29:53.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039204184000000003, 2024-09-17T21:29:53.702 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.702 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.702 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.702 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.702 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.702 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.702 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12260 2024-09-17T21:29:53.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.703 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:29:53.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:29:53.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142107+0000", 2024-09-17T21:29:53.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:29:53.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142108+0000", 2024-09-17T21:29:53.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.753e-06 2024-09-17T21:29:53.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142114+0000", 2024-09-17T21:29:53.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7660000000000004e-06 2024-09-17T21:29:53.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142119+0000", 2024-09-17T21:29:53.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:29:53.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181216+0000", 2024-09-17T21:29:53.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039097191000000003 2024-09-17T21:29:53.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181260+0000", 2024-09-17T21:29:53.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4731000000000003e-05 2024-09-17T21:29:53.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181310+0000", 2024-09-17T21:29:53.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9818000000000003e-05 2024-09-17T21:29:53.706 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.706 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.706 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.706 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12261 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.706 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:29:53.706 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5118492620000001, 2024-09-17T21:29:53.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039292200999999999, 2024-09-17T21:29:53.706 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.706 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.706 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.706 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.707 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.707 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.707 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12261 2024-09-17T21:29:53.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.707 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:29:53.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:29:53.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142125+0000", 2024-09-17T21:29:53.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:29:53.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142127+0000", 2024-09-17T21:29:53.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.792e-06 2024-09-17T21:29:53.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142132+0000", 2024-09-17T21:29:53.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3979999999999998e-06 2024-09-17T21:29:53.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142136+0000", 2024-09-17T21:29:53.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.211e-06 2024-09-17T21:29:53.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181324+0000", 2024-09-17T21:29:53.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039187560000000003 2024-09-17T21:29:53.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181363+0000", 2024-09-17T21:29:53.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9375000000000002e-05 2024-09-17T21:29:53.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181417+0000", 2024-09-17T21:29:53.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3372000000000003e-05 2024-09-17T21:29:53.710 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.710 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.710 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.710 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12262 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.710 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:29:53.710 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5118314689999997, 2024-09-17T21:29:53.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039384875, 2024-09-17T21:29:53.710 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.711 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.711 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.711 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.711 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.711 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.711 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12262 2024-09-17T21:29:53.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.711 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:29:53.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:29:53.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142143+0000", 2024-09-17T21:29:53.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:29:53.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142148+0000", 2024-09-17T21:29:53.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4360000000000001e-06 2024-09-17T21:29:53.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142154+0000", 2024-09-17T21:29:53.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8719999999999999e-06 2024-09-17T21:29:53.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142158+0000", 2024-09-17T21:29:53.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.476e-06 2024-09-17T21:29:53.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181436+0000", 2024-09-17T21:29:53.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039277919000000001 2024-09-17T21:29:53.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181480+0000", 2024-09-17T21:29:53.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3649999999999997e-05 2024-09-17T21:29:53.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181527+0000", 2024-09-17T21:29:53.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7131e-05 2024-09-17T21:29:53.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.714 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.715 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12263 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1079296~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.715 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:29:53.715 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5118090339999997, 2024-09-17T21:29:53.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039473339000000003, 2024-09-17T21:29:53.715 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.715 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.715 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.715 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12263 2024-09-17T21:29:53.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.715 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:29:53.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:29:53.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:29:53.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:29:53.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142167+0000", 2024-09-17T21:29:53.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.81e-06 2024-09-17T21:29:53.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142172+0000", 2024-09-17T21:29:53.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:29:53.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142177+0000", 2024-09-17T21:29:53.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T21:29:53.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181553+0000", 2024-09-17T21:29:53.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039376130000000002 2024-09-17T21:29:53.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181592+0000", 2024-09-17T21:29:53.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9153999999999998e-05 2024-09-17T21:29:53.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181638+0000", 2024-09-17T21:29:53.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5864e-05 2024-09-17T21:29:53.718 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.718 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.718 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.719 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12278 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1095680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.719 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:29:53.719 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5117910590000001, 2024-09-17T21:29:53.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039454222999999997, 2024-09-17T21:29:53.719 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.719 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.719 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.719 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.719 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.719 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.719 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12278 2024-09-17T21:29:53.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.719 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:29:53.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:29:53.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:29:53.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:29:53.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142185+0000", 2024-09-17T21:29:53.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.697e-06 2024-09-17T21:29:53.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142191+0000", 2024-09-17T21:29:53.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:29:53.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142196+0000", 2024-09-17T21:29:53.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4829999999999999e-06 2024-09-17T21:29:53.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181548+0000", 2024-09-17T21:29:53.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039351200000000003 2024-09-17T21:29:53.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181591+0000", 2024-09-17T21:29:53.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3492000000000003e-05 2024-09-17T21:29:53.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181637+0000", 2024-09-17T21:29:53.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5847000000000003e-05 2024-09-17T21:29:53.723 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.723 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.723 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.723 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12279 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.723 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:29:53.723 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5117677699999996, 2024-09-17T21:29:53.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039539801999999999, 2024-09-17T21:29:53.723 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.723 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.723 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.724 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.724 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.724 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.724 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12279 2024-09-17T21:29:53.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.724 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:29:53.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:29:53.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:29:53.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.89e-07 2024-09-17T21:29:53.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142208+0000", 2024-09-17T21:29:53.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7579999999999999e-06 2024-09-17T21:29:53.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142214+0000", 2024-09-17T21:29:53.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:29:53.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142218+0000", 2024-09-17T21:29:53.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3429999999999998e-06 2024-09-17T21:29:53.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181653+0000", 2024-09-17T21:29:53.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039435322000000002 2024-09-17T21:29:53.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181698+0000", 2024-09-17T21:29:53.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4978999999999997e-05 2024-09-17T21:29:53.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181746+0000", 2024-09-17T21:29:53.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571999999999998e-05 2024-09-17T21:29:53.727 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.727 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.727 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.727 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12280 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1124352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.727 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:29:53.728 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5117502510000005, 2024-09-17T21:29:53.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039704358000000002, 2024-09-17T21:29:53.728 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.728 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.728 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.728 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.728 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.728 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.728 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12280 2024-09-17T21:29:53.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.728 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:29:53.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:29:53.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142224+0000", 2024-09-17T21:29:53.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:29:53.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142226+0000", 2024-09-17T21:29:53.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:29:53.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142231+0000", 2024-09-17T21:29:53.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:29:53.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142240+0000", 2024-09-17T21:29:53.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4590000000000004e-06 2024-09-17T21:29:53.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181774+0000", 2024-09-17T21:29:53.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039533833999999997 2024-09-17T21:29:53.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181859+0000", 2024-09-17T21:29:53.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4328999999999998e-05 2024-09-17T21:29:53.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181928+0000", 2024-09-17T21:29:53.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9139000000000003e-05 2024-09-17T21:29:53.731 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.731 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.731 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.732 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12281 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1136640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.732 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:29:53.732 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5117273449999997, 2024-09-17T21:29:53.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039787799999999998, 2024-09-17T21:29:53.732 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.732 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.732 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.732 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.732 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.732 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.732 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12281 2024-09-17T21:29:53.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.732 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:29:53.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:29:53.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142309+0000", 2024-09-17T21:29:53.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2323999999999995e-05 2024-09-17T21:29:53.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142317+0000", 2024-09-17T21:29:53.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1859999999999992e-06 2024-09-17T21:29:53.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142324+0000", 2024-09-17T21:29:53.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4699999999999996e-06 2024-09-17T21:29:53.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142330+0000", 2024-09-17T21:29:53.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4650000000000002e-06 2024-09-17T21:29:53.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181948+0000", 2024-09-17T21:29:53.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039618056999999998 2024-09-17T21:29:53.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181986+0000", 2024-09-17T21:29:53.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8442999999999998e-05 2024-09-17T21:29:53.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182034+0000", 2024-09-17T21:29:53.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7855000000000002e-05 2024-09-17T21:29:53.735 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.735 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.736 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.736 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12282 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1150976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.736 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:29:53.736 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5116351159999999, 2024-09-17T21:29:53.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039871919999999998, 2024-09-17T21:29:53.736 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.736 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.736 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:53.736 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.736 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.736 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.736 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12282 2024-09-17T21:29:53.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.737 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:29:53.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:29:53.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:29:53.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:29:53.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142341+0000", 2024-09-17T21:29:53.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.232e-06 2024-09-17T21:29:53.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142348+0000", 2024-09-17T21:29:53.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7320000000000001e-06 2024-09-17T21:29:53.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142353+0000", 2024-09-17T21:29:53.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:29:53.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182062+0000", 2024-09-17T21:29:53.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039709739000000001 2024-09-17T21:29:53.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:53.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182128+0000", 2024-09-17T21:29:53.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5739000000000002e-05 2024-09-17T21:29:53.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182210+0000", 2024-09-17T21:29:53.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:29:53.740 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.740 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.740 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.740 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12283 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:53.740 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:29:53.740 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5116147049999995, 2024-09-17T21:29:53.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039896434000000001, 2024-09-17T21:29:53.740 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:53.740 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:53.740 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:29:53.741 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:53.741 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:53.741 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:53.741 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12283 2024-09-17T21:29:53.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.741 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:53.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:53.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:29:53.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:53.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:29:53.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:53.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:53.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144852+0000", 2024-09-17T21:29:53.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024934079999999999 2024-09-17T21:29:53.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:53.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144856+0000", 2024-09-17T21:29:53.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:29:53.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:53.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144864+0000", 2024-09-17T21:29:53.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0160000000000008e-06 2024-09-17T21:29:53.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:53.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144869+0000", 2024-09-17T21:29:53.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:29:53.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:53.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182246+0000", 2024-09-17T21:29:53.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037377207000000003 2024-09-17T21:29:53.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:53.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:53.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:53.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182255+0000", 2024-09-17T21:29:53.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1840000000000005e-06 2024-09-17T21:29:53.744 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.744 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.744 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.744 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:53.744 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:53.744 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:53.746 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:29:53.746 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-17T21:29:53.747 INFO:teuthology.orchestra.run.smithi129.stdout:698 punch from 0x60f4d78 to 0x6101547, (0xc7cf bytes) 2024-09-17T21:29:53.747 INFO:teuthology.orchestra.run.smithi129.stdout:699 punch from 0x82332ea to 0x8234428, (0x113e bytes) 2024-09-17T21:29:53.747 INFO:teuthology.orchestra.run.smithi129.stdout:701 write 0x9cc8da5 thru 0x9cd291d (0x9b79 bytes) 2024-09-17T21:29:53.747 INFO:teuthology.orchestra.run.smithi129.stdout:702 write 0x80a4801 thru 0x80a6f9a (0x279a bytes) 2024-09-17T21:29:53.747 INFO:teuthology.orchestra.run.smithi129.stdout:704 read 0x32849ad thru 0x32856bf (0xd13 bytes) 2024-09-17T21:29:53.747 INFO:teuthology.orchestra.run.smithi129.stdout:705 write 0x99c07d7 thru 0x99ca53a (0x9d64 bytes) 2024-09-17T21:29:53.747 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:53.747 INFO:teuthology.orchestra.run.smithi129.stdout:707 write 0xe0ea25c thru 0xe0f3481 (0x9226 bytes) 2024-09-17T21:29:53.747 INFO:teuthology.orchestra.run.smithi129.stdout:708 read 0x5d67152 thru 0x5d6de38 (0x6ce7 bytes) 2024-09-17T21:29:53.747 INFO:teuthology.orchestra.run.smithi129.stdout:710 trunc from 0xe0f3482 to 0x33aa363 2024-09-17T21:29:53.751 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:53.748+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:29:53.767 INFO:teuthology.orchestra.run.smithi129.stdout:711 write 0x17b83ea thru 0x17c389c (0xb4b3 bytes) 2024-09-17T21:29:53.778 INFO:teuthology.orchestra.run.smithi129.stdout:712 trunc from 0x33aa363 to 0xb032d28 2024-09-17T21:29:53.783 INFO:teuthology.orchestra.run.smithi129.stdout:713 write 0x6e59010 thru 0x6e5a474 (0x1465 bytes) 2024-09-17T21:29:53.786 INFO:teuthology.orchestra.run.smithi129.stdout:714 read 0xa754d2b thru 0xa755094 (0x36a bytes) 2024-09-17T21:29:53.786 INFO:teuthology.orchestra.run.smithi129.stdout:715 read 0x1cd47ba thru 0x1ce3b07 (0xf34e bytes) 2024-09-17T21:29:53.786 INFO:teuthology.orchestra.run.smithi129.stdout:716 read 0x93f961e thru 0x94039d3 (0xa3b6 bytes) 2024-09-17T21:29:53.787 INFO:teuthology.orchestra.run.smithi129.stdout:717 read 0x26a3627 thru 0x26a88f5 (0x52cf bytes) 2024-09-17T21:29:53.787 INFO:teuthology.orchestra.run.smithi129.stdout:718 write 0x5c5dc56 thru 0x5c6381a (0x5bc5 bytes) 2024-09-17T21:29:53.789 INFO:teuthology.orchestra.run.smithi129.stdout:719 read 0x431f60b thru 0x432e596 (0xef8c bytes) 2024-09-17T21:29:53.798 INFO:teuthology.orchestra.run.smithi129.stdout:720 trunc from 0xb032d28 to 0x3d7ec0d 2024-09-17T21:29:53.821 INFO:teuthology.orchestra.run.smithi129.stdout:721 write 0x9987504 thru 0x99912f7 (0x9df4 bytes) 2024-09-17T21:29:53.827 INFO:teuthology.orchestra.run.smithi129.stdout:722 trunc from 0x99912f8 to 0x5c07f6a 2024-09-17T21:29:53.849 INFO:teuthology.orchestra.run.smithi129.stdout:723 write 0xe6197c9 thru 0xe6230cd (0x9905 bytes) 2024-09-17T21:29:53.852 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:53.850+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:53.853 INFO:teuthology.orchestra.run.smithi129.stdout:724 writesame 0xb56f758 thru 0xb578ef9 data_size 0xd(0x97a2 bytes) 2024-09-17T21:29:53.856 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:53.856 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:29:53.856 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:29:53.856 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:53.857 INFO:teuthology.orchestra.run.smithi129.stdout:726 trunc from 0xe6230ce to 0xd62ac4c 2024-09-17T21:29:53.865 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:29:53.869 INFO:teuthology.orchestra.run.smithi129.stdout:727 writesame 0xa8d45ba thru 0xa8d5bb5 data_size 0x2a(0x15fc bytes) 2024-09-17T21:29:53.870 INFO:teuthology.orchestra.run.smithi129.stdout:728 writesame 0x26da809 thru 0x26e8c2e data_size 0x3a(0xe426 bytes) 2024-09-17T21:29:53.872 INFO:teuthology.orchestra.run.smithi129.stdout:729 punch from 0x176270f to 0x1765107, (0x29f8 bytes) 2024-09-17T21:29:53.872 INFO:teuthology.orchestra.run.smithi129.stdout:730 read 0xd4b83f7 thru 0xd4c298a (0xa594 bytes) 2024-09-17T21:29:53.872 INFO:teuthology.orchestra.run.smithi129.stdout:731 read 0x1e44b20 thru 0x1e51264 (0xc745 bytes) 2024-09-17T21:29:53.873 INFO:teuthology.orchestra.run.smithi129.stdout:732 read 0xa8b2c1e thru 0xa8b52e0 (0x26c3 bytes) 2024-09-17T21:29:53.873 INFO:teuthology.orchestra.run.smithi129.stdout:733 writesame 0x84ae92c thru 0x84bce24 data_size 0x75(0xe4f9 bytes) 2024-09-17T21:29:53.874 INFO:teuthology.orchestra.run.smithi129.stdout:735 read 0x16654cd thru 0x1670eed (0xba21 bytes) 2024-09-17T21:29:53.874 INFO:teuthology.orchestra.run.smithi129.stdout:736 read 0x897f2b6 thru 0x898df90 (0xecdb bytes) 2024-09-17T21:29:53.875 INFO:teuthology.orchestra.run.smithi129.stdout:737 write 0xa468333 thru 0xa46f52a (0x71f8 bytes) 2024-09-17T21:29:53.876 INFO:teuthology.orchestra.run.smithi129.stdout:738 write 0x62af59a thru 0x62b7f5a (0x89c1 bytes) 2024-09-17T21:29:53.878 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:53.878 INFO:teuthology.orchestra.run.smithi129.stdout:740 punch from 0x4c9b7 to 0x58579, (0xbbc2 bytes) 2024-09-17T21:29:53.878 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:53.878 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:53.878 INFO:teuthology.orchestra.run.smithi129.stdout:744 writesame 0x84288df thru 0x843584e data_size 0x8(0xcf70 bytes) 2024-09-17T21:29:53.881 INFO:teuthology.orchestra.run.smithi129.stdout:745 write 0x7e61845 thru 0x7e6bfad (0xa769 bytes) 2024-09-17T21:29:53.892 INFO:teuthology.orchestra.run.smithi129.stdout:746 trunc from 0xd62ac4c to 0x5e40e54 2024-09-17T21:29:53.906 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:53.906 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:53.906 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:53.906 INFO:teuthology.orchestra.run.smithi129.stdout:752 writesame 0x38670d2 thru 0x38730c7 data_size 0x1(0xbff6 bytes) 2024-09-17T21:29:53.932 INFO:teuthology.orchestra.run.smithi129.stdout:753 trunc from 0x5e40e54 to 0x996ee47 2024-09-17T21:29:53.937 INFO:teuthology.orchestra.run.smithi129.stdout:754 read 0x3d63197 thru 0x3d67ade (0x4948 bytes) 2024-09-17T21:29:53.937 INFO:teuthology.orchestra.run.smithi129.stdout:755 writesame 0xf6466e thru 0xf6b3fc data_size 0x1(0x6d8f bytes) 2024-09-17T21:29:53.950 INFO:teuthology.orchestra.run.smithi129.stdout:756 punch from 0x92c9a12 to 0x92cae43, (0x1431 bytes) 2024-09-17T21:29:53.953 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:53.951+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:53.956 INFO:teuthology.orchestra.run.smithi129.stdout:757 write 0x9be137a thru 0x9bebecc (0xab53 bytes) 2024-09-17T21:29:53.958 INFO:teuthology.orchestra.run.smithi129.stdout:759 punch from 0x9524223 to 0x9530088, (0xbe65 bytes) 2024-09-17T21:29:53.965 INFO:teuthology.orchestra.run.smithi129.stdout:761 writesame 0xb49b66b thru 0xb4ab037 data_size 0x1(0xf9cd bytes) 2024-09-17T21:29:53.968 INFO:teuthology.orchestra.run.smithi129.stdout:762 read 0x68dfdf8 thru 0x68e1b2c (0x1d35 bytes) 2024-09-17T21:29:53.968 INFO:teuthology.orchestra.run.smithi129.stdout:763 read 0x2351bc9 thru 0x2359e2a (0x8262 bytes) 2024-09-17T21:29:53.968 INFO:teuthology.orchestra.run.smithi129.stdout:764 punch from 0x6adce24 to 0x6ae79f9, (0xabd5 bytes) 2024-09-17T21:29:53.974 INFO:teuthology.orchestra.run.smithi129.stdout:765 write 0xca274b5 thru 0xca2d111 (0x5c5d bytes) 2024-09-17T21:29:53.976 INFO:teuthology.orchestra.run.smithi129.stdout:766 read 0x76529af thru 0x7658fd2 (0x6624 bytes) 2024-09-17T21:29:53.976 INFO:teuthology.orchestra.run.smithi129.stdout:767 write 0x8d81d5a thru 0x8d91439 (0xf6e0 bytes) 2024-09-17T21:29:53.978 INFO:teuthology.orchestra.run.smithi129.stdout:768 read 0x32f6bd5 thru 0x32f846e (0x189a bytes) 2024-09-17T21:29:53.978 INFO:teuthology.orchestra.run.smithi129.stdout:769 read 0x97b7e01 thru 0x97badf9 (0x2ff9 bytes) 2024-09-17T21:29:53.991 INFO:teuthology.orchestra.run.smithi129.stdout:772 trunc from 0xca2d112 to 0x2489dee 2024-09-17T21:29:54.011 INFO:teuthology.orchestra.run.smithi129.stdout:773 punch from 0x5c4909 to 0x5d0cf6, (0xc3ed bytes) 2024-09-17T21:29:54.013 INFO:teuthology.orchestra.run.smithi129.stdout:774 trunc from 0x2489dee to 0x3e7dede 2024-09-17T21:29:54.013 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:29:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:29:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:29:54.014 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:54.019 INFO:teuthology.orchestra.run.smithi129.stdout:775 read 0x3b7bbd4 thru 0x3b7cc46 (0x1073 bytes) 2024-09-17T21:29:54.019 INFO:teuthology.orchestra.run.smithi129.stdout:777 read 0x1d21d thru 0x287ee (0xb5d2 bytes) 2024-09-17T21:29:54.020 INFO:teuthology.orchestra.run.smithi129.stdout:778 punch from 0x374b1c to 0x37ef03, (0xa3e7 bytes) 2024-09-17T21:29:54.020 INFO:teuthology.orchestra.run.smithi129.stdout:779 read 0x7fcd4b thru 0x80bf1e (0xf1d4 bytes) 2024-09-17T21:29:54.020 INFO:teuthology.orchestra.run.smithi129.stdout:781 write 0x37f6bca thru 0x37faaa3 (0x3eda bytes) 2024-09-17T21:29:54.022 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:29:54.037 INFO:teuthology.orchestra.run.smithi129.stdout:785 writesame 0x9b9fa8b thru 0x9ba8278 data_size 0x7f(0x87ee bytes) 2024-09-17T21:29:54.037 INFO:teuthology.orchestra.run.smithi129.stdout:786 read 0x86284d9 thru 0x862d75d (0x5285 bytes) 2024-09-17T21:29:54.040 INFO:teuthology.orchestra.run.smithi129.stdout:787 trunc from 0x9ba8279 to 0x831e69a 2024-09-17T21:29:54.049 INFO:teuthology.orchestra.run.smithi129.stdout:788 trunc from 0x831e69a to 0x55df3a0 2024-09-17T21:29:54.053 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:54.051+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:54.056 INFO:teuthology.orchestra.run.smithi129.stdout:789 write 0x21ebff8 thru 0x21ec07a (0x83 bytes) 2024-09-17T21:29:54.063 INFO:teuthology.orchestra.run.smithi129.stdout:790 write 0x6ac7924 thru 0x6ace421 (0x6afe bytes) 2024-09-17T21:29:54.070 INFO:teuthology.orchestra.run.smithi129.stdout:791 write 0x6b307c3 thru 0x6b36851 (0x608f bytes) 2024-09-17T21:29:54.071 INFO:teuthology.orchestra.run.smithi129.stdout:793 punch from 0x199d787 to 0x19a8750, (0xafc9 bytes) 2024-09-17T21:29:54.071 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:54.072 INFO:teuthology.orchestra.run.smithi129.stdout:796 punch from 0xc203c0 to 0xc20c20, (0x860 bytes) 2024-09-17T21:29:54.083 INFO:teuthology.orchestra.run.smithi129.stdout:797 writesame 0xc169b91 thru 0xc16da9f data_size 0x1(0x3f0f bytes) 2024-09-17T21:29:54.084 INFO:teuthology.orchestra.run.smithi129.stdout:798 writesame 0x92f2d57 thru 0x92ff5d0 data_size 0x2(0xc87a bytes) 2024-09-17T21:29:54.095 INFO:teuthology.orchestra.run.smithi129.stdout:799 punch from 0x56c88ee to 0x56d55f1, (0xcd03 bytes) 2024-09-17T21:29:54.095 INFO:teuthology.orchestra.run.smithi129.stdout:800 write 0x8cbccd7 thru 0x8cc8a7e (0xbda8 bytes) 2024-09-17T21:29:54.097 INFO:teuthology.orchestra.run.smithi129.stdout:801 writesame 0x99ea45 thru 0x9a0892 data_size 0x9(0x1e4e bytes) 2024-09-17T21:29:54.099 INFO:teuthology.orchestra.run.smithi129.stdout:804 write 0x6bbb167 thru 0x6bc1413 (0x62ad bytes) 2024-09-17T21:29:54.100 INFO:teuthology.orchestra.run.smithi129.stdout:806 writesame 0x4b4b10d thru 0x4b56952 data_size 0x2(0xb846 bytes) 2024-09-17T21:29:54.118 INFO:teuthology.orchestra.run.smithi129.stdout:807 trunc from 0xc16daa0 to 0x46c50b1 2024-09-17T21:29:54.128 INFO:teuthology.orchestra.run.smithi129.stdout:810 read 0x46ca90 thru 0x46e5bd (0x1b2e bytes) 2024-09-17T21:29:54.128 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:54.128 INFO:teuthology.orchestra.run.smithi129.stdout:812 read 0x44cb5d3 thru 0x44d34ae (0x7edc bytes) 2024-09-17T21:29:54.134 INFO:teuthology.orchestra.run.smithi129.stdout:814 write 0x59b71fe thru 0x59c4b97 (0xd99a bytes) 2024-09-17T21:29:54.140 INFO:teuthology.orchestra.run.smithi129.stdout:815 trunc from 0x59c4b98 to 0x924e298 2024-09-17T21:29:54.145 INFO:teuthology.orchestra.run.smithi129.stdout:817 read 0x478fd3a thru 0x4795190 (0x5457 bytes) 2024-09-17T21:29:54.145 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:54.145 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:54.154 INFO:teuthology.orchestra.run.smithi129.stdout:821 write 0xc654cc1 thru 0xc65e15c (0x949c bytes) 2024-09-17T21:29:54.154 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:54.152+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:54.155 INFO:teuthology.orchestra.run.smithi129.stdout:822 write 0x8857d30 thru 0x88595eb (0x18bc bytes) 2024-09-17T21:29:54.162 INFO:teuthology.orchestra.run.smithi129.stdout:824 trunc from 0xc65e15d to 0x78e41b3 2024-09-17T21:29:54.170 INFO:teuthology.orchestra.run.smithi129.stdout:825 punch from 0x13c744f to 0x13cbeb0, (0x4a61 bytes) 2024-09-17T21:29:54.170 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:54.171 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:54.171 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:29:54.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:29:54.171 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:29:54.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.172 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:2881 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:54.172 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.556463+0000", 2024-09-17T21:29:54.172 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 6.6120215350000002, 2024-09-17T21:29:54.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010359903, 2024-09-17T21:29:54.172 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.172 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.172 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:54.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.172 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 2881 2024-09-17T21:29:54.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.172 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556463+0000", 2024-09-17T21:29:54.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556463+0000", 2024-09-17T21:29:54.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556465+0000", 2024-09-17T21:29:54.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:29:54.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556485+0000", 2024-09-17T21:29:54.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9865999999999999e-05 2024-09-17T21:29:54.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556522+0000", 2024-09-17T21:29:54.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7126999999999999e-05 2024-09-17T21:29:54.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556544+0000", 2024-09-17T21:29:54.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2135999999999999e-05 2024-09-17T21:29:54.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556748+0000", 2024-09-17T21:29:54.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000204222 2024-09-17T21:29:54.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:54.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556829+0000", 2024-09-17T21:29:54.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0049999999999994e-05 2024-09-17T21:29:54.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:29:54.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557015+0000", 2024-09-17T21:29:54.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018693200000000001 2024-09-17T21:29:54.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:54.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557744+0000", 2024-09-17T21:29:54.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00072873299999999996 2024-09-17T21:29:54.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:54.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.566721+0000", 2024-09-17T21:29:54.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0089770119999999995 2024-09-17T21:29:54.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:54.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.566779+0000", 2024-09-17T21:29:54.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8041000000000002e-05 2024-09-17T21:29:54.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.566823+0000", 2024-09-17T21:29:54.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3343999999999997e-05 2024-09-17T21:29:54.181 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.181 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.182 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.182 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4746 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 1363968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:54.182 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.958850+0000", 2024-09-17T21:29:54.182 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 6.2096345629999998, 2024-09-17T21:29:54.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073491540000000001, 2024-09-17T21:29:54.183 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.183 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.183 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:54.183 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.184 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.184 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.184 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4746 2024-09-17T21:29:54.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.184 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958850+0000", 2024-09-17T21:29:54.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958850+0000", 2024-09-17T21:29:54.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958850+0000", 2024-09-17T21:29:54.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:29:54.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958852+0000", 2024-09-17T21:29:54.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.091e-06 2024-09-17T21:29:54.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958858+0000", 2024-09-17T21:29:54.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2349999999999996e-06 2024-09-17T21:29:54.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958863+0000", 2024-09-17T21:29:54.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.656e-06 2024-09-17T21:29:54.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966041+0000", 2024-09-17T21:29:54.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0071779510000000001 2024-09-17T21:29:54.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:54.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966122+0000", 2024-09-17T21:29:54.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0947999999999998e-05 2024-09-17T21:29:54.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966199+0000", 2024-09-17T21:29:54.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6843999999999998e-05 2024-09-17T21:29:54.191 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.191 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.191 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.191 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4747 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 1378304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:54.191 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.958869+0000", 2024-09-17T21:29:54.191 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 6.2096148260000001, 2024-09-17T21:29:54.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075441639999999999, 2024-09-17T21:29:54.191 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.192 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.192 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:54.192 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.192 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.192 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.192 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4747 2024-09-17T21:29:54.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.192 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958869+0000", 2024-09-17T21:29:54.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958869+0000", 2024-09-17T21:29:54.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958870+0000", 2024-09-17T21:29:54.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3399999999999999e-07 2024-09-17T21:29:54.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958872+0000", 2024-09-17T21:29:54.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:29:54.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958878+0000", 2024-09-17T21:29:54.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0430000000000004e-06 2024-09-17T21:29:54.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958883+0000", 2024-09-17T21:29:54.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8620000000000002e-06 2024-09-17T21:29:54.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966238+0000", 2024-09-17T21:29:54.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073557700000000002 2024-09-17T21:29:54.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:54.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966331+0000", 2024-09-17T21:29:54.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2048999999999999e-05 2024-09-17T21:29:54.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966414+0000", 2024-09-17T21:29:54.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:29:54.195 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.195 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.195 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.196 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4748 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 1390592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:54.196 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.958889+0000", 2024-09-17T21:29:54.196 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 6.2095950630000001, 2024-09-17T21:29:54.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077162630000000001, 2024-09-17T21:29:54.196 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.196 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.196 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:54.196 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.196 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.196 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.196 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4748 2024-09-17T21:29:54.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.197 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958889+0000", 2024-09-17T21:29:54.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958889+0000", 2024-09-17T21:29:54.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958890+0000", 2024-09-17T21:29:54.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:29:54.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958899+0000", 2024-09-17T21:29:54.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0699999999999996e-06 2024-09-17T21:29:54.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958905+0000", 2024-09-17T21:29:54.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2820000000000002e-06 2024-09-17T21:29:54.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958910+0000", 2024-09-17T21:29:54.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7570000000000004e-06 2024-09-17T21:29:54.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966459+0000", 2024-09-17T21:29:54.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075494009999999999 2024-09-17T21:29:54.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:54.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966550+0000", 2024-09-17T21:29:54.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0603e-05 2024-09-17T21:29:54.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966605+0000", 2024-09-17T21:29:54.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5754999999999997e-05 2024-09-17T21:29:54.200 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.200 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.200 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.200 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4749 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 1404928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:54.200 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.958916+0000", 2024-09-17T21:29:54.200 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 6.2095678830000001, 2024-09-17T21:29:54.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0078880389999999995, 2024-09-17T21:29:54.201 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.201 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.201 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:54.201 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.201 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.201 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.201 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4749 2024-09-17T21:29:54.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.201 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958916+0000", 2024-09-17T21:29:54.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958916+0000", 2024-09-17T21:29:54.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958917+0000", 2024-09-17T21:29:54.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4299999999999998e-07 2024-09-17T21:29:54.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958919+0000", 2024-09-17T21:29:54.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9130000000000001e-06 2024-09-17T21:29:54.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958930+0000", 2024-09-17T21:29:54.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1401e-05 2024-09-17T21:29:54.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958935+0000", 2024-09-17T21:29:54.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1479999999999997e-06 2024-09-17T21:29:54.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966649+0000", 2024-09-17T21:29:54.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077142110000000003 2024-09-17T21:29:54.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:54.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966725+0000", 2024-09-17T21:29:54.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5115000000000002e-05 2024-09-17T21:29:54.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966804+0000", 2024-09-17T21:29:54.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9808000000000006e-05 2024-09-17T21:29:54.205 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.205 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.205 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.205 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4750 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 1419264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:54.205 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.958942+0000", 2024-09-17T21:29:54.205 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 6.2095417900000003, 2024-09-17T21:29:54.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0080617380000000006, 2024-09-17T21:29:54.205 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.205 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.205 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:54.206 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.206 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.206 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.206 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4750 2024-09-17T21:29:54.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.206 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958942+0000", 2024-09-17T21:29:54.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958942+0000", 2024-09-17T21:29:54.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958943+0000", 2024-09-17T21:29:54.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1199999999999998e-07 2024-09-17T21:29:54.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958945+0000", 2024-09-17T21:29:54.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0269999999999998e-06 2024-09-17T21:29:54.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958951+0000", 2024-09-17T21:29:54.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9719999999999997e-06 2024-09-17T21:29:54.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958964+0000", 2024-09-17T21:29:54.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3078000000000001e-05 2024-09-17T21:29:54.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966848+0000", 2024-09-17T21:29:54.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0078837279999999996 2024-09-17T21:29:54.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:54.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966930+0000", 2024-09-17T21:29:54.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2352e-05 2024-09-17T21:29:54.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.967004+0000", 2024-09-17T21:29:54.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4169000000000001e-05 2024-09-17T21:29:54.209 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.209 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.209 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.210 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16081 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:54.210 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.330132+0000", 2024-09-17T21:29:54.210 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8383517630000004, 2024-09-17T21:29:54.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073045319999999999, 2024-09-17T21:29:54.210 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.210 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.210 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:54.210 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.210 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.210 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.210 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16081 2024-09-17T21:29:54.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.210 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.330132+0000", 2024-09-17T21:29:54.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.330132+0000", 2024-09-17T21:29:54.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.330133+0000", 2024-09-17T21:29:54.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:29:54.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.330136+0000", 2024-09-17T21:29:54.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8480000000000001e-06 2024-09-17T21:29:54.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.330141+0000", 2024-09-17T21:29:54.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.699e-06 2024-09-17T21:29:54.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.330146+0000", 2024-09-17T21:29:54.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2320000000000003e-06 2024-09-17T21:29:54.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.337342+0000", 2024-09-17T21:29:54.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0071963230000000001 2024-09-17T21:29:54.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:54.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.337380+0000", 2024-09-17T21:29:54.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8454999999999997e-05 2024-09-17T21:29:54.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.337437+0000", 2024-09-17T21:29:54.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6616000000000001e-05 2024-09-17T21:29:54.214 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.214 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.214 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.214 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16082 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:54.214 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.330151+0000", 2024-09-17T21:29:54.215 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8383327600000001, 2024-09-17T21:29:54.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075125319999999997, 2024-09-17T21:29:54.215 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.215 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.215 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:54.215 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.215 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.215 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.215 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16082 2024-09-17T21:29:54.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.215 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.330151+0000", 2024-09-17T21:29:54.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.330151+0000", 2024-09-17T21:29:54.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.330152+0000", 2024-09-17T21:29:54.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:29:54.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.330154+0000", 2024-09-17T21:29:54.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2230000000000001e-06 2024-09-17T21:29:54.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.330160+0000", 2024-09-17T21:29:54.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.84e-06 2024-09-17T21:29:54.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.330164+0000", 2024-09-17T21:29:54.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.352e-06 2024-09-17T21:29:54.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.337468+0000", 2024-09-17T21:29:54.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007303698 2024-09-17T21:29:54.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:54.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.337559+0000", 2024-09-17T21:29:54.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1423999999999997e-05 2024-09-17T21:29:54.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.337664+0000", 2024-09-17T21:29:54.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104598 2024-09-17T21:29:54.218 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.218 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.219 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.219 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16174 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:54.219 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.332031+0000", 2024-09-17T21:29:54.219 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8364527940000004, 2024-09-17T21:29:54.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073554650000000003, 2024-09-17T21:29:54.219 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.219 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.219 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:54.219 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.219 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.219 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.219 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16174 2024-09-17T21:29:54.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.220 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332031+0000", 2024-09-17T21:29:54.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332031+0000", 2024-09-17T21:29:54.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332032+0000", 2024-09-17T21:29:54.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:29:54.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332034+0000", 2024-09-17T21:29:54.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.012e-06 2024-09-17T21:29:54.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332040+0000", 2024-09-17T21:29:54.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.908e-06 2024-09-17T21:29:54.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332044+0000", 2024-09-17T21:29:54.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2899999999999996e-06 2024-09-17T21:29:54.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339213+0000", 2024-09-17T21:29:54.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0071694610000000002 2024-09-17T21:29:54.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:54.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339295+0000", 2024-09-17T21:29:54.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1806999999999998e-05 2024-09-17T21:29:54.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339387+0000", 2024-09-17T21:29:54.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1605000000000001e-05 2024-09-17T21:29:54.223 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.223 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.223 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.223 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16175 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 411648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:54.224 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.332051+0000", 2024-09-17T21:29:54.224 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8364332350000003, 2024-09-17T21:29:54.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075336270000000002, 2024-09-17T21:29:54.224 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.224 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.224 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:54.224 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.224 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.224 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.224 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16175 2024-09-17T21:29:54.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.225 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332051+0000", 2024-09-17T21:29:54.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332051+0000", 2024-09-17T21:29:54.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332051+0000", 2024-09-17T21:29:54.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T21:29:54.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332053+0000", 2024-09-17T21:29:54.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8199999999999999e-06 2024-09-17T21:29:54.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332059+0000", 2024-09-17T21:29:54.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.293e-06 2024-09-17T21:29:54.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332064+0000", 2024-09-17T21:29:54.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6210000000000001e-06 2024-09-17T21:29:54.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339418+0000", 2024-09-17T21:29:54.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073537250000000002 2024-09-17T21:29:54.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:54.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339497+0000", 2024-09-17T21:29:54.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9425000000000006e-05 2024-09-17T21:29:54.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339585+0000", 2024-09-17T21:29:54.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7321e-05 2024-09-17T21:29:54.228 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.229 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.229 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.229 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16176 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 423936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:54.229 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.332070+0000", 2024-09-17T21:29:54.229 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8364138700000003, 2024-09-17T21:29:54.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077110490000000002, 2024-09-17T21:29:54.229 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.229 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.229 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:54.229 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.230 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.230 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.230 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16176 2024-09-17T21:29:54.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.230 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332070+0000", 2024-09-17T21:29:54.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332070+0000", 2024-09-17T21:29:54.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332071+0000", 2024-09-17T21:29:54.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T21:29:54.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332073+0000", 2024-09-17T21:29:54.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8530000000000001e-06 2024-09-17T21:29:54.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332078+0000", 2024-09-17T21:29:54.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8409999999999996e-06 2024-09-17T21:29:54.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332083+0000", 2024-09-17T21:29:54.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8509999999999998e-06 2024-09-17T21:29:54.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339615+0000", 2024-09-17T21:29:54.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075321609999999999 2024-09-17T21:29:54.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:54.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339698+0000", 2024-09-17T21:29:54.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2663999999999993e-05 2024-09-17T21:29:54.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339781+0000", 2024-09-17T21:29:54.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3304000000000001e-05 2024-09-17T21:29:54.233 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.233 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.233 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.234 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16177 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:54.234 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.332089+0000", 2024-09-17T21:29:54.234 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.836394812, 2024-09-17T21:29:54.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079627510000000006, 2024-09-17T21:29:54.234 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.234 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.234 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:54.234 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.234 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.234 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.234 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16177 2024-09-17T21:29:54.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.234 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332089+0000", 2024-09-17T21:29:54.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332089+0000", 2024-09-17T21:29:54.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332090+0000", 2024-09-17T21:29:54.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:29:54.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332093+0000", 2024-09-17T21:29:54.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7310000000000002e-06 2024-09-17T21:29:54.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332100+0000", 2024-09-17T21:29:54.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5820000000000004e-06 2024-09-17T21:29:54.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332105+0000", 2024-09-17T21:29:54.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5220000000000001e-06 2024-09-17T21:29:54.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339810+0000", 2024-09-17T21:29:54.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007705796 2024-09-17T21:29:54.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:54.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339958+0000", 2024-09-17T21:29:54.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000147391 2024-09-17T21:29:54.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.340052+0000", 2024-09-17T21:29:54.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4313000000000003e-05 2024-09-17T21:29:54.238 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.238 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.238 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.238 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16178 3.a 3.150c2d0a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:54.238 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.332111+0000", 2024-09-17T21:29:54.238 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8363727609999998, 2024-09-17T21:29:54.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079742430000000007, 2024-09-17T21:29:54.238 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.238 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.238 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:29:54.238 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.238 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.238 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.238 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16178 2024-09-17T21:29:54.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.239 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332111+0000", 2024-09-17T21:29:54.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332111+0000", 2024-09-17T21:29:54.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332112+0000", 2024-09-17T21:29:54.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.58e-07 2024-09-17T21:29:54.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332117+0000", 2024-09-17T21:29:54.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4929999999999998e-06 2024-09-17T21:29:54.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332125+0000", 2024-09-17T21:29:54.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2169999999999997e-06 2024-09-17T21:29:54.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332130+0000", 2024-09-17T21:29:54.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1630000000000004e-06 2024-09-17T21:29:54.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.340080+0000", 2024-09-17T21:29:54.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079505689999999993 2024-09-17T21:29:54.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.340086+0000", 2024-09-17T21:29:54.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3430000000000003e-06 2024-09-17T21:29:54.242 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.242 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.242 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.242 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16179 3.a 3.150c2d0a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:54.242 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.332136+0000", 2024-09-17T21:29:54.242 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8363478339999997, 2024-09-17T21:29:54.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079794670000000005, 2024-09-17T21:29:54.242 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.243 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.243 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:29:54.243 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.243 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.243 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.243 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16179 2024-09-17T21:29:54.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.243 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332136+0000", 2024-09-17T21:29:54.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332136+0000", 2024-09-17T21:29:54.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332137+0000", 2024-09-17T21:29:54.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0900000000000002e-07 2024-09-17T21:29:54.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332139+0000", 2024-09-17T21:29:54.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0549999999999998e-06 2024-09-17T21:29:54.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332145+0000", 2024-09-17T21:29:54.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.088e-06 2024-09-17T21:29:54.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332150+0000", 2024-09-17T21:29:54.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6399999999999996e-06 2024-09-17T21:29:54.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.340111+0000", 2024-09-17T21:29:54.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079615510000000007 2024-09-17T21:29:54.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.340116+0000", 2024-09-17T21:29:54.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.724e-06 2024-09-17T21:29:54.246 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.246 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.246 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.246 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16180 3.a 3.150c2d0a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:54.246 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.332156+0000", 2024-09-17T21:29:54.246 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8363283849999998, 2024-09-17T21:29:54.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079971169999999998, 2024-09-17T21:29:54.247 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.247 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.247 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:29:54.247 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.247 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.247 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.247 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16180 2024-09-17T21:29:54.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.247 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332156+0000", 2024-09-17T21:29:54.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332156+0000", 2024-09-17T21:29:54.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332156+0000", 2024-09-17T21:29:54.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:29:54.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332158+0000", 2024-09-17T21:29:54.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.846e-06 2024-09-17T21:29:54.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332165+0000", 2024-09-17T21:29:54.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9970000000000001e-06 2024-09-17T21:29:54.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332170+0000", 2024-09-17T21:29:54.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7859999999999996e-06 2024-09-17T21:29:54.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.340144+0000", 2024-09-17T21:29:54.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079745760000000006 2024-09-17T21:29:54.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.340153+0000", 2024-09-17T21:29:54.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5329999999999994e-06 2024-09-17T21:29:54.250 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.250 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.250 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.250 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16468 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 610304~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:54.251 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.342505+0000", 2024-09-17T21:29:54.251 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8259789050000004, 2024-09-17T21:29:54.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007360213, 2024-09-17T21:29:54.251 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.251 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.251 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:54.251 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.251 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.251 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.251 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16468 2024-09-17T21:29:54.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.251 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342505+0000", 2024-09-17T21:29:54.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342505+0000", 2024-09-17T21:29:54.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342506+0000", 2024-09-17T21:29:54.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5400000000000002e-07 2024-09-17T21:29:54.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342508+0000", 2024-09-17T21:29:54.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.88e-06 2024-09-17T21:29:54.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342513+0000", 2024-09-17T21:29:54.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5670000000000002e-06 2024-09-17T21:29:54.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342518+0000", 2024-09-17T21:29:54.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6619999999999996e-06 2024-09-17T21:29:54.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.349747+0000", 2024-09-17T21:29:54.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0072289060000000002 2024-09-17T21:29:54.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:54.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.349801+0000", 2024-09-17T21:29:54.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4326000000000001e-05 2024-09-17T21:29:54.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.349866+0000", 2024-09-17T21:29:54.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4517999999999995e-05 2024-09-17T21:29:54.254 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.255 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.255 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.255 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16469 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 624640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:54.255 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.342549+0000", 2024-09-17T21:29:54.255 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8259350149999998, 2024-09-17T21:29:54.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0074639520000000003, 2024-09-17T21:29:54.255 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.255 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.255 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:54.255 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.255 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.255 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.256 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16469 2024-09-17T21:29:54.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.256 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342549+0000", 2024-09-17T21:29:54.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342549+0000", 2024-09-17T21:29:54.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342550+0000", 2024-09-17T21:29:54.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9299999999999998e-07 2024-09-17T21:29:54.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342552+0000", 2024-09-17T21:29:54.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4509999999999999e-06 2024-09-17T21:29:54.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342559+0000", 2024-09-17T21:29:54.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2389999999999996e-06 2024-09-17T21:29:54.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342565+0000", 2024-09-17T21:29:54.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5849999999999999e-06 2024-09-17T21:29:54.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.349892+0000", 2024-09-17T21:29:54.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073271650000000001 2024-09-17T21:29:54.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:54.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.349949+0000", 2024-09-17T21:29:54.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6854999999999997e-05 2024-09-17T21:29:54.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350013+0000", 2024-09-17T21:29:54.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4064000000000002e-05 2024-09-17T21:29:54.259 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.259 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.259 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.259 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16470 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 636928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:54.259 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.342573+0000", 2024-09-17T21:29:54.259 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8259112650000002, 2024-09-17T21:29:54.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075758229999999998, 2024-09-17T21:29:54.259 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.259 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.260 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:54.260 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.260 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.260 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.260 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16470 2024-09-17T21:29:54.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.260 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342573+0000", 2024-09-17T21:29:54.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342573+0000", 2024-09-17T21:29:54.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342573+0000", 2024-09-17T21:29:54.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6800000000000001e-07 2024-09-17T21:29:54.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342575+0000", 2024-09-17T21:29:54.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.984e-06 2024-09-17T21:29:54.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342581+0000", 2024-09-17T21:29:54.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8690000000000001e-06 2024-09-17T21:29:54.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342586+0000", 2024-09-17T21:29:54.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4889999999999997e-06 2024-09-17T21:29:54.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350046+0000", 2024-09-17T21:29:54.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0074604249999999997 2024-09-17T21:29:54.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:54.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350097+0000", 2024-09-17T21:29:54.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0464999999999999e-05 2024-09-17T21:29:54.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350149+0000", 2024-09-17T21:29:54.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2123000000000002e-05 2024-09-17T21:29:54.263 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.263 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.263 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.264 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16471 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 651264~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:54.264 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.342592+0000", 2024-09-17T21:29:54.264 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8258922039999996, 2024-09-17T21:29:54.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007701856, 2024-09-17T21:29:54.264 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.264 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.264 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:54.264 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.264 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.264 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.264 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16471 2024-09-17T21:29:54.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.264 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342592+0000", 2024-09-17T21:29:54.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342592+0000", 2024-09-17T21:29:54.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342592+0000", 2024-09-17T21:29:54.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:29:54.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342594+0000", 2024-09-17T21:29:54.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.906e-06 2024-09-17T21:29:54.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342600+0000", 2024-09-17T21:29:54.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8039999999999998e-06 2024-09-17T21:29:54.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342605+0000", 2024-09-17T21:29:54.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4649999999999996e-06 2024-09-17T21:29:54.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350178+0000", 2024-09-17T21:29:54.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075734920000000002 2024-09-17T21:29:54.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:54.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350240+0000", 2024-09-17T21:29:54.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2352000000000002e-05 2024-09-17T21:29:54.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350294+0000", 2024-09-17T21:29:54.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3431999999999999e-05 2024-09-17T21:29:54.268 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.268 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.268 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.268 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16472 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 665600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:54.268 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.342611+0000", 2024-09-17T21:29:54.268 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8258737050000002, 2024-09-17T21:29:54.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0078017429999999999, 2024-09-17T21:29:54.268 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:54.268 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:54.268 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:54.268 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:54.269 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:54.269 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:54.269 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16472 2024-09-17T21:29:54.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.269 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:54.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:54.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342611+0000", 2024-09-17T21:29:54.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:54.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342611+0000", 2024-09-17T21:29:54.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:54.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:54.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342611+0000", 2024-09-17T21:29:54.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:29:54.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:54.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342614+0000", 2024-09-17T21:29:54.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9510000000000002e-06 2024-09-17T21:29:54.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:54.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342620+0000", 2024-09-17T21:29:54.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1e-06 2024-09-17T21:29:54.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:54.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342631+0000", 2024-09-17T21:29:54.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1467999999999999e-05 2024-09-17T21:29:54.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:54.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350320+0000", 2024-09-17T21:29:54.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007688388 2024-09-17T21:29:54.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:54.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350360+0000", 2024-09-17T21:29:54.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9987999999999997e-05 2024-09-17T21:29:54.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:54.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:54.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:54.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350412+0000", 2024-09-17T21:29:54.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2437e-05 2024-09-17T21:29:54.272 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.272 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.272 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.272 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:54.272 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:54.272 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:54.274 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:29:54.276 INFO:teuthology.orchestra.run.smithi129.stdout:829 write 0x94a788d thru 0x94ade08 (0x657c bytes) 2024-09-17T21:29:54.276 INFO:teuthology.orchestra.run.smithi129.stdout:832 punch from 0x572c416 to 0x573348c, (0x7076 bytes) 2024-09-17T21:29:54.276 INFO:teuthology.orchestra.run.smithi129.stdout:833 trunc from 0x94ade09 to 0x61102d3 2024-09-17T21:29:54.276 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:54.276 INFO:teuthology.orchestra.run.smithi129.stdout:836 write 0x2262f1e thru 0x22657fa (0x28dd bytes) 2024-09-17T21:29:54.276 INFO:teuthology.orchestra.run.smithi129.stdout:837 punch from 0x1db425 to 0x1e3581, (0x815c bytes) 2024-09-17T21:29:54.276 INFO:teuthology.orchestra.run.smithi129.stdout:839 write 0xe72194f thru 0xe728e1f (0x74d1 bytes) 2024-09-17T21:29:54.277 INFO:teuthology.orchestra.run.smithi129.stdout:842 punch from 0x1c63101 to 0x1c66527, (0x3426 bytes) 2024-09-17T21:29:54.277 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:54.277 INFO:teuthology.orchestra.run.smithi129.stdout:845 write 0xa669ead thru 0xa66a0e6 (0x23a bytes) 2024-09-17T21:29:54.277 INFO:teuthology.orchestra.run.smithi129.stdout:846 read 0x7e70d9a thru 0x7e714a7 (0x70e bytes) 2024-09-17T21:29:54.277 INFO:teuthology.orchestra.run.smithi129.stdout:848 write 0x61e3b3e thru 0x61eb27e (0x7741 bytes) 2024-09-17T21:29:54.277 INFO:teuthology.orchestra.run.smithi129.stdout:849 punch from 0xcd95a7b to 0xcd9b34a, (0x58cf bytes) 2024-09-17T21:29:54.277 INFO:teuthology.orchestra.run.smithi129.stdout:851 read 0x933a33f thru 0x933e391 (0x4053 bytes) 2024-09-17T21:29:54.277 INFO:teuthology.orchestra.run.smithi129.stdout:852 write 0x185db1d thru 0x18693e4 (0xb8c8 bytes) 2024-09-17T21:29:54.277 INFO:teuthology.orchestra.run.smithi129.stdout:853 read 0x21f2e8b thru 0x21fe16a (0xb2e0 bytes) 2024-09-17T21:29:54.277 INFO:teuthology.orchestra.run.smithi129.stdout:854 writesame 0x1108da2 thru 0x1113f21 data_size 0x50(0xb180 bytes) 2024-09-17T21:29:54.277 INFO:teuthology.orchestra.run.smithi129.stdout:855 writesame 0xb0691cb thru 0xb071576 data_size 0x6a(0x83ac bytes) 2024-09-17T21:29:54.277 INFO:teuthology.orchestra.run.smithi129.stdout:856 read 0x4528fd0 thru 0x4537b11 (0xeb42 bytes) 2024-09-17T21:29:54.277 INFO:teuthology.orchestra.run.smithi129.stdout:860 trunc from 0xe728e20 to 0x545192d 2024-09-17T21:29:54.277 INFO:teuthology.orchestra.run.smithi129.stdout:862 punch from 0x35d0be5 to 0x35d1962, (0xd7d bytes) 2024-09-17T21:29:54.278 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:54.278 INFO:teuthology.orchestra.run.smithi129.stdout:865 trunc from 0x545192d to 0x9d3118 2024-09-17T21:29:54.278 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:54.271+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:54.278 INFO:teuthology.orchestra.run.smithi129.stdout:866 trunc from 0x9d3118 to 0x9bcb908 2024-09-17T21:29:54.290 INFO:teuthology.orchestra.run.smithi129.stdout:867 writesame 0xc8a7890 thru 0xc8ae99e data_size 0x1(0x710f bytes) 2024-09-17T21:29:54.290 INFO:teuthology.orchestra.run.smithi129.stdout:868 read 0x6876656 thru 0x687a15b (0x3b06 bytes) 2024-09-17T21:29:54.290 INFO:teuthology.orchestra.run.smithi129.stdout:870 writesame 0xad726e1 thru 0xad7f3d7 data_size 0x1(0xccf7 bytes) 2024-09-17T21:29:54.298 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:29:54.313 INFO:teuthology.orchestra.run.smithi129.stdout:871 read 0x4c2f1ab thru 0x4c38326 (0x917c bytes) 2024-09-17T21:29:54.325 INFO:teuthology.orchestra.run.smithi129.stdout:872 trunc from 0xc8ae99f to 0x303638b 2024-09-17T21:29:54.331 INFO:teuthology.orchestra.run.smithi129.stdout:875 read 0x2239db3 thru 0x22499c5 (0xfc13 bytes) 2024-09-17T21:29:54.344 INFO:teuthology.orchestra.run.smithi129.stdout:876 write 0x7e622e8 thru 0x7e6886b (0x6584 bytes) 2024-09-17T21:29:54.346 INFO:teuthology.orchestra.run.smithi129.stdout:877 read 0x456cf80 thru 0x457c2de (0xf35f bytes) 2024-09-17T21:29:54.347 INFO:teuthology.orchestra.run.smithi129.stdout:878 write 0x32051ab thru 0x320e3a4 (0x91fa bytes) 2024-09-17T21:29:54.350 INFO:teuthology.orchestra.run.smithi129.stdout:879 punch from 0x4bfca55 to 0x4c095d5, (0xcb80 bytes) 2024-09-17T21:29:54.352 INFO:teuthology.orchestra.run.smithi129.stdout:880 trunc from 0x7e6886c to 0x65c9065 2024-09-17T21:29:54.366 INFO:teuthology.orchestra.run.smithi129.stdout:881 writesame 0x7bd2e65 thru 0x7be26fb data_size 0x9(0xf897 bytes) 2024-09-17T21:29:54.367 INFO:teuthology.orchestra.run.smithi129.stdout:882 read 0x2df8d35 thru 0x2e05547 (0xc813 bytes) 2024-09-17T21:29:54.374 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:54.372+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:54.381 INFO:teuthology.orchestra.run.smithi129.stdout:883 write 0xee5db70 thru 0xee66de2 (0x9273 bytes) 2024-09-17T21:29:54.383 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:54.383 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:29:54.383 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:29:54.383 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:54.384 INFO:teuthology.orchestra.run.smithi129.stdout:884 writesame 0xc4135b8 thru 0xc41bfd6 data_size 0x1(0x8a1f bytes) 2024-09-17T21:29:54.393 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:29:54.401 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:54.401 INFO:teuthology.orchestra.run.smithi129.stdout:889 writesame 0x5a7066 thru 0x5a9a32 data_size 0x29(0x29cd bytes) 2024-09-17T21:29:54.402 INFO:teuthology.orchestra.run.smithi129.stdout:890 read 0x6e0e87e thru 0x6e10041 (0x17c4 bytes) 2024-09-17T21:29:54.402 INFO:teuthology.orchestra.run.smithi129.stdout:891 punch from 0xc5dd011 to 0xc5ea9a0, (0xd98f bytes) 2024-09-17T21:29:54.403 INFO:teuthology.orchestra.run.smithi129.stdout:892 write 0x36bd681 thru 0x36c307a (0x59fa bytes) 2024-09-17T21:29:54.405 INFO:teuthology.orchestra.run.smithi129.stdout:893 write 0xa9cec5 thru 0xa9cfc5 (0x101 bytes) 2024-09-17T21:29:54.407 INFO:teuthology.orchestra.run.smithi129.stdout:894 writesame 0x24eb9d2 thru 0x24f8b4e data_size 0x1(0xd17d bytes) 2024-09-17T21:29:54.416 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:29:54.436 INFO:teuthology.orchestra.run.smithi129.stdout:896 read 0xddf2241 thru 0xddff510 (0xd2d0 bytes) 2024-09-17T21:29:54.437 INFO:teuthology.orchestra.run.smithi129.stdout:898 write 0x7d3010e thru 0x7d3662f (0x6522 bytes) 2024-09-17T21:29:54.439 INFO:teuthology.orchestra.run.smithi129.stdout:899 write 0x543182f thru 0x543eb1e (0xd2f0 bytes) 2024-09-17T21:29:54.446 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:29:54.446 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:29:54.452 INFO:teuthology.orchestra.run.smithi129.stdout:901 trunc from 0xee66de3 to 0x548d917 2024-09-17T21:29:54.465 INFO:teuthology.orchestra.run.smithi129.stdout:905 write 0x4295cd3 thru 0x429d97e (0x7cac bytes) 2024-09-17T21:29:54.466 INFO:teuthology.orchestra.run.smithi129.stdout:906 read 0x24632ba thru 0x246ad03 (0x7a4a bytes) 2024-09-17T21:29:54.468 INFO:teuthology.orchestra.run.smithi129.stdout:907 trunc from 0x548d917 to 0x472f36f 2024-09-17T21:29:54.469 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:29:54.469 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:29:54.475 INFO:teuthology.orchestra.run.smithi129.stdout:908 punch from 0x43d2270 to 0x43d5f89, (0x3d19 bytes) 2024-09-17T21:29:54.475 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:54.475 INFO:teuthology.orchestra.run.smithi129.stdout:911 punch from 0x2d9e36f to 0x2da4a9b, (0x672c bytes) 2024-09-17T21:29:54.475 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:54.473+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:29:54.483 INFO:teuthology.orchestra.run.smithi129.stdout:912 write 0x61f087d thru 0x61f96a0 (0x8e24 bytes) 2024-09-17T21:29:54.486 INFO:teuthology.orchestra.run.smithi129.stdout:913 read 0x5707ff2 thru 0x570c10c (0x411b bytes) 2024-09-17T21:29:54.486 INFO:teuthology.orchestra.run.smithi129.stdout:914 write 0x3d6bcd2 thru 0x3d7a4db (0xe80a bytes) 2024-09-17T21:29:54.489 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:29:54.489 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:29:54.496 INFO:teuthology.orchestra.run.smithi129.stdout:915 trunc from 0x61f96a1 to 0xc134547 2024-09-17T21:29:54.500 INFO:teuthology.orchestra.run.smithi129.stdout:916 write 0x87f377 thru 0x886982 (0x760c bytes) 2024-09-17T21:29:54.502 INFO:teuthology.orchestra.run.smithi129.stdout:918 write 0x3b07ead thru 0x3b0ea17 (0x6b6b bytes) 2024-09-17T21:29:54.503 INFO:teuthology.orchestra.run.smithi129.stdout:919 punch from 0x5ccbbd2 to 0x5cd7cc2, (0xc0f0 bytes) 2024-09-17T21:29:54.504 INFO:teuthology.orchestra.run.smithi129.stdout:920 write 0x4f71744 thru 0x4f71c3f (0x4fc bytes) 2024-09-17T21:29:54.508 INFO:teuthology.orchestra.run.smithi129.stdout:922 trunc from 0xc134547 to 0xdcbafab 2024-09-17T21:29:54.509 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:29:54.509 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:29:54.513 INFO:teuthology.orchestra.run.smithi129.stdout:924 read 0xdb573f5 thru 0xdb5e069 (0x6c75 bytes) 2024-09-17T21:29:54.514 INFO:teuthology.orchestra.run.smithi129.stdout:925 write 0x2532ebb thru 0x253a705 (0x784b bytes) 2024-09-17T21:29:54.515 INFO:teuthology.orchestra.run.smithi129.stdout:926 read 0x984df4e thru 0x984e202 (0x2b5 bytes) 2024-09-17T21:29:54.516 INFO:teuthology.orchestra.run.smithi129.stdout:927 read 0xc67113 thru 0xc7137e (0xa26c bytes) 2024-09-17T21:29:54.516 INFO:teuthology.orchestra.run.smithi129.stdout:928 punch from 0xb49f8a7 to 0xb4af4ce, (0xfc27 bytes) 2024-09-17T21:29:54.524 INFO:teuthology.orchestra.run.smithi129.stdout:929 write 0xec07f7c thru 0xec14e0f (0xce94 bytes) 2024-09-17T21:29:54.530 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:29:54.530 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:29:54.532 INFO:teuthology.orchestra.run.smithi129.stdout:930 trunc from 0xec14e10 to 0xa810c4b 2024-09-17T21:29:54.543 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:54.543 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:29:54.543 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:29:54.543 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:29:54.543 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:54.546 INFO:teuthology.orchestra.run.smithi129.stdout:931 read 0x579605b thru 0x5798520 (0x24c6 bytes) 2024-09-17T21:29:54.546 INFO:teuthology.orchestra.run.smithi129.stdout:932 write 0x32c1d61 thru 0x32cdc0c (0xbeac bytes) 2024-09-17T21:29:54.548 INFO:teuthology.orchestra.run.smithi129.stdout:934 punch from 0x4b9a6e2 to 0x4ba7036, (0xc954 bytes) 2024-09-17T21:29:54.548 INFO:teuthology.orchestra.run.smithi129.stdout:935 write 0x5d10bb1 thru 0x5d1863a (0x7a8a bytes) 2024-09-17T21:29:54.549 INFO:teuthology.orchestra.run.smithi129.stdout:937 write 0x6116f1a thru 0x6121d0d (0xadf4 bytes) 2024-09-17T21:29:54.551 INFO:teuthology.orchestra.run.smithi129.stdout:939 write 0x1077e66 thru 0x10830ed (0xb288 bytes) 2024-09-17T21:29:54.552 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:29:54.552 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:29:54.555 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:29:54.561 INFO:teuthology.orchestra.run.smithi129.stdout:940 write 0xb187693 thru 0xb189525 (0x1e93 bytes) 2024-09-17T21:29:54.570 INFO:teuthology.orchestra.run.smithi129.stdout:941 write 0xd307e40 thru 0xd308906 (0xac7 bytes) 2024-09-17T21:29:54.571 INFO:teuthology.orchestra.run.smithi129.stdout:942 writesame 0x24274b4 thru 0x242b878 data_size 0x1(0x43c5 bytes) 2024-09-17T21:29:54.572 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:29:54.572 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:29:54.579 INFO:teuthology.orchestra.run.smithi129.stdout:943 read 0x56f1715 thru 0x56f20a3 (0x98f bytes) 2024-09-17T21:29:54.579 INFO:teuthology.orchestra.run.smithi129.stdout:944 writesame 0x2bca70f thru 0x2bce5db data_size 0x17(0x3ecd bytes) 2024-09-17T21:29:54.581 INFO:teuthology.orchestra.run.smithi129.stdout:946 write 0xbef15e8 thru 0xbeff84f (0xe268 bytes) 2024-09-17T21:29:54.589 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:29:54.589 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:29:54.590 INFO:teuthology.orchestra.run.smithi129.stdout:948 trunc from 0xd308907 to 0x6c84319 2024-09-17T21:29:54.598 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:54.596+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:54.600 INFO:teuthology.orchestra.run.smithi129.stdout:949 writesame 0x3af411f thru 0x3b037ec data_size 0x7(0xf6ce bytes) 2024-09-17T21:29:54.605 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:54.620 INFO:teuthology.orchestra.run.smithi129.stdout:951 writesame 0xe854158 thru 0xe85be39 data_size 0x73(0x7ce2 bytes) 2024-09-17T21:29:54.621 INFO:teuthology.orchestra.run.smithi129.stdout:952 read 0x1d728bf thru 0x1d80a23 (0xe165 bytes) 2024-09-17T21:29:54.621 INFO:teuthology.orchestra.run.smithi129.stdout:953 write 0x6558a30 thru 0x655d5bc (0x4b8d bytes) 2024-09-17T21:29:54.625 INFO:teuthology.orchestra.run.smithi129.stdout:956 read 0x7bcf1c0 thru 0x7bd50c0 (0x5f01 bytes) 2024-09-17T21:29:54.625 INFO:teuthology.orchestra.run.smithi129.stdout:957 read 0x5c84a75 thru 0x5c9243e (0xd9ca bytes) 2024-09-17T21:29:54.626 INFO:teuthology.orchestra.run.smithi129.stdout:958 write 0x4763acb thru 0x4767341 (0x3877 bytes) 2024-09-17T21:29:54.627 INFO:teuthology.orchestra.run.smithi129.stdout:960 clone 7 order 25 su 4194304 sc 6 2024-09-17T21:29:54.662 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:54.662 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:54.670 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:29:54.714 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:54.712+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:54.775 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:54.775 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:54.778 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:54.776+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:54.784 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:29:54.879 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:54.877+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:54.928 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:54.928 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:54.937 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:29:54.980 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:54.978+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:55.040 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:55.040 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:55.048 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:29:55.081 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:55.078+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:55.151 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:55.151 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:55.159 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:29:55.203 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:55.201+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:29:55.262 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:55.262 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:55.271 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:29:55.311 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:55.309+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:29:55.384 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:55.381+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:55.413 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:55.414 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:55.423 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:29:55.506 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:55.504+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:55.526 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is set 2024-09-17T21:29:55.579 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:55.579 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:55.585 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:55.583+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:55.589 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:29:55.593 INFO:teuthology.orchestra.run.smithi129.stdout:leaving image image_client.0-clone6 intact 2024-09-17T21:29:55.687 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:55.684+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:55.701 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:55.701 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:55.710 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:29:55.751 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-17T21:29:55.787 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:55.785+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:55.825 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:55.825 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:55.835 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:29:55.888 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:55.886+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:55.918 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:55.918 INFO:teuthology.orchestra.run.smithi129.stdout:963 read 0x514ebed thru 0x515cb14 (0xdf28 bytes) 2024-09-17T21:29:55.919 INFO:teuthology.orchestra.run.smithi129.stdout:964 read 0xa22d07e thru 0xa23cff6 (0xff79 bytes) 2024-09-17T21:29:55.919 INFO:teuthology.orchestra.run.smithi129.stdout:965 punch from 0x4db3ee9 to 0x4db4a95, (0xbac bytes) 2024-09-17T21:29:55.923 INFO:teuthology.orchestra.run.smithi129.stdout:966 writesame 0x8cf61ad thru 0x8cf67f3 data_size 0x1(0x647 bytes) 2024-09-17T21:29:55.928 INFO:teuthology.orchestra.run.smithi129.stdout:967 write 0x4346df9 thru 0x4354ba4 (0xddac bytes) 2024-09-17T21:29:55.933 INFO:teuthology.orchestra.run.smithi129.stdout:968 read 0x39dbcd3 thru 0x39dee2f (0x315d bytes) 2024-09-17T21:29:55.934 INFO:teuthology.orchestra.run.smithi129.stdout:970 read 0x6d92830 thru 0x6d9b544 (0x8d15 bytes) 2024-09-17T21:29:55.934 INFO:teuthology.orchestra.run.smithi129.stdout:971 write 0xbfe6072 thru 0xbfe6089 (0x18 bytes) 2024-09-17T21:29:55.935 INFO:teuthology.orchestra.run.smithi129.stdout:972 read 0x5f09d62 thru 0x5f1350e (0x97ad bytes) 2024-09-17T21:29:55.946 INFO:teuthology.orchestra.run.smithi129.stdout:974 trunc from 0xe85be3a to 0x6194947 2024-09-17T21:29:55.963 INFO:teuthology.orchestra.run.smithi129.stdout:975 trunc from 0x6194947 to 0x5f7e16f 2024-09-17T21:29:55.970 INFO:teuthology.orchestra.run.smithi129.stdout:977 write 0x22c9ffe thru 0x22d3487 (0x948a bytes) 2024-09-17T21:29:55.976 INFO:teuthology.orchestra.run.smithi129.stdout:978 trunc from 0x5f7e16f to 0x2bb08ae 2024-09-17T21:29:55.984 INFO:teuthology.orchestra.run.smithi129.stdout:979 trunc from 0x2bb08ae to 0x3e9c4e2 2024-09-17T21:29:55.989 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:55.989 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:55.989 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:55.987+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:55.996 INFO:teuthology.orchestra.run.smithi129.stdout:980 writesame 0x4f824d3 thru 0x4f8e267 data_size 0x1(0xbd95 bytes) 2024-09-17T21:29:55.998 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.001 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:29:56.005 INFO:teuthology.orchestra.run.smithi129.stdout:982 write 0x6c51865 thru 0x6c5b16d (0x9909 bytes) 2024-09-17T21:29:56.013 INFO:teuthology.orchestra.run.smithi129.stdout:983 trunc from 0x6c5b16e to 0xc2028bb 2024-09-17T21:29:56.027 INFO:teuthology.orchestra.run.smithi129.stdout:986 writesame 0xe582c88 thru 0xe588bec data_size 0x1(0x5f65 bytes) 2024-09-17T21:29:56.027 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.027 INFO:teuthology.orchestra.run.smithi129.stdout:988 write 0xc211339 thru 0xc21f152 (0xde1a bytes) 2024-09-17T21:29:56.030 INFO:teuthology.orchestra.run.smithi129.stdout:989 writesame 0xd1d4f2 thru 0xd239bf data_size 0x33(0x64ce bytes) 2024-09-17T21:29:56.032 INFO:teuthology.orchestra.run.smithi129.stdout:991 punch from 0xb99ab35 to 0xb99c880, (0x1d4b bytes) 2024-09-17T21:29:56.032 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.032 INFO:teuthology.orchestra.run.smithi129.stdout:995 write 0x95df22e thru 0x95e7f33 (0x8d06 bytes) 2024-09-17T21:29:56.033 INFO:teuthology.orchestra.run.smithi129.stdout:998 write 0x74d9a3b thru 0x74dcad2 (0x3098 bytes) 2024-09-17T21:29:56.035 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.035 INFO:teuthology.orchestra.run.smithi129.stdout:1000 read 0x978a32e thru 0x978a8be (0x591 bytes) 2024-09-17T21:29:56.035 INFO:teuthology.orchestra.run.smithi129.stdout:1003 write 0x43ca042 thru 0x43d3381 (0x9340 bytes) 2024-09-17T21:29:56.037 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.037 INFO:teuthology.orchestra.run.smithi129.stdout:1006 punch from 0x28b5db to 0x29521a, (0x9c3f bytes) 2024-09-17T21:29:56.037 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.037 INFO:teuthology.orchestra.run.smithi129.stdout:1008 writesame 0xa7fc7e6 thru 0xa806ddb data_size 0x49(0xa5f6 bytes) 2024-09-17T21:29:56.038 INFO:teuthology.orchestra.run.smithi129.stdout:1009 writesame 0xbffa36a thru 0xbffeb29 data_size 0x20(0x47c0 bytes) 2024-09-17T21:29:56.045 INFO:teuthology.orchestra.run.smithi129.stdout:1010 trunc from 0xe588bed to 0xa0e7ecf 2024-09-17T21:29:56.059 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.059 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.059 INFO:teuthology.orchestra.run.smithi129.stdout:1013 read 0x5fe8ad7 thru 0x5febc5c (0x3186 bytes) 2024-09-17T21:29:56.059 INFO:teuthology.orchestra.run.smithi129.stdout:1014 write 0xceec45 thru 0xcfa4ca (0xb886 bytes) 2024-09-17T21:29:56.061 INFO:teuthology.orchestra.run.smithi129.stdout:1015 read 0x27164a9 thru 0x271b038 (0x4b90 bytes) 2024-09-17T21:29:56.061 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.061 INFO:teuthology.orchestra.run.smithi129.stdout:1018 read 0x94f1aaa thru 0x95017c9 (0xfd20 bytes) 2024-09-17T21:29:56.062 INFO:teuthology.orchestra.run.smithi129.stdout:1019 read 0x7d6ef16 thru 0x7d798b5 (0xa9a0 bytes) 2024-09-17T21:29:56.062 INFO:teuthology.orchestra.run.smithi129.stdout:1020 write 0x7e593bc thru 0x7e60e58 (0x7a9d bytes) 2024-09-17T21:29:56.064 INFO:teuthology.orchestra.run.smithi129.stdout:1021 punch from 0x64b8b55 to 0x64c658b, (0xda36 bytes) 2024-09-17T21:29:56.064 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.069 INFO:teuthology.orchestra.run.smithi129.stdout:1024 write 0xa786472 thru 0xa78f40e (0x8f9d bytes) 2024-09-17T21:29:56.071 INFO:teuthology.orchestra.run.smithi129.stdout:1025 punch from 0x502d12 to 0x504d71, (0x205f bytes) 2024-09-17T21:29:56.072 INFO:teuthology.orchestra.run.smithi129.stdout:1026 punch from 0x625a29a to 0x6263ee6, (0x9c4c bytes) 2024-09-17T21:29:56.072 INFO:teuthology.orchestra.run.smithi129.stdout:1027 punch from 0x3ecb599 to 0x3ed4fdf, (0x9a46 bytes) 2024-09-17T21:29:56.072 INFO:teuthology.orchestra.run.smithi129.stdout:1028 read 0x24d3964 thru 0x24d949f (0x5b3c bytes) 2024-09-17T21:29:56.072 INFO:teuthology.orchestra.run.smithi129.stdout:1029 writesame 0x4380a64 thru 0x4389e84 data_size 0x1(0x9421 bytes) 2024-09-17T21:29:56.089 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.089 INFO:teuthology.orchestra.run.smithi129.stdout:1031 read 0x5e5c84f thru 0x5e6647f (0x9c31 bytes) 2024-09-17T21:29:56.089 INFO:teuthology.orchestra.run.smithi129.stdout:1032 writesame 0xa61acde thru 0xa6293ef data_size 0x3(0xe712 bytes) 2024-09-17T21:29:56.090 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:56.088+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:56.099 INFO:teuthology.orchestra.run.smithi129.stdout:1033 write 0x847f68b thru 0x848b1d3 (0xbb49 bytes) 2024-09-17T21:29:56.101 INFO:teuthology.orchestra.run.smithi129.stdout:1034 flatten 2024-09-17T21:29:56.107 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.107 INFO:teuthology.orchestra.run.smithi129.stdout:1036 punch from 0x170e661 to 0x171d300, (0xec9f bytes) 2024-09-17T21:29:56.113 INFO:teuthology.orchestra.run.smithi129.stdout:1037 trunc from 0xa78f40f to 0x5a312f5 2024-09-17T21:29:56.122 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.123 INFO:teuthology.orchestra.run.smithi129.stdout:1045 write 0x27b6bf2 thru 0x27c1395 (0xa7a4 bytes) 2024-09-17T21:29:56.123 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:56.123 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:56.124 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.124 INFO:teuthology.orchestra.run.smithi129.stdout:1048 read 0x289a4a5 thru 0x28a4d89 (0xa8e5 bytes) 2024-09-17T21:29:56.124 INFO:teuthology.orchestra.run.smithi129.stdout:1049 read 0x4a70c33 thru 0x4a7fe57 (0xf225 bytes) 2024-09-17T21:29:56.125 INFO:teuthology.orchestra.run.smithi129.stdout:1050 punch from 0x2ba34d6 to 0x2bad39f, (0x9ec9 bytes) 2024-09-17T21:29:56.130 INFO:teuthology.orchestra.run.smithi129.stdout:1051 trunc from 0x5a312f5 to 0x1d4990b 2024-09-17T21:29:56.136 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:29:56.157 INFO:teuthology.orchestra.run.smithi129.stdout:1052 writesame 0xc7b7dbb thru 0xc7bbb14 data_size 0x1(0x3d5a bytes) 2024-09-17T21:29:56.157 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.163 INFO:teuthology.orchestra.run.smithi129.stdout:1055 write 0xce9b5ad thru 0xce9b7e7 (0x23b bytes) 2024-09-17T21:29:56.166 INFO:teuthology.orchestra.run.smithi129.stdout:1056 read 0x1f93b81 thru 0x1f9fb0c (0xbf8c bytes) 2024-09-17T21:29:56.173 INFO:teuthology.orchestra.run.smithi129.stdout:1057 write 0xd87923f thru 0xd881fb1 (0x8d73 bytes) 2024-09-17T21:29:56.176 INFO:teuthology.orchestra.run.smithi129.stdout:1058 read 0x7c35f3a thru 0x7c38684 (0x274b bytes) 2024-09-17T21:29:56.176 INFO:teuthology.orchestra.run.smithi129.stdout:1059 punch from 0x8657900 to 0x8665c5a, (0xe35a bytes) 2024-09-17T21:29:56.176 INFO:teuthology.orchestra.run.smithi129.stdout:1060 write 0xc5ae868 thru 0xc5b01b8 (0x1951 bytes) 2024-09-17T21:29:56.179 INFO:teuthology.orchestra.run.smithi129.stdout:1061 write 0x6228f89 thru 0x622f353 (0x63cb bytes) 2024-09-17T21:29:56.181 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.181 INFO:teuthology.orchestra.run.smithi129.stdout:1063 write 0xb528ba0 thru 0xb52b39d (0x27fe bytes) 2024-09-17T21:29:56.182 INFO:teuthology.orchestra.run.smithi129.stdout:1065 punch from 0x9aeeebe to 0x9af19ca, (0x2b0c bytes) 2024-09-17T21:29:56.182 INFO:teuthology.orchestra.run.smithi129.stdout:1067 read 0x9245ebf thru 0x9251b4e (0xbc90 bytes) 2024-09-17T21:29:56.182 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.183 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.191 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:56.188+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:56.193 INFO:teuthology.orchestra.run.smithi129.stdout:1070 trunc from 0xd881fb2 to 0x4b9f4a7 2024-09-17T21:29:56.209 INFO:teuthology.orchestra.run.smithi129.stdout:1071 trunc from 0x4b9f4a7 to 0x1d934cf 2024-09-17T21:29:56.216 INFO:teuthology.orchestra.run.smithi129.stdout:1073 read 0x71b457 thru 0x729bfc (0xe7a6 bytes) 2024-09-17T21:29:56.217 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.224 INFO:teuthology.orchestra.run.smithi129.stdout:1075 trunc from 0x1d934cf to 0x7c3b7f6 2024-09-17T21:29:56.229 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.230 INFO:teuthology.orchestra.run.smithi129.stdout:1078 trunc from 0x7c3b7f6 to 0x8a3ef8e 2024-09-17T21:29:56.236 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.236 INFO:teuthology.orchestra.run.smithi129.stdout:1081 punch from 0x93a8cb to 0x93e4c4, (0x3bf9 bytes) 2024-09-17T21:29:56.244 INFO:teuthology.orchestra.run.smithi129.stdout:1082 write 0xae617ef thru 0xae6f44b (0xdc5d bytes) 2024-09-17T21:29:56.246 INFO:teuthology.orchestra.run.smithi129.stdout:1083 writesame 0x3bb1a16 thru 0x3bb2047 data_size 0x2(0x632 bytes) 2024-09-17T21:29:56.247 INFO:teuthology.orchestra.run.smithi129.stdout:1084 read 0x85b09f7 thru 0x85b2376 (0x1980 bytes) 2024-09-17T21:29:56.247 INFO:teuthology.orchestra.run.smithi129.stdout:1085 write 0x1b5e811 thru 0x1b654aa (0x6c9a bytes) 2024-09-17T21:29:56.263 INFO:teuthology.orchestra.run.smithi129.stdout:1087 trunc from 0xae6f44c to 0x591d4e 2024-09-17T21:29:56.281 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:56.281 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:56.291 INFO:teuthology.orchestra.run.smithi129.stdout:1090 writesame 0x1ad0ccd thru 0x1addc44 data_size 0xc(0xcf78 bytes) 2024-09-17T21:29:56.292 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:29:56.292 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:56.289+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:56.309 INFO:teuthology.orchestra.run.smithi129.stdout:1091 write 0xbecdeb9 thru 0xbed37fb (0x5943 bytes) 2024-09-17T21:29:56.312 INFO:teuthology.orchestra.run.smithi129.stdout:1092 write 0xbeaaaed thru 0xbeb6d87 (0xc29b bytes) 2024-09-17T21:29:56.313 INFO:teuthology.orchestra.run.smithi129.stdout:1093 punch from 0x2870602 to 0x2878e89, (0x8887 bytes) 2024-09-17T21:29:56.328 INFO:teuthology.orchestra.run.smithi129.stdout:1094 trunc from 0xbed37fc to 0x23777 2024-09-17T21:29:56.361 INFO:teuthology.orchestra.run.smithi129.stdout:1095 write 0xddbc152 thru 0xddbfb4d (0x39fc bytes) 2024-09-17T21:29:56.364 INFO:teuthology.orchestra.run.smithi129.stdout:1096 write 0x93dd008 thru 0x93e1e12 (0x4e0b bytes) 2024-09-17T21:29:56.366 INFO:teuthology.orchestra.run.smithi129.stdout:1097 punch from 0x34d1559 to 0x34dd42b, (0xbed2 bytes) 2024-09-17T21:29:56.366 INFO:teuthology.orchestra.run.smithi129.stdout:1098 write 0x7e66740 thru 0x7e6c33c (0x5bfd bytes) 2024-09-17T21:29:56.369 INFO:teuthology.orchestra.run.smithi129.stdout:1099 punch from 0x421eeb9 to 0x422ec1d, (0xfd64 bytes) 2024-09-17T21:29:56.369 INFO:teuthology.orchestra.run.smithi129.stdout:1101 writesame 0x250348 thru 0x253999 data_size 0x11(0x3652 bytes) 2024-09-17T21:29:56.370 INFO:teuthology.orchestra.run.smithi129.stdout:1102 writesame 0xa2f62c2 thru 0xa303a5e data_size 0x3(0xd79d bytes) 2024-09-17T21:29:56.380 INFO:teuthology.orchestra.run.smithi129.stdout:1103 writesame 0xae1e2ad thru 0xae1e6c4 data_size 0x4(0x418 bytes) 2024-09-17T21:29:56.382 INFO:teuthology.orchestra.run.smithi129.stdout:1104 write 0xa970462 thru 0xa97f4f3 (0xf092 bytes) 2024-09-17T21:29:56.383 INFO:teuthology.orchestra.run.smithi129.stdout:1105 writesame 0x27c8bc2 thru 0x27d4649 data_size 0x7f(0xba88 bytes) 2024-09-17T21:29:56.386 INFO:teuthology.orchestra.run.smithi129.stdout:1106 read 0xa6b3c2c thru 0xa6be11d (0xa4f2 bytes) 2024-09-17T21:29:56.392 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:56.389+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:56.392 INFO:teuthology.orchestra.run.smithi129.stdout:1108 write 0xddb3a46 thru 0xddc2555 (0xeb10 bytes) 2024-09-17T21:29:56.394 INFO:teuthology.orchestra.run.smithi129.stdout:1109 write 0xe72ca0 thru 0xe76d49 (0x40aa bytes) 2024-09-17T21:29:56.401 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:56.401 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:56.409 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:29:56.412 INFO:teuthology.orchestra.run.smithi129.stdout:1110 trunc from 0xddc2556 to 0xf0b16 2024-09-17T21:29:56.429 INFO:teuthology.orchestra.run.smithi129.stdout:1111 read 0x6e66c thru 0x77f21 (0x98b6 bytes) 2024-09-17T21:29:56.440 INFO:teuthology.orchestra.run.smithi129.stdout:1112 write 0x4718026 thru 0x471b0de (0x30b9 bytes) 2024-09-17T21:29:56.443 INFO:teuthology.orchestra.run.smithi129.stdout:1113 writesame 0x256fad6 thru 0x257109e data_size 0x21(0x15c9 bytes) 2024-09-17T21:29:56.446 INFO:teuthology.orchestra.run.smithi129.stdout:1114 read 0xb17f06 thru 0xb22560 (0xa65b bytes) 2024-09-17T21:29:56.446 INFO:teuthology.orchestra.run.smithi129.stdout:1115 read 0x42bdf12 thru 0x42c5875 (0x7964 bytes) 2024-09-17T21:29:56.446 INFO:teuthology.orchestra.run.smithi129.stdout:1116 writesame 0x3f6e002 thru 0x3f79ec6 data_size 0x49(0xbec5 bytes) 2024-09-17T21:29:56.447 INFO:teuthology.orchestra.run.smithi129.stdout:1118 writesame 0x15562da thru 0x1557e4c data_size 0x1(0x1b73 bytes) 2024-09-17T21:29:56.450 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.465 INFO:teuthology.orchestra.run.smithi129.stdout:1122 writesame 0xc53aa18 thru 0xc548171 data_size 0x1(0xd75a bytes) 2024-09-17T21:29:56.465 INFO:teuthology.orchestra.run.smithi129.stdout:1123 writesame 0xb524f29 thru 0xb534adb data_size 0x31(0xfbb3 bytes) 2024-09-17T21:29:56.467 INFO:teuthology.orchestra.run.smithi129.stdout:1125 punch from 0xa4f721a to 0xa4fcdc6, (0x5bac bytes) 2024-09-17T21:29:56.467 INFO:teuthology.orchestra.run.smithi129.stdout:1127 punch from 0x9ef7c0b to 0x9f002ee, (0x86e3 bytes) 2024-09-17T21:29:56.467 INFO:teuthology.orchestra.run.smithi129.stdout:1128 read 0x8d2ca26 thru 0x8d2ded9 (0x14b4 bytes) 2024-09-17T21:29:56.470 INFO:teuthology.orchestra.run.smithi129.stdout:1129 trunc from 0xc548172 to 0xa04ff3c 2024-09-17T21:29:56.478 INFO:teuthology.orchestra.run.smithi129.stdout:1130 read 0x8155525 thru 0x815b973 (0x644f bytes) 2024-09-17T21:29:56.478 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.478 INFO:teuthology.orchestra.run.smithi129.stdout:1133 read 0x5316fbc thru 0x5325a98 (0xeadd bytes) 2024-09-17T21:29:56.478 INFO:teuthology.orchestra.run.smithi129.stdout:1134 write 0x4cfe364 thru 0x4d04128 (0x5dc5 bytes) 2024-09-17T21:29:56.481 INFO:teuthology.orchestra.run.smithi129.stdout:1136 write 0x183ad42 thru 0x18433e6 (0x86a5 bytes) 2024-09-17T21:29:56.482 INFO:teuthology.orchestra.run.smithi129.stdout:1137 read 0x3d88923 thru 0x3d8acde (0x23bc bytes) 2024-09-17T21:29:56.492 INFO:teuthology.orchestra.run.smithi129.stdout:1138 trunc from 0xa04ff3c to 0x2574754 2024-09-17T21:29:56.493 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:56.491+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:56.511 INFO:teuthology.orchestra.run.smithi129.stdout:1139 write 0x6bbb4e8 thru 0x6bc4728 (0x9241 bytes) 2024-09-17T21:29:56.514 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.514 INFO:teuthology.orchestra.run.smithi129.stdout:1143 punch from 0x21ef3b5 to 0x21fcf52, (0xdb9d bytes) 2024-09-17T21:29:56.518 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:56.518 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:56.518 INFO:teuthology.orchestra.run.smithi129.stdout:1144 write 0x6d667f5 thru 0x6d741e1 (0xd9ed bytes) 2024-09-17T21:29:56.527 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:29:56.529 INFO:teuthology.orchestra.run.smithi129.stdout:1145 write 0x9219d7d thru 0x92269bd (0xcc41 bytes) 2024-09-17T21:29:56.531 INFO:teuthology.orchestra.run.smithi129.stdout:1146 write 0x63de06b thru 0x63e845c (0xa3f2 bytes) 2024-09-17T21:29:56.542 INFO:teuthology.orchestra.run.smithi129.stdout:1148 trunc from 0x92269be to 0xe6104f 2024-09-17T21:29:56.566 INFO:teuthology.orchestra.run.smithi129.stdout:1149 write 0x8ebe117 thru 0x8eca419 (0xc303 bytes) 2024-09-17T21:29:56.569 INFO:teuthology.orchestra.run.smithi129.stdout:1150 writesame 0x8d42b31 thru 0x8d49691 data_size 0x15(0x6b61 bytes) 2024-09-17T21:29:56.570 INFO:teuthology.orchestra.run.smithi129.stdout:1152 write 0x770e4ec thru 0x770f7be (0x12d3 bytes) 2024-09-17T21:29:56.576 INFO:teuthology.orchestra.run.smithi129.stdout:1153 trunc from 0x8eca41a to 0x50d1ed6 2024-09-17T21:29:56.582 INFO:teuthology.orchestra.run.smithi129.stdout:1154 read 0x1a73488 thru 0x1a8186b (0xe3e4 bytes) 2024-09-17T21:29:56.593 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:56.591+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:56.598 INFO:teuthology.orchestra.run.smithi129.stdout:1155 write 0xe10a27c thru 0xe10d3ee (0x3173 bytes) 2024-09-17T21:29:56.601 INFO:teuthology.orchestra.run.smithi129.stdout:1157 write 0x557f906 thru 0x5586d84 (0x747f bytes) 2024-09-17T21:29:56.603 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.603 INFO:teuthology.orchestra.run.smithi129.stdout:1161 read 0xb74dc8d thru 0xb75bf2a (0xe29e bytes) 2024-09-17T21:29:56.603 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:29:56.603 INFO:teuthology.orchestra.run.smithi129.stdout:1163 writesame 0x2e751bf thru 0x2e7e0b3 data_size 0xb(0x8ef5 bytes) 2024-09-17T21:29:56.606 INFO:teuthology.orchestra.run.smithi129.stdout:1164 write 0x954e24c thru 0x955b617 (0xd3cc bytes) 2024-09-17T21:29:56.606 INFO:teuthology.orchestra.run.smithi129.stdout:1165 read 0x18765ab thru 0x188541f (0xee75 bytes) 2024-09-17T21:29:56.607 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.613 INFO:teuthology.orchestra.run.smithi129.stdout:1168 write 0xedad111 thru 0xedb2b7a (0x5a6a bytes) 2024-09-17T21:29:56.616 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.616 INFO:teuthology.orchestra.run.smithi129.stdout:1170 write 0x6df6248 thru 0x6e023e4 (0xc19d bytes) 2024-09-17T21:29:56.621 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.621 INFO:teuthology.orchestra.run.smithi129.stdout:1172 write 0x5efecaf thru 0x5f01f3d (0x328f bytes) 2024-09-17T21:29:56.621 INFO:teuthology.orchestra.run.smithi129.stdout:1173 writesame 0x8e86c29 thru 0x8e93b09 data_size 0x1(0xcee1 bytes) 2024-09-17T21:29:56.642 INFO:teuthology.orchestra.run.smithi129.stdout:1175 punch from 0x9034931 to 0x903db45, (0x9214 bytes) 2024-09-17T21:29:56.643 INFO:teuthology.orchestra.run.smithi129.stdout:1176 writesame 0x5939b3f thru 0x593b092 data_size 0x7(0x1554 bytes) 2024-09-17T21:29:56.646 INFO:teuthology.orchestra.run.smithi129.stdout:1177 read 0xdcccd16 thru 0xdcd6d09 (0x9ff4 bytes) 2024-09-17T21:29:56.646 INFO:teuthology.orchestra.run.smithi129.stdout:1178 read 0x7ecd13d thru 0x7ed581d (0x86e1 bytes) 2024-09-17T21:29:56.646 INFO:teuthology.orchestra.run.smithi129.stdout:1179 write 0xec0cf66 thru 0xec1b7c1 (0xe85c bytes) 2024-09-17T21:29:56.648 INFO:teuthology.orchestra.run.smithi129.stdout:1180 punch from 0xd451169 to 0xd451a39, (0x8d0 bytes) 2024-09-17T21:29:56.648 INFO:teuthology.orchestra.run.smithi129.stdout:1181 writesame 0x6546f7d thru 0x654b94a data_size 0x5e(0x49ce bytes) 2024-09-17T21:29:56.650 INFO:teuthology.orchestra.run.smithi129.stdout:1182 read 0x299556f thru 0x299560a (0x9c bytes) 2024-09-17T21:29:56.650 INFO:teuthology.orchestra.run.smithi129.stdout:1183 write 0x73d9d37 thru 0x73e1c22 (0x7eec bytes) 2024-09-17T21:29:56.654 INFO:teuthology.orchestra.run.smithi129.stdout:1184 trunc from 0xedb2b7b to 0xce0d189 2024-09-17T21:29:56.663 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.670 INFO:teuthology.orchestra.run.smithi129.stdout:1187 write 0xe46ecb5 thru 0xe47b65e (0xc9aa bytes) 2024-09-17T21:29:56.672 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.673 INFO:teuthology.orchestra.run.smithi129.stdout:1189 writesame 0x5cdb66b thru 0x5cdf68b data_size 0x1(0x4021 bytes) 2024-09-17T21:29:56.680 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.680 INFO:teuthology.orchestra.run.smithi129.stdout:1191 write 0x51a3a66 thru 0x51a41ec (0x787 bytes) 2024-09-17T21:29:56.680 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:56.680 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:56.681 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.681 INFO:teuthology.orchestra.run.smithi129.stdout:1194 read 0xe3466f6 thru 0xe3512fa (0xac05 bytes) 2024-09-17T21:29:56.681 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.681 INFO:teuthology.orchestra.run.smithi129.stdout:1196 read 0xcb33e66 thru 0xcb3d248 (0x93e3 bytes) 2024-09-17T21:29:56.682 INFO:teuthology.orchestra.run.smithi129.stdout:1197 write 0x4f565a8 thru 0x4f6247f (0xbed8 bytes) 2024-09-17T21:29:56.689 INFO:teuthology.orchestra.run.smithi129.stdout:1198 trunc from 0xe47b65f to 0x9f3b9a7 2024-09-17T21:29:56.690 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:29:56.694 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:56.692+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:56.706 INFO:teuthology.orchestra.run.smithi129.stdout:1199 trunc from 0x9f3b9a7 to 0xe9483b7 2024-09-17T21:29:56.711 INFO:teuthology.orchestra.run.smithi129.stdout:1200 writesame 0xa0f889d thru 0xa107a5b data_size 0x31(0xf1bf bytes) 2024-09-17T21:29:56.713 INFO:teuthology.orchestra.run.smithi129.stdout:1202 read 0x7af095f thru 0x7afd860 (0xcf02 bytes) 2024-09-17T21:29:56.713 INFO:teuthology.orchestra.run.smithi129.stdout:1204 write 0x367e72 thru 0x36be88 (0x4017 bytes) 2024-09-17T21:29:56.715 INFO:teuthology.orchestra.run.smithi129.stdout:1205 read 0xc94bc32 thru 0xc957563 (0xb932 bytes) 2024-09-17T21:29:56.715 INFO:teuthology.orchestra.run.smithi129.stdout:1206 read 0x664ac61 thru 0x6655b9e (0xaf3e bytes) 2024-09-17T21:29:56.715 INFO:teuthology.orchestra.run.smithi129.stdout:1209 write 0xc8bb46a thru 0xc8c02a0 (0x4e37 bytes) 2024-09-17T21:29:56.719 INFO:teuthology.orchestra.run.smithi129.stdout:1214 read 0x691ebc1 thru 0x692462c (0x5a6c bytes) 2024-09-17T21:29:56.719 INFO:teuthology.orchestra.run.smithi129.stdout:1215 write 0xe2dbd68 thru 0xe2e8628 (0xc8c1 bytes) 2024-09-17T21:29:56.720 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.736 INFO:teuthology.orchestra.run.smithi129.stdout:1218 trunc from 0xe9483b7 to 0x1b50f4b 2024-09-17T21:29:56.736 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:29:56.736 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:29:56.755 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:29:56.755 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:29:56.761 INFO:teuthology.orchestra.run.smithi129.stdout:1219 write 0x6da5f65 thru 0x6dacba7 (0x6c43 bytes) 2024-09-17T21:29:56.762 INFO:teuthology.orchestra.run.smithi129.stdout:1220 read 0x135dee1 thru 0x1365c3d (0x7d5d bytes) 2024-09-17T21:29:56.771 INFO:teuthology.orchestra.run.smithi129.stdout:1223 trunc from 0x6dacba8 to 0xde253a1 2024-09-17T21:29:56.773 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:29:56.773 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:29:56.790 INFO:teuthology.orchestra.run.smithi129.stdout:1224 trunc from 0xde253a1 to 0x28960c6 2024-09-17T21:29:56.791 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:29:56.791 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:29:56.795 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:56.793+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:56.796 INFO:teuthology.orchestra.run.smithi129.stdout:1225 read 0x666f33 thru 0x66ca32 (0x5b00 bytes) 2024-09-17T21:29:56.797 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.797 INFO:teuthology.orchestra.run.smithi129.stdout:1229 punch from 0x1b38b88 to 0x1b441ad, (0xb625 bytes) 2024-09-17T21:29:56.799 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:56.799 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:56.807 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:29:56.808 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:29:56.809 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:29:56.809 INFO:teuthology.orchestra.run.smithi129.stdout:1230 write 0x7bd9a14 thru 0x7bdac83 (0x1270 bytes) 2024-09-17T21:29:56.810 INFO:teuthology.orchestra.run.smithi129.stdout:1231 read 0x23a7f60 thru 0x23b3899 (0xb93a bytes) 2024-09-17T21:29:56.810 INFO:teuthology.orchestra.run.smithi129.stdout:1232 read 0x44dea0f thru 0x44ed531 (0xeb23 bytes) 2024-09-17T21:29:56.811 INFO:teuthology.orchestra.run.smithi129.stdout:1233 read 0x4ccef79 thru 0x4cde3f6 (0xf47e bytes) 2024-09-17T21:29:56.811 INFO:teuthology.orchestra.run.smithi129.stdout:1234 write 0xe00471 thru 0xe030fd (0x2c8d bytes) 2024-09-17T21:29:56.826 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:29:56.827 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:29:56.827 INFO:teuthology.orchestra.run.smithi129.stdout:1236 write 0xed0fb0e thru 0xed152c2 (0x57b5 bytes) 2024-09-17T21:29:56.829 INFO:teuthology.orchestra.run.smithi129.stdout:1237 read 0x78dabc8 thru 0x78dc1c8 (0x1601 bytes) 2024-09-17T21:29:56.830 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.830 INFO:teuthology.orchestra.run.smithi129.stdout:1241 writesame 0x7d25d79 thru 0x7d2ab8f data_size 0x1(0x4e17 bytes) 2024-09-17T21:29:56.839 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.839 INFO:teuthology.orchestra.run.smithi129.stdout:1244 write 0x6989320 thru 0x6996af0 (0xd7d1 bytes) 2024-09-17T21:29:56.841 INFO:teuthology.orchestra.run.smithi129.stdout:1245 write 0x418bf05 thru 0x4190237 (0x4333 bytes) 2024-09-17T21:29:56.843 INFO:teuthology.orchestra.run.smithi129.stdout:1246 punch from 0x6ed38e to 0x6ee937, (0x15a9 bytes) 2024-09-17T21:29:56.843 INFO:teuthology.orchestra.run.smithi129.stdout:1247 write 0x7a09450 thru 0x7a0cd38 (0x38e9 bytes) 2024-09-17T21:29:56.843 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:29:56.843 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:29:56.844 INFO:teuthology.orchestra.run.smithi129.stdout:1249 write 0xaef3904 thru 0xaf01db1 (0xe4ae bytes) 2024-09-17T21:29:56.846 INFO:teuthology.orchestra.run.smithi129.stdout:1250 read 0xac0cc9 thru 0xac80ff (0x7437 bytes) 2024-09-17T21:29:56.846 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.847 INFO:teuthology.orchestra.run.smithi129.stdout:1252 write 0x652c284 thru 0x65342fc (0x8079 bytes) 2024-09-17T21:29:56.848 INFO:teuthology.orchestra.run.smithi129.stdout:1254 write 0x22d25aa thru 0x22d469a (0x20f1 bytes) 2024-09-17T21:29:56.850 INFO:teuthology.orchestra.run.smithi129.stdout:1255 writesame 0xe60ae68 thru 0xe60b2f9 data_size 0x12(0x492 bytes) 2024-09-17T21:29:56.862 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:29:56.862 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:29:56.864 INFO:teuthology.orchestra.run.smithi129.stdout:1256 trunc from 0xed152c3 to 0x581b628 2024-09-17T21:29:56.884 INFO:teuthology.orchestra.run.smithi129.stdout:1259 trunc from 0x581b628 to 0x9c733c0 2024-09-17T21:29:56.889 INFO:teuthology.orchestra.run.smithi129.stdout:1260 writesame 0x577bcf5 thru 0x578b73c data_size 0x4(0xfa48 bytes) 2024-09-17T21:29:56.895 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:56.893+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:56.908 INFO:teuthology.orchestra.run.smithi129.stdout:1262 trunc from 0x9c733c0 to 0xce7f69 2024-09-17T21:29:56.919 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:56.919 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:29:56.919 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:29:56.919 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:29:56.919 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:56.928 INFO:teuthology.orchestra.run.smithi129.stdout:1264 read 0x7d5ced thru 0x7e059d (0xa8b1 bytes) 2024-09-17T21:29:56.929 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:29:56.937 INFO:teuthology.orchestra.run.smithi129.stdout:1265 writesame 0x2ae6a0c thru 0x2af55e8 data_size 0x3(0xebdd bytes) 2024-09-17T21:29:56.937 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.937 INFO:teuthology.orchestra.run.smithi129.stdout:1268 punch from 0x253d8c7 to 0x25415f8, (0x3d31 bytes) 2024-09-17T21:29:56.937 INFO:teuthology.orchestra.run.smithi129.stdout:1269 read 0x1f902de thru 0x1f9743b (0x715e bytes) 2024-09-17T21:29:56.937 INFO:teuthology.orchestra.run.smithi129.stdout:1270 read 0x2b7704 thru 0x2bc215 (0x4b12 bytes) 2024-09-17T21:29:56.937 INFO:teuthology.orchestra.run.smithi129.stdout:1271 punch from 0x16c4d65 to 0x16c827e, (0x3519 bytes) 2024-09-17T21:29:56.955 INFO:teuthology.orchestra.run.smithi129.stdout:1272 write 0xd1a86f1 thru 0xd1b6ab9 (0xe3c9 bytes) 2024-09-17T21:29:56.958 INFO:teuthology.orchestra.run.smithi129.stdout:1273 writesame 0xca6e0ff thru 0xca79398 data_size 0x1(0xb29a bytes) 2024-09-17T21:29:56.981 INFO:teuthology.orchestra.run.smithi129.stdout:1274 punch from 0x7e65744 to 0x7e6ebb4, (0x9470 bytes) 2024-09-17T21:29:56.981 INFO:teuthology.orchestra.run.smithi129.stdout:1275 writesame 0xb25c412 thru 0xb263f5d data_size 0x1(0x7b4c bytes) 2024-09-17T21:29:56.995 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:56.996 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:56.994+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:56.996 INFO:teuthology.orchestra.run.smithi129.stdout:1278 trunc from 0xd1b6aba to 0xe382c52 2024-09-17T21:29:57.001 INFO:teuthology.orchestra.run.smithi129.stdout:1279 read 0x4d02e44 thru 0x4d0552d (0x26ea bytes) 2024-09-17T21:29:57.002 INFO:teuthology.orchestra.run.smithi129.stdout:1280 read 0xc0b836b thru 0xc0c1810 (0x94a6 bytes) 2024-09-17T21:29:57.002 INFO:teuthology.orchestra.run.smithi129.stdout:1281 writesame 0xb897720 thru 0xb89ecd3 data_size 0x7c(0x75b4 bytes) 2024-09-17T21:29:57.004 INFO:teuthology.orchestra.run.smithi129.stdout:1282 writesame 0xce0cc51 thru 0xce16d29 data_size 0x1(0xa0d9 bytes) 2024-09-17T21:29:57.023 INFO:teuthology.orchestra.run.smithi129.stdout:1283 write 0xdd3e72c thru 0xdd41f87 (0x385c bytes) 2024-09-17T21:29:57.028 INFO:teuthology.orchestra.run.smithi129.stdout:1287 write 0x1675c0a thru 0x1682ce4 (0xd0db bytes) 2024-09-17T21:29:57.030 INFO:teuthology.orchestra.run.smithi129.stdout:1288 read 0xa8f597d thru 0xa903c58 (0xe2dc bytes) 2024-09-17T21:29:57.030 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.038 INFO:teuthology.orchestra.run.smithi129.stdout:1291 trunc from 0xe382c52 to 0x7bbb987 2024-09-17T21:29:57.039 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:57.039 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:29:57.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:29:57.039 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:29:57.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.039 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10799 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.039 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:29:57.039 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.9706940050000004, 2024-09-17T21:29:57.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039122745, 2024-09-17T21:29:57.040 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.040 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.040 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:57.040 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.040 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.040 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.040 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10799 2024-09-17T21:29:57.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.040 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:29:57.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:29:57.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065469+0000", 2024-09-17T21:29:57.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:29:57.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065477+0000", 2024-09-17T21:29:57.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1319999999999994e-06 2024-09-17T21:29:57.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065483+0000", 2024-09-17T21:29:57.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2040000000000002e-06 2024-09-17T21:29:57.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065487+0000", 2024-09-17T21:29:57.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.42e-06 2024-09-17T21:29:57.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099984+0000", 2024-09-17T21:29:57.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034496484000000001 2024-09-17T21:29:57.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100065+0000", 2024-09-17T21:29:57.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1200999999999997e-05 2024-09-17T21:29:57.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:29:57.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100422+0000", 2024-09-17T21:29:57.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035657700000000001 2024-09-17T21:29:57.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:57.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104214+0000", 2024-09-17T21:29:57.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003792045 2024-09-17T21:29:57.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:57.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104505+0000", 2024-09-17T21:29:57.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029141399999999999 2024-09-17T21:29:57.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:57.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104556+0000", 2024-09-17T21:29:57.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1388999999999999e-05 2024-09-17T21:29:57.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104591+0000", 2024-09-17T21:29:57.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4455999999999997e-05 2024-09-17T21:29:57.045 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.045 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.045 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.046 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12187 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.046 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:29:57.046 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.896601274, 2024-09-17T21:29:57.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038825750999999999, 2024-09-17T21:29:57.046 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.046 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.046 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.046 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.046 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.046 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.046 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12187 2024-09-17T21:29:57.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.046 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:29:57.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:29:57.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:29:57.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:29:57.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139563+0000", 2024-09-17T21:29:57.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.624e-06 2024-09-17T21:29:57.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139569+0000", 2024-09-17T21:29:57.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1090000000000003e-06 2024-09-17T21:29:57.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139573+0000", 2024-09-17T21:29:57.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4429999999999996e-06 2024-09-17T21:29:57.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178206+0000", 2024-09-17T21:29:57.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038632376000000003 2024-09-17T21:29:57.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178282+0000", 2024-09-17T21:29:57.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6076999999999997e-05 2024-09-17T21:29:57.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178387+0000", 2024-09-17T21:29:57.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104682 2024-09-17T21:29:57.050 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.050 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.050 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.051 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12188 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.051 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:29:57.051 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.8965830930000003, 2024-09-17T21:29:57.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039053826999999999, 2024-09-17T21:29:57.051 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.051 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.051 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.051 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.051 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.051 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.051 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12188 2024-09-17T21:29:57.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.052 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:29:57.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:29:57.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139746+0000", 2024-09-17T21:29:57.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000167281 2024-09-17T21:29:57.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139749+0000", 2024-09-17T21:29:57.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9660000000000001e-06 2024-09-17T21:29:57.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139757+0000", 2024-09-17T21:29:57.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8029999999999997e-06 2024-09-17T21:29:57.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139771+0000", 2024-09-17T21:29:57.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3541e-05 2024-09-17T21:29:57.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178415+0000", 2024-09-17T21:29:57.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038644803999999998 2024-09-17T21:29:57.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178517+0000", 2024-09-17T21:29:57.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101298 2024-09-17T21:29:57.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178633+0000", 2024-09-17T21:29:57.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116134 2024-09-17T21:29:57.055 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.055 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.055 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.055 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12189 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.055 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:29:57.056 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.8963826160000004, 2024-09-17T21:29:57.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039135991000000002, 2024-09-17T21:29:57.056 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.056 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.056 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.056 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.056 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.056 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.056 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12189 2024-09-17T21:29:57.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.057 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:29:57.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:29:57.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139780+0000", 2024-09-17T21:29:57.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:29:57.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139782+0000", 2024-09-17T21:29:57.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1950000000000002e-06 2024-09-17T21:29:57.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139788+0000", 2024-09-17T21:29:57.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0639999999999999e-06 2024-09-17T21:29:57.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139792+0000", 2024-09-17T21:29:57.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1960000000000001e-06 2024-09-17T21:29:57.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178671+0000", 2024-09-17T21:29:57.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038878176 2024-09-17T21:29:57.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178778+0000", 2024-09-17T21:29:57.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107278 2024-09-17T21:29:57.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178915+0000", 2024-09-17T21:29:57.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013757900000000001 2024-09-17T21:29:57.060 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.060 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.060 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.060 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12190 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.060 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:29:57.060 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.8963634330000003, 2024-09-17T21:29:57.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039343781000000001, 2024-09-17T21:29:57.060 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.060 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.061 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.061 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.061 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.061 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.061 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12190 2024-09-17T21:29:57.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.061 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:29:57.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:29:57.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:29:57.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:29:57.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139801+0000", 2024-09-17T21:29:57.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.916e-06 2024-09-17T21:29:57.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139807+0000", 2024-09-17T21:29:57.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4540000000000004e-06 2024-09-17T21:29:57.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139812+0000", 2024-09-17T21:29:57.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9729999999999997e-06 2024-09-17T21:29:57.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178960+0000", 2024-09-17T21:29:57.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039147567000000001 2024-09-17T21:29:57.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179051+0000", 2024-09-17T21:29:57.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0754000000000006e-05 2024-09-17T21:29:57.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179142+0000", 2024-09-17T21:29:57.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1712999999999994e-05 2024-09-17T21:29:57.065 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.065 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.065 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.065 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12191 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.065 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:29:57.065 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.8963435749999995, 2024-09-17T21:29:57.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039464336000000003, 2024-09-17T21:29:57.065 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.065 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.065 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.066 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.066 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.066 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.066 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12191 2024-09-17T21:29:57.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.066 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:29:57.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:29:57.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:29:57.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:29:57.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139821+0000", 2024-09-17T21:29:57.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1260000000000001e-06 2024-09-17T21:29:57.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139827+0000", 2024-09-17T21:29:57.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T21:29:57.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139840+0000", 2024-09-17T21:29:57.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2607e-05 2024-09-17T21:29:57.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179162+0000", 2024-09-17T21:29:57.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039322863999999999 2024-09-17T21:29:57.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179204+0000", 2024-09-17T21:29:57.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1938999999999999e-05 2024-09-17T21:29:57.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179283+0000", 2024-09-17T21:29:57.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8547000000000005e-05 2024-09-17T21:29:57.069 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.069 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.069 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.070 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12192 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.070 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:29:57.070 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.8963158629999999, 2024-09-17T21:29:57.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039652318999999998, 2024-09-17T21:29:57.070 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.070 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.070 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.070 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.070 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.070 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.070 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12192 2024-09-17T21:29:57.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.071 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:29:57.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:29:57.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141458+0000", 2024-09-17T21:29:57.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001612098 2024-09-17T21:29:57.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141469+0000", 2024-09-17T21:29:57.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-17T21:29:57.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141478+0000", 2024-09-17T21:29:57.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T21:29:57.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141485+0000", 2024-09-17T21:29:57.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4760000000000003e-06 2024-09-17T21:29:57.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179317+0000", 2024-09-17T21:29:57.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037831832000000003 2024-09-17T21:29:57.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179395+0000", 2024-09-17T21:29:57.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8349000000000005e-05 2024-09-17T21:29:57.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179499+0000", 2024-09-17T21:29:57.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103628 2024-09-17T21:29:57.074 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.074 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.074 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.075 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12257 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.075 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:29:57.075 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.8941270550000002, 2024-09-17T21:29:57.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038862751000000001, 2024-09-17T21:29:57.075 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.075 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.075 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.075 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.075 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.075 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.075 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12257 2024-09-17T21:29:57.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.075 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:29:57.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:29:57.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:29:57.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T21:29:57.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142037+0000", 2024-09-17T21:29:57.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.03e-06 2024-09-17T21:29:57.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142043+0000", 2024-09-17T21:29:57.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6319999999999996e-06 2024-09-17T21:29:57.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142052+0000", 2024-09-17T21:29:57.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2709999999999998e-06 2024-09-17T21:29:57.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180794+0000", 2024-09-17T21:29:57.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038741758000000001 2024-09-17T21:29:57.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180843+0000", 2024-09-17T21:29:57.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9021999999999999e-05 2024-09-17T21:29:57.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180898+0000", 2024-09-17T21:29:57.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673000000000003e-05 2024-09-17T21:29:57.079 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.079 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.079 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.079 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12258 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.079 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:29:57.079 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.8941035379999995, 2024-09-17T21:29:57.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038950717000000003, 2024-09-17T21:29:57.079 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.079 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.079 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.080 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.080 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.080 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.080 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12258 2024-09-17T21:29:57.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.080 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:29:57.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:29:57.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:29:57.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199999999999999e-07 2024-09-17T21:29:57.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142061+0000", 2024-09-17T21:29:57.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8279999999999999e-06 2024-09-17T21:29:57.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142067+0000", 2024-09-17T21:29:57.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5600000000000001e-06 2024-09-17T21:29:57.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142071+0000", 2024-09-17T21:29:57.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6240000000000001e-06 2024-09-17T21:29:57.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180921+0000", 2024-09-17T21:29:57.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038849366000000003 2024-09-17T21:29:57.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180961+0000", 2024-09-17T21:29:57.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0219000000000002e-05 2024-09-17T21:29:57.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181009+0000", 2024-09-17T21:29:57.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8437999999999999e-05 2024-09-17T21:29:57.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.083 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.084 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12259 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.084 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:29:57.084 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.8940762910000002, 2024-09-17T21:29:57.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039116832999999997, 2024-09-17T21:29:57.084 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.084 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.084 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.084 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.084 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.084 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.084 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12259 2024-09-17T21:29:57.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.084 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:29:57.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:29:57.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:29:57.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2200000000000004e-07 2024-09-17T21:29:57.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142088+0000", 2024-09-17T21:29:57.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0329999999999998e-06 2024-09-17T21:29:57.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142095+0000", 2024-09-17T21:29:57.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4320000000000004e-06 2024-09-17T21:29:57.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142100+0000", 2024-09-17T21:29:57.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:29:57.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181037+0000", 2024-09-17T21:29:57.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038937601000000002 2024-09-17T21:29:57.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181114+0000", 2024-09-17T21:29:57.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6857e-05 2024-09-17T21:29:57.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181203+0000", 2024-09-17T21:29:57.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8437999999999996e-05 2024-09-17T21:29:57.088 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.088 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.088 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.088 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12260 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.088 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:29:57.089 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.8940560040000003, 2024-09-17T21:29:57.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039204184000000003, 2024-09-17T21:29:57.089 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.089 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.089 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.089 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.089 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.089 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.089 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12260 2024-09-17T21:29:57.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.089 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:29:57.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:29:57.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142107+0000", 2024-09-17T21:29:57.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:29:57.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142108+0000", 2024-09-17T21:29:57.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.753e-06 2024-09-17T21:29:57.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142114+0000", 2024-09-17T21:29:57.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7660000000000004e-06 2024-09-17T21:29:57.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142119+0000", 2024-09-17T21:29:57.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:29:57.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181216+0000", 2024-09-17T21:29:57.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039097191000000003 2024-09-17T21:29:57.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181260+0000", 2024-09-17T21:29:57.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4731000000000003e-05 2024-09-17T21:29:57.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181310+0000", 2024-09-17T21:29:57.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9818000000000003e-05 2024-09-17T21:29:57.093 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.093 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.093 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.093 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12261 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.093 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:29:57.093 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.8940376800000003, 2024-09-17T21:29:57.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039292200999999999, 2024-09-17T21:29:57.094 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.094 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.094 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.094 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.094 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.094 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.094 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12261 2024-09-17T21:29:57.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.094 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:29:57.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:29:57.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142125+0000", 2024-09-17T21:29:57.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:29:57.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142127+0000", 2024-09-17T21:29:57.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.792e-06 2024-09-17T21:29:57.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142132+0000", 2024-09-17T21:29:57.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3979999999999998e-06 2024-09-17T21:29:57.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142136+0000", 2024-09-17T21:29:57.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.211e-06 2024-09-17T21:29:57.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181324+0000", 2024-09-17T21:29:57.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039187560000000003 2024-09-17T21:29:57.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181363+0000", 2024-09-17T21:29:57.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9375000000000002e-05 2024-09-17T21:29:57.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181417+0000", 2024-09-17T21:29:57.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3372000000000003e-05 2024-09-17T21:29:57.098 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.098 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.098 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.098 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12262 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.098 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:29:57.098 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.8940198869999998, 2024-09-17T21:29:57.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039384875, 2024-09-17T21:29:57.098 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.099 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.099 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.099 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.099 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.099 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.099 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12262 2024-09-17T21:29:57.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.099 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:29:57.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:29:57.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142143+0000", 2024-09-17T21:29:57.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:29:57.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142148+0000", 2024-09-17T21:29:57.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4360000000000001e-06 2024-09-17T21:29:57.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142154+0000", 2024-09-17T21:29:57.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8719999999999999e-06 2024-09-17T21:29:57.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142158+0000", 2024-09-17T21:29:57.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.476e-06 2024-09-17T21:29:57.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181436+0000", 2024-09-17T21:29:57.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039277919000000001 2024-09-17T21:29:57.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181480+0000", 2024-09-17T21:29:57.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3649999999999997e-05 2024-09-17T21:29:57.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181527+0000", 2024-09-17T21:29:57.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7131e-05 2024-09-17T21:29:57.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.103 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.103 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12263 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1079296~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.103 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:29:57.103 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.8939974519999998, 2024-09-17T21:29:57.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039473339000000003, 2024-09-17T21:29:57.103 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.103 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.103 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.103 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.103 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.104 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12263 2024-09-17T21:29:57.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.104 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:29:57.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:29:57.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:29:57.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:29:57.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142167+0000", 2024-09-17T21:29:57.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.81e-06 2024-09-17T21:29:57.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142172+0000", 2024-09-17T21:29:57.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:29:57.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142177+0000", 2024-09-17T21:29:57.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T21:29:57.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181553+0000", 2024-09-17T21:29:57.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039376130000000002 2024-09-17T21:29:57.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181592+0000", 2024-09-17T21:29:57.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9153999999999998e-05 2024-09-17T21:29:57.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181638+0000", 2024-09-17T21:29:57.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5864e-05 2024-09-17T21:29:57.107 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.107 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.107 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.107 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12278 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1095680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.107 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:29:57.107 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.8939794770000002, 2024-09-17T21:29:57.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039454222999999997, 2024-09-17T21:29:57.108 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.108 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.108 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.108 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12278 2024-09-17T21:29:57.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.108 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:29:57.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:29:57.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:29:57.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:29:57.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142185+0000", 2024-09-17T21:29:57.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.697e-06 2024-09-17T21:29:57.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142191+0000", 2024-09-17T21:29:57.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:29:57.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142196+0000", 2024-09-17T21:29:57.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4829999999999999e-06 2024-09-17T21:29:57.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181548+0000", 2024-09-17T21:29:57.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039351200000000003 2024-09-17T21:29:57.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181591+0000", 2024-09-17T21:29:57.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3492000000000003e-05 2024-09-17T21:29:57.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181637+0000", 2024-09-17T21:29:57.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5847000000000003e-05 2024-09-17T21:29:57.111 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.111 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.112 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.112 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12279 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.112 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:29:57.112 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.8939561879999998, 2024-09-17T21:29:57.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039539801999999999, 2024-09-17T21:29:57.112 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.112 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.112 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.112 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.112 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.112 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.113 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12279 2024-09-17T21:29:57.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.113 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:29:57.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:29:57.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:29:57.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.89e-07 2024-09-17T21:29:57.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142208+0000", 2024-09-17T21:29:57.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7579999999999999e-06 2024-09-17T21:29:57.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142214+0000", 2024-09-17T21:29:57.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:29:57.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142218+0000", 2024-09-17T21:29:57.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3429999999999998e-06 2024-09-17T21:29:57.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181653+0000", 2024-09-17T21:29:57.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039435322000000002 2024-09-17T21:29:57.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181698+0000", 2024-09-17T21:29:57.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4978999999999997e-05 2024-09-17T21:29:57.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181746+0000", 2024-09-17T21:29:57.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571999999999998e-05 2024-09-17T21:29:57.116 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.116 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.116 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.116 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12280 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1124352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.116 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:29:57.117 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.8939386689999997, 2024-09-17T21:29:57.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039704358000000002, 2024-09-17T21:29:57.117 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.117 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.117 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.117 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.117 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.117 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.117 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12280 2024-09-17T21:29:57.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.117 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:29:57.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:29:57.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142224+0000", 2024-09-17T21:29:57.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:29:57.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142226+0000", 2024-09-17T21:29:57.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:29:57.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142231+0000", 2024-09-17T21:29:57.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:29:57.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142240+0000", 2024-09-17T21:29:57.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4590000000000004e-06 2024-09-17T21:29:57.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181774+0000", 2024-09-17T21:29:57.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039533833999999997 2024-09-17T21:29:57.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181859+0000", 2024-09-17T21:29:57.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4328999999999998e-05 2024-09-17T21:29:57.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181928+0000", 2024-09-17T21:29:57.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9139000000000003e-05 2024-09-17T21:29:57.120 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.121 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.121 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.121 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12281 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1136640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.121 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:29:57.121 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.8939157629999999, 2024-09-17T21:29:57.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039787799999999998, 2024-09-17T21:29:57.121 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.121 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.121 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.121 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.121 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.121 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.122 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12281 2024-09-17T21:29:57.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.122 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:29:57.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:29:57.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142309+0000", 2024-09-17T21:29:57.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2323999999999995e-05 2024-09-17T21:29:57.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142317+0000", 2024-09-17T21:29:57.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1859999999999992e-06 2024-09-17T21:29:57.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142324+0000", 2024-09-17T21:29:57.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4699999999999996e-06 2024-09-17T21:29:57.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142330+0000", 2024-09-17T21:29:57.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4650000000000002e-06 2024-09-17T21:29:57.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181948+0000", 2024-09-17T21:29:57.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039618056999999998 2024-09-17T21:29:57.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181986+0000", 2024-09-17T21:29:57.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8442999999999998e-05 2024-09-17T21:29:57.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182034+0000", 2024-09-17T21:29:57.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7855000000000002e-05 2024-09-17T21:29:57.125 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.125 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.125 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.125 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12282 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1150976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.125 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:29:57.125 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.893823534, 2024-09-17T21:29:57.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039871919999999998, 2024-09-17T21:29:57.126 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.126 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.126 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.126 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.126 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.126 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.126 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12282 2024-09-17T21:29:57.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.126 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:29:57.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:29:57.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:29:57.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:29:57.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142341+0000", 2024-09-17T21:29:57.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.232e-06 2024-09-17T21:29:57.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142348+0000", 2024-09-17T21:29:57.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7320000000000001e-06 2024-09-17T21:29:57.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142353+0000", 2024-09-17T21:29:57.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:29:57.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182062+0000", 2024-09-17T21:29:57.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039709739000000001 2024-09-17T21:29:57.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182128+0000", 2024-09-17T21:29:57.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5739000000000002e-05 2024-09-17T21:29:57.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182210+0000", 2024-09-17T21:29:57.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:29:57.129 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.130 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.130 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.130 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12283 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.130 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:29:57.130 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.8938031229999996, 2024-09-17T21:29:57.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039896434000000001, 2024-09-17T21:29:57.130 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.130 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.130 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:29:57.130 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.130 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.131 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12283 2024-09-17T21:29:57.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.131 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:29:57.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:29:57.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144852+0000", 2024-09-17T21:29:57.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024934079999999999 2024-09-17T21:29:57.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144856+0000", 2024-09-17T21:29:57.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:29:57.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144864+0000", 2024-09-17T21:29:57.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0160000000000008e-06 2024-09-17T21:29:57.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144869+0000", 2024-09-17T21:29:57.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:29:57.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182246+0000", 2024-09-17T21:29:57.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037377207000000003 2024-09-17T21:29:57.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182255+0000", 2024-09-17T21:29:57.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1840000000000005e-06 2024-09-17T21:29:57.134 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.134 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.134 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.134 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.134 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.134 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:57.136 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:29:57.137 INFO:teuthology.orchestra.run.smithi129.stdout:1293 read 0x32e60cf thru 0x32e6838 (0x76a bytes) 2024-09-17T21:29:57.137 INFO:teuthology.orchestra.run.smithi129.stdout:1296 trunc from 0x7bbb987 to 0x4537957 2024-09-17T21:29:57.137 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.137 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.137 INFO:teuthology.orchestra.run.smithi129.stdout:1301 trunc from 0x4537957 to 0x52d550f 2024-09-17T21:29:57.137 INFO:teuthology.orchestra.run.smithi129.stdout:1302 write 0x9e0b701 thru 0x9e10b8d (0x548d bytes) 2024-09-17T21:29:57.137 INFO:teuthology.orchestra.run.smithi129.stdout:1303 write 0x966dd86 thru 0x967d5da (0xf855 bytes) 2024-09-17T21:29:57.137 INFO:teuthology.orchestra.run.smithi129.stdout:1304 trunc from 0x9e10b8e to 0x72ec5bd 2024-09-17T21:29:57.138 INFO:teuthology.orchestra.run.smithi129.stdout:1305 write 0xa260935 thru 0xa266a6d (0x6139 bytes) 2024-09-17T21:29:57.138 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.138 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.138 INFO:teuthology.orchestra.run.smithi129.stdout:1308 write 0xd7963b7 thru 0xd79aa95 (0x46df bytes) 2024-09-17T21:29:57.138 INFO:teuthology.orchestra.run.smithi129.stdout:1310 punch from 0xbe94797 to 0xbe9d7d9, (0x9042 bytes) 2024-09-17T21:29:57.138 INFO:teuthology.orchestra.run.smithi129.stdout:1311 write 0x3020202 thru 0x302f9e8 (0xf7e7 bytes) 2024-09-17T21:29:57.138 INFO:teuthology.orchestra.run.smithi129.stdout:1312 write 0x33ef37a thru 0x33fc9e1 (0xd668 bytes) 2024-09-17T21:29:57.138 INFO:teuthology.orchestra.run.smithi129.stdout:1314 write 0xb033998 thru 0xb0358a5 (0x1f0e bytes) 2024-09-17T21:29:57.138 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:57.133+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:57.138 INFO:teuthology.orchestra.run.smithi129.stdout:1317 trunc from 0xd79aa96 to 0x4d80396 2024-09-17T21:29:57.158 INFO:teuthology.orchestra.run.smithi129.stdout:1318 write 0x6ee4a1d thru 0x6ee5b3e (0x1122 bytes) 2024-09-17T21:29:57.159 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.164 INFO:teuthology.orchestra.run.smithi129.stdout:1320 trunc from 0x6ee5b3f to 0x350538c 2024-09-17T21:29:57.170 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.170 INFO:teuthology.orchestra.run.smithi129.stdout:1324 writesame 0x22994d8 thru 0x22a17cf data_size 0x58(0x82f8 bytes) 2024-09-17T21:29:57.171 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.171 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.171 INFO:teuthology.orchestra.run.smithi129.stdout:1328 punch from 0x4c06ca to 0x4c26d5, (0x200b bytes) 2024-09-17T21:29:57.171 INFO:teuthology.orchestra.run.smithi129.stdout:1329 punch from 0x2cecd8e to 0x2cedda6, (0x1018 bytes) 2024-09-17T21:29:57.172 INFO:teuthology.orchestra.run.smithi129.stdout:1330 read 0x336f599 thru 0x337a30f (0xad77 bytes) 2024-09-17T21:29:57.172 INFO:teuthology.orchestra.run.smithi129.stdout:1332 punch from 0x1d5f875 to 0x1d63de8, (0x4573 bytes) 2024-09-17T21:29:57.172 INFO:teuthology.orchestra.run.smithi129.stdout:1333 read 0x6231b5 thru 0x62c176 (0x8fc2 bytes) 2024-09-17T21:29:57.173 INFO:teuthology.orchestra.run.smithi129.stdout:1335 punch from 0x2b34812 to 0x2b44554, (0xfd42 bytes) 2024-09-17T21:29:57.173 INFO:teuthology.orchestra.run.smithi129.stdout:1337 write 0x131ee29 thru 0x132d637 (0xe80f bytes) 2024-09-17T21:29:57.190 INFO:teuthology.orchestra.run.smithi129.stdout:1338 write 0xc95d679 thru 0xc96475d (0x70e5 bytes) 2024-09-17T21:29:57.193 INFO:teuthology.orchestra.run.smithi129.stdout:1341 writesame 0x6869e2e thru 0x6879017 data_size 0x37(0xf1ea bytes) 2024-09-17T21:29:57.194 INFO:teuthology.orchestra.run.smithi129.stdout:1342 read 0x1022387 thru 0x102962a (0x72a4 bytes) 2024-09-17T21:29:57.195 INFO:teuthology.orchestra.run.smithi129.stdout:1343 write 0x978bdc2 thru 0x979b538 (0xf777 bytes) 2024-09-17T21:29:57.197 INFO:teuthology.orchestra.run.smithi129.stdout:1344 write 0x27a848e thru 0x27b5f0a (0xda7d bytes) 2024-09-17T21:29:57.198 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.199 INFO:teuthology.orchestra.run.smithi129.stdout:1348 write 0xa7504c4 thru 0xa75142d (0xf6a bytes) 2024-09-17T21:29:57.200 INFO:teuthology.orchestra.run.smithi129.stdout:1349 read 0x667e98c thru 0x6680ab2 (0x2127 bytes) 2024-09-17T21:29:57.200 INFO:teuthology.orchestra.run.smithi129.stdout:1350 write 0x7f889d7 thru 0x7f93627 (0xac51 bytes) 2024-09-17T21:29:57.202 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.207 INFO:teuthology.orchestra.run.smithi129.stdout:1352 write 0xd073064 thru 0xd0757cb (0x2768 bytes) 2024-09-17T21:29:57.224 INFO:teuthology.orchestra.run.smithi129.stdout:1353 trunc from 0xd0757cc to 0x15b5517 2024-09-17T21:29:57.236 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:57.234+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:57.246 INFO:teuthology.orchestra.run.smithi129.stdout:1354 writesame 0x5b0ea27 thru 0x5b13644 data_size 0x1(0x4c1e bytes) 2024-09-17T21:29:57.248 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.248 INFO:teuthology.orchestra.run.smithi129.stdout:1357 punch from 0x5d18e3 to 0x5d8833, (0x6f50 bytes) 2024-09-17T21:29:57.248 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.250 INFO:teuthology.orchestra.run.smithi129.stdout:1359 trunc from 0x5b13645 to 0x34719c2 2024-09-17T21:29:57.257 INFO:teuthology.orchestra.run.smithi129.stdout:1361 write 0xc93fb0 thru 0xc9e592 (0xa5e3 bytes) 2024-09-17T21:29:57.258 INFO:teuthology.orchestra.run.smithi129.stdout:1362 read 0x1037d9a thru 0x103f00b (0x7272 bytes) 2024-09-17T21:29:57.266 INFO:teuthology.orchestra.run.smithi129.stdout:1363 trunc from 0x34719c2 to 0x9bfb06d 2024-09-17T21:29:57.271 INFO:teuthology.orchestra.run.smithi129.stdout:1364 write 0x2e7344c thru 0x2e7730e (0x3ec3 bytes) 2024-09-17T21:29:57.273 INFO:teuthology.orchestra.run.smithi129.stdout:1367 write 0x1dec215 thru 0x1df2d83 (0x6b6f bytes) 2024-09-17T21:29:57.274 INFO:teuthology.orchestra.run.smithi129.stdout:1368 write 0x3a6ca9f thru 0x3a6f853 (0x2db5 bytes) 2024-09-17T21:29:57.278 INFO:teuthology.orchestra.run.smithi129.stdout:1369 trunc from 0x9bfb06d to 0xb320b98 2024-09-17T21:29:57.283 INFO:teuthology.orchestra.run.smithi129.stdout:1370 read 0x8e77d18 thru 0x8e7b414 (0x36fd bytes) 2024-09-17T21:29:57.283 INFO:teuthology.orchestra.run.smithi129.stdout:1371 write 0x1023a68 thru 0x1032e3a (0xf3d3 bytes) 2024-09-17T21:29:57.285 INFO:teuthology.orchestra.run.smithi129.stdout:1372 punch from 0x65cba5b to 0x65d847f, (0xca24 bytes) 2024-09-17T21:29:57.285 INFO:teuthology.orchestra.run.smithi129.stdout:1374 read 0x359fd74 thru 0x35a96fe (0x998b bytes) 2024-09-17T21:29:57.285 INFO:teuthology.orchestra.run.smithi129.stdout:1378 punch from 0x1d287e0 to 0x1d311f5, (0x8a15 bytes) 2024-09-17T21:29:57.285 INFO:teuthology.orchestra.run.smithi129.stdout:1379 write 0xa6a5b96 thru 0xa6b371a (0xdb85 bytes) 2024-09-17T21:29:57.287 INFO:teuthology.orchestra.run.smithi129.stdout:1380 punch from 0x9051750 to 0x905b9d7, (0xa287 bytes) 2024-09-17T21:29:57.287 INFO:teuthology.orchestra.run.smithi129.stdout:1381 write 0xbe20a3 thru 0xbf13d2 (0xf330 bytes) 2024-09-17T21:29:57.288 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:57.288 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:29:57.288 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:29:57.288 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:57.295 INFO:teuthology.orchestra.run.smithi129.stdout:1382 trunc from 0xb320b98 to 0x5470299 2024-09-17T21:29:57.298 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:29:57.320 INFO:teuthology.orchestra.run.smithi129.stdout:1383 write 0xd9e3de9 thru 0xd9f2338 (0xe550 bytes) 2024-09-17T21:29:57.323 INFO:teuthology.orchestra.run.smithi129.stdout:1384 writesame 0xa170f66 thru 0xa17bdd6 data_size 0x1(0xae71 bytes) 2024-09-17T21:29:57.337 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:57.335+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:29:57.342 INFO:teuthology.orchestra.run.smithi129.stdout:1385 read 0x4b4dd95 thru 0x4b5bd21 (0xdf8d bytes) 2024-09-17T21:29:57.342 INFO:teuthology.orchestra.run.smithi129.stdout:1386 write 0xa2b0cea thru 0xa2b1005 (0x31c bytes) 2024-09-17T21:29:57.344 INFO:teuthology.orchestra.run.smithi129.stdout:1388 read 0xc4bced8 thru 0xc4bda05 (0xb2e bytes) 2024-09-17T21:29:57.344 INFO:teuthology.orchestra.run.smithi129.stdout:1390 read 0x9bc4679 thru 0x9bcc81b (0x81a3 bytes) 2024-09-17T21:29:57.344 INFO:teuthology.orchestra.run.smithi129.stdout:1391 read 0x69ada5 thru 0x6aa2e3 (0xf53f bytes) 2024-09-17T21:29:57.345 INFO:teuthology.orchestra.run.smithi129.stdout:1392 write 0x121481a thru 0x12150fa (0x8e1 bytes) 2024-09-17T21:29:57.351 INFO:teuthology.orchestra.run.smithi129.stdout:1393 write 0xdb5a524 thru 0xdb5e47d (0x3f5a bytes) 2024-09-17T21:29:57.353 INFO:teuthology.orchestra.run.smithi129.stdout:1397 punch from 0x5df118a to 0x5e0087d, (0xf6f3 bytes) 2024-09-17T21:29:57.353 INFO:teuthology.orchestra.run.smithi129.stdout:1398 read 0x1baf131 thru 0x1bb6a27 (0x78f7 bytes) 2024-09-17T21:29:57.353 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.353 INFO:teuthology.orchestra.run.smithi129.stdout:1400 writesame 0x5b053e4 thru 0x5b0f2b2 data_size 0x5(0x9ecf bytes) 2024-09-17T21:29:57.357 INFO:teuthology.orchestra.run.smithi129.stdout:1401 write 0x70ec48f thru 0x70fa73c (0xe2ae bytes) 2024-09-17T21:29:57.358 INFO:teuthology.orchestra.run.smithi129.stdout:1403 writesame 0x9792072 thru 0x979379d data_size 0x2(0x172c bytes) 2024-09-17T21:29:57.361 INFO:teuthology.orchestra.run.smithi129.stdout:1405 writesame 0x4c26153 thru 0x4c26468 data_size 0x5(0x316 bytes) 2024-09-17T21:29:57.368 INFO:teuthology.orchestra.run.smithi129.stdout:1406 write 0xe1aabad thru 0xe1b6372 (0xb7c6 bytes) 2024-09-17T21:29:57.371 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.371 INFO:teuthology.orchestra.run.smithi129.stdout:1408 read 0xbb6146e thru 0xbb6f90a (0xe49d bytes) 2024-09-17T21:29:57.371 INFO:teuthology.orchestra.run.smithi129.stdout:1409 punch from 0x55ce94c to 0x55d759a, (0x8c4e bytes) 2024-09-17T21:29:57.372 INFO:teuthology.orchestra.run.smithi129.stdout:1410 punch from 0x20692fc to 0x206d036, (0x3d3a bytes) 2024-09-17T21:29:57.372 INFO:teuthology.orchestra.run.smithi129.stdout:1411 write 0x94e0c41 thru 0x94f05cc (0xf98c bytes) 2024-09-17T21:29:57.376 INFO:teuthology.orchestra.run.smithi129.stdout:1412 trunc from 0xe1b6373 to 0xb359091 2024-09-17T21:29:57.390 INFO:teuthology.orchestra.run.smithi129.stdout:1413 read 0x63fa452 thru 0x63fd4fc (0x30ab bytes) 2024-09-17T21:29:57.390 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.390 INFO:teuthology.orchestra.run.smithi129.stdout:1415 write 0x20ab3e2 thru 0x20aeb4e (0x376d bytes) 2024-09-17T21:29:57.402 INFO:teuthology.orchestra.run.smithi129.stdout:1416 writesame 0xe623680 thru 0xe623d84 data_size 0x1(0x705 bytes) 2024-09-17T21:29:57.402 INFO:teuthology.orchestra.run.smithi129.stdout:1417 read 0x3268076 thru 0x326c018 (0x3fa3 bytes) 2024-09-17T21:29:57.403 INFO:teuthology.orchestra.run.smithi129.stdout:1418 writesame 0xca145a7 thru 0xca1fa57 data_size 0x11(0xb4b1 bytes) 2024-09-17T21:29:57.406 INFO:teuthology.orchestra.run.smithi129.stdout:1420 read 0xcc81cd0 thru 0xcc8b204 (0x9535 bytes) 2024-09-17T21:29:57.406 INFO:teuthology.orchestra.run.smithi129.stdout:1422 write 0x30455f8 thru 0x3048f41 (0x394a bytes) 2024-09-17T21:29:57.408 INFO:teuthology.orchestra.run.smithi129.stdout:1424 write 0xc931e39 thru 0xc93483b (0x2a03 bytes) 2024-09-17T21:29:57.410 INFO:teuthology.orchestra.run.smithi129.stdout:1425 write 0x88a2944 thru 0x88aba2d (0x90ea bytes) 2024-09-17T21:29:57.416 INFO:teuthology.orchestra.run.smithi129.stdout:1426 write 0xe857f0b thru 0xe86530d (0xd403 bytes) 2024-09-17T21:29:57.419 INFO:teuthology.orchestra.run.smithi129.stdout:1428 read 0x84d3e73 thru 0x84df219 (0xb3a7 bytes) 2024-09-17T21:29:57.436 INFO:teuthology.orchestra.run.smithi129.stdout:1429 trunc from 0xe86530e to 0xf59300 2024-09-17T21:29:57.438 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:57.436+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:57.450 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:57.450 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:29:57.450 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:29:57.450 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:29:57.450 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:57.458 INFO:teuthology.orchestra.run.smithi129.stdout:1431 punch from 0x79cff3 to 0x7a5f28, (0x8f35 bytes) 2024-09-17T21:29:57.459 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:29:57.473 INFO:teuthology.orchestra.run.smithi129.stdout:1432 write 0x7c703bc thru 0x7c766a5 (0x62ea bytes) 2024-09-17T21:29:57.475 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.475 INFO:teuthology.orchestra.run.smithi129.stdout:1434 write 0x5702620 thru 0x5711d8f (0xf770 bytes) 2024-09-17T21:29:57.476 INFO:teuthology.orchestra.run.smithi129.stdout:1435 write 0x484e92a thru 0x4852f87 (0x465e bytes) 2024-09-17T21:29:57.478 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.478 INFO:teuthology.orchestra.run.smithi129.stdout:1438 write 0x2ba7e9 thru 0x2c766b (0xce83 bytes) 2024-09-17T21:29:57.479 INFO:teuthology.orchestra.run.smithi129.stdout:1440 writesame 0xce0bc3 thru 0xceeadd data_size 0x1(0xdf1b bytes) 2024-09-17T21:29:57.501 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:29:57.501 INFO:teuthology.orchestra.run.smithi129.stdout:1443 read 0x2f1d2c6 thru 0x2f21e6d (0x4ba8 bytes) 2024-09-17T21:29:57.501 INFO:teuthology.orchestra.run.smithi129.stdout:1445 punch from 0xee7f0e to 0xeeec69, (0x6d5b bytes) 2024-09-17T21:29:57.508 INFO:teuthology.orchestra.run.smithi129.stdout:1446 trunc from 0x7c766a6 to 0xd629d44 2024-09-17T21:29:57.513 INFO:teuthology.orchestra.run.smithi129.stdout:1447 read 0xb9d634f thru 0xb9db66d (0x531f bytes) 2024-09-17T21:29:57.519 INFO:teuthology.orchestra.run.smithi129.stdout:1448 write 0xdfffde2 thru 0xe002bf4 (0x2e13 bytes) 2024-09-17T21:29:57.525 INFO:teuthology.orchestra.run.smithi129.stdout:1449 write 0xcaf9267 thru 0xcaf9f86 (0xd20 bytes) 2024-09-17T21:29:57.527 INFO:teuthology.orchestra.run.smithi129.stdout:1450 read 0x2eef29a thru 0x2efb4c7 (0xc22e bytes) 2024-09-17T21:29:57.527 INFO:teuthology.orchestra.run.smithi129.stdout:1451 write 0x5928a8d thru 0x5937e19 (0xf38d bytes) 2024-09-17T21:29:57.531 INFO:teuthology.orchestra.run.smithi129.stdout:1452 writesame 0x3887a9e thru 0x388e20b data_size 0x9(0x676e bytes) 2024-09-17T21:29:57.534 INFO:teuthology.orchestra.run.smithi129.stdout:1453 read 0xb73b27c thru 0xb73ba9b (0x820 bytes) 2024-09-17T21:29:57.534 INFO:teuthology.orchestra.run.smithi129.stdout:1454 punch from 0xdb90892 to 0xdb9455b, (0x3cc9 bytes) 2024-09-17T21:29:57.534 INFO:teuthology.orchestra.run.smithi129.stdout:1455 writesame 0x147f2c4 thru 0x148c810 data_size 0x43(0xd54d bytes) 2024-09-17T21:29:57.537 INFO:teuthology.orchestra.run.smithi129.stdout:1456 write 0x51ce178 thru 0x51d2c80 (0x4b09 bytes) 2024-09-17T21:29:57.539 INFO:teuthology.orchestra.run.smithi129.stdout:1457 clone 8 order 21 su 524288 sc 6 2024-09-17T21:29:57.543 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:29:57.544 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:57.542+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:57.575 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T21:29:57.575 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:29:57.575 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T21:29:57.575 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2024-09-17T21:29:57.605 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:57.605 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:29:57.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:29:57.605 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:29:57.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.605 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:2881 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:57.605 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.556463+0000", 2024-09-17T21:29:57.606 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.045742159, 2024-09-17T21:29:57.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010359903, 2024-09-17T21:29:57.606 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.606 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.606 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:57.606 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.606 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.606 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.606 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 2881 2024-09-17T21:29:57.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.606 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556463+0000", 2024-09-17T21:29:57.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556463+0000", 2024-09-17T21:29:57.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556465+0000", 2024-09-17T21:29:57.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:29:57.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556485+0000", 2024-09-17T21:29:57.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9865999999999999e-05 2024-09-17T21:29:57.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556522+0000", 2024-09-17T21:29:57.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7126999999999999e-05 2024-09-17T21:29:57.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556544+0000", 2024-09-17T21:29:57.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2135999999999999e-05 2024-09-17T21:29:57.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556748+0000", 2024-09-17T21:29:57.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000204222 2024-09-17T21:29:57.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.556829+0000", 2024-09-17T21:29:57.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0049999999999994e-05 2024-09-17T21:29:57.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:29:57.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557015+0000", 2024-09-17T21:29:57.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018693200000000001 2024-09-17T21:29:57.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:57.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.557744+0000", 2024-09-17T21:29:57.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00072873299999999996 2024-09-17T21:29:57.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:57.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.566721+0000", 2024-09-17T21:29:57.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0089770119999999995 2024-09-17T21:29:57.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:57.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.566779+0000", 2024-09-17T21:29:57.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8041000000000002e-05 2024-09-17T21:29:57.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.566823+0000", 2024-09-17T21:29:57.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3343999999999997e-05 2024-09-17T21:29:57.612 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.612 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.612 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.612 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4746 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 1363968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:57.613 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.958850+0000", 2024-09-17T21:29:57.613 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.6433551870000009, 2024-09-17T21:29:57.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073491540000000001, 2024-09-17T21:29:57.613 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.613 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.613 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.613 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.613 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.613 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.613 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4746 2024-09-17T21:29:57.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.614 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958850+0000", 2024-09-17T21:29:57.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958850+0000", 2024-09-17T21:29:57.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958850+0000", 2024-09-17T21:29:57.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:29:57.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958852+0000", 2024-09-17T21:29:57.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.091e-06 2024-09-17T21:29:57.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958858+0000", 2024-09-17T21:29:57.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2349999999999996e-06 2024-09-17T21:29:57.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958863+0000", 2024-09-17T21:29:57.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.656e-06 2024-09-17T21:29:57.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966041+0000", 2024-09-17T21:29:57.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0071779510000000001 2024-09-17T21:29:57.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966122+0000", 2024-09-17T21:29:57.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0947999999999998e-05 2024-09-17T21:29:57.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966199+0000", 2024-09-17T21:29:57.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6843999999999998e-05 2024-09-17T21:29:57.617 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.617 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.618 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.618 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4747 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 1378304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:57.618 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.958869+0000", 2024-09-17T21:29:57.618 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.6433354500000004, 2024-09-17T21:29:57.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075441639999999999, 2024-09-17T21:29:57.618 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.618 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.618 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.618 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.618 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.619 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.619 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4747 2024-09-17T21:29:57.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.619 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958869+0000", 2024-09-17T21:29:57.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958869+0000", 2024-09-17T21:29:57.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958870+0000", 2024-09-17T21:29:57.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3399999999999999e-07 2024-09-17T21:29:57.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958872+0000", 2024-09-17T21:29:57.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:29:57.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958878+0000", 2024-09-17T21:29:57.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0430000000000004e-06 2024-09-17T21:29:57.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958883+0000", 2024-09-17T21:29:57.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8620000000000002e-06 2024-09-17T21:29:57.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966238+0000", 2024-09-17T21:29:57.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073557700000000002 2024-09-17T21:29:57.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966331+0000", 2024-09-17T21:29:57.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2048999999999999e-05 2024-09-17T21:29:57.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966414+0000", 2024-09-17T21:29:57.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:29:57.623 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.623 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.623 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.623 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4748 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 1390592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:57.623 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.958889+0000", 2024-09-17T21:29:57.623 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.6433156869999994, 2024-09-17T21:29:57.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077162630000000001, 2024-09-17T21:29:57.624 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.624 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.624 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.624 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.624 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.624 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.624 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4748 2024-09-17T21:29:57.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.624 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958889+0000", 2024-09-17T21:29:57.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958889+0000", 2024-09-17T21:29:57.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958890+0000", 2024-09-17T21:29:57.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:29:57.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958899+0000", 2024-09-17T21:29:57.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0699999999999996e-06 2024-09-17T21:29:57.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958905+0000", 2024-09-17T21:29:57.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2820000000000002e-06 2024-09-17T21:29:57.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958910+0000", 2024-09-17T21:29:57.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7570000000000004e-06 2024-09-17T21:29:57.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966459+0000", 2024-09-17T21:29:57.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075494009999999999 2024-09-17T21:29:57.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966550+0000", 2024-09-17T21:29:57.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0603e-05 2024-09-17T21:29:57.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966605+0000", 2024-09-17T21:29:57.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5754999999999997e-05 2024-09-17T21:29:57.628 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.628 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.629 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.629 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4749 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 1404928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:57.629 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.958916+0000", 2024-09-17T21:29:57.629 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.6432885069999994, 2024-09-17T21:29:57.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0078880389999999995, 2024-09-17T21:29:57.629 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.629 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.629 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.629 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.629 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.630 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.630 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4749 2024-09-17T21:29:57.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.630 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958916+0000", 2024-09-17T21:29:57.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958916+0000", 2024-09-17T21:29:57.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958917+0000", 2024-09-17T21:29:57.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4299999999999998e-07 2024-09-17T21:29:57.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958919+0000", 2024-09-17T21:29:57.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9130000000000001e-06 2024-09-17T21:29:57.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958930+0000", 2024-09-17T21:29:57.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1401e-05 2024-09-17T21:29:57.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958935+0000", 2024-09-17T21:29:57.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1479999999999997e-06 2024-09-17T21:29:57.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966649+0000", 2024-09-17T21:29:57.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077142110000000003 2024-09-17T21:29:57.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966725+0000", 2024-09-17T21:29:57.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5115000000000002e-05 2024-09-17T21:29:57.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966804+0000", 2024-09-17T21:29:57.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9808000000000006e-05 2024-09-17T21:29:57.634 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.634 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.634 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.634 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:4750 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 1419264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:29:57.634 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:47.958942+0000", 2024-09-17T21:29:57.635 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.6432624140000005, 2024-09-17T21:29:57.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0080617380000000006, 2024-09-17T21:29:57.635 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.635 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.635 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.635 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.635 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.635 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.635 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4750 2024-09-17T21:29:57.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.635 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958942+0000", 2024-09-17T21:29:57.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958942+0000", 2024-09-17T21:29:57.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958943+0000", 2024-09-17T21:29:57.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1199999999999998e-07 2024-09-17T21:29:57.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958945+0000", 2024-09-17T21:29:57.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0269999999999998e-06 2024-09-17T21:29:57.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958951+0000", 2024-09-17T21:29:57.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9719999999999997e-06 2024-09-17T21:29:57.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.958964+0000", 2024-09-17T21:29:57.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3078000000000001e-05 2024-09-17T21:29:57.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966848+0000", 2024-09-17T21:29:57.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0078837279999999996 2024-09-17T21:29:57.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.966930+0000", 2024-09-17T21:29:57.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2352e-05 2024-09-17T21:29:57.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:47.967004+0000", 2024-09-17T21:29:57.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4169000000000001e-05 2024-09-17T21:29:57.639 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.639 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.639 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.640 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16082 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 477184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.640 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.330151+0000", 2024-09-17T21:29:57.640 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.2720533839999995, 2024-09-17T21:29:57.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075125319999999997, 2024-09-17T21:29:57.640 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.640 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.640 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.640 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.641 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.641 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.641 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16082 2024-09-17T21:29:57.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.641 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.330151+0000", 2024-09-17T21:29:57.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.330151+0000", 2024-09-17T21:29:57.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.330152+0000", 2024-09-17T21:29:57.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:29:57.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.330154+0000", 2024-09-17T21:29:57.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2230000000000001e-06 2024-09-17T21:29:57.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.330160+0000", 2024-09-17T21:29:57.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.84e-06 2024-09-17T21:29:57.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.330164+0000", 2024-09-17T21:29:57.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.352e-06 2024-09-17T21:29:57.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.337468+0000", 2024-09-17T21:29:57.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007303698 2024-09-17T21:29:57.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.337559+0000", 2024-09-17T21:29:57.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1423999999999997e-05 2024-09-17T21:29:57.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.337664+0000", 2024-09-17T21:29:57.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104598 2024-09-17T21:29:57.645 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.645 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.645 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.645 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16174 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.646 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.332031+0000", 2024-09-17T21:29:57.646 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.2701734180000006, 2024-09-17T21:29:57.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073554650000000003, 2024-09-17T21:29:57.646 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.646 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.646 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.646 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.646 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.646 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.646 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16174 2024-09-17T21:29:57.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.647 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332031+0000", 2024-09-17T21:29:57.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332031+0000", 2024-09-17T21:29:57.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332032+0000", 2024-09-17T21:29:57.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:29:57.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332034+0000", 2024-09-17T21:29:57.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.012e-06 2024-09-17T21:29:57.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332040+0000", 2024-09-17T21:29:57.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.908e-06 2024-09-17T21:29:57.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332044+0000", 2024-09-17T21:29:57.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2899999999999996e-06 2024-09-17T21:29:57.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339213+0000", 2024-09-17T21:29:57.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0071694610000000002 2024-09-17T21:29:57.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339295+0000", 2024-09-17T21:29:57.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1806999999999998e-05 2024-09-17T21:29:57.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339387+0000", 2024-09-17T21:29:57.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1605000000000001e-05 2024-09-17T21:29:57.651 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.651 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.651 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.651 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16175 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 411648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.651 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.332051+0000", 2024-09-17T21:29:57.651 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.2701538590000006, 2024-09-17T21:29:57.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075336270000000002, 2024-09-17T21:29:57.651 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.652 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.652 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.652 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.652 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.652 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.652 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16175 2024-09-17T21:29:57.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.652 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332051+0000", 2024-09-17T21:29:57.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332051+0000", 2024-09-17T21:29:57.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332051+0000", 2024-09-17T21:29:57.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T21:29:57.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332053+0000", 2024-09-17T21:29:57.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8199999999999999e-06 2024-09-17T21:29:57.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332059+0000", 2024-09-17T21:29:57.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.293e-06 2024-09-17T21:29:57.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332064+0000", 2024-09-17T21:29:57.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6210000000000001e-06 2024-09-17T21:29:57.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339418+0000", 2024-09-17T21:29:57.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073537250000000002 2024-09-17T21:29:57.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339497+0000", 2024-09-17T21:29:57.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9425000000000006e-05 2024-09-17T21:29:57.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339585+0000", 2024-09-17T21:29:57.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7321e-05 2024-09-17T21:29:57.656 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.656 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.656 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.657 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16176 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 423936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.657 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.332070+0000", 2024-09-17T21:29:57.657 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.2701344940000006, 2024-09-17T21:29:57.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077110490000000002, 2024-09-17T21:29:57.657 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.657 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.657 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.657 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.657 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.657 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.658 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16176 2024-09-17T21:29:57.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.658 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332070+0000", 2024-09-17T21:29:57.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332070+0000", 2024-09-17T21:29:57.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332071+0000", 2024-09-17T21:29:57.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T21:29:57.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332073+0000", 2024-09-17T21:29:57.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8530000000000001e-06 2024-09-17T21:29:57.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332078+0000", 2024-09-17T21:29:57.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8409999999999996e-06 2024-09-17T21:29:57.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332083+0000", 2024-09-17T21:29:57.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8509999999999998e-06 2024-09-17T21:29:57.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339615+0000", 2024-09-17T21:29:57.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075321609999999999 2024-09-17T21:29:57.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339698+0000", 2024-09-17T21:29:57.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2663999999999993e-05 2024-09-17T21:29:57.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339781+0000", 2024-09-17T21:29:57.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3304000000000001e-05 2024-09-17T21:29:57.662 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.662 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.662 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.662 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16177 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.662 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.332089+0000", 2024-09-17T21:29:57.662 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.2701154359999993, 2024-09-17T21:29:57.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079627510000000006, 2024-09-17T21:29:57.662 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.663 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.663 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.663 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.663 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.663 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.663 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16177 2024-09-17T21:29:57.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.663 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332089+0000", 2024-09-17T21:29:57.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332089+0000", 2024-09-17T21:29:57.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332090+0000", 2024-09-17T21:29:57.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:29:57.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332093+0000", 2024-09-17T21:29:57.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7310000000000002e-06 2024-09-17T21:29:57.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332100+0000", 2024-09-17T21:29:57.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5820000000000004e-06 2024-09-17T21:29:57.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332105+0000", 2024-09-17T21:29:57.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5220000000000001e-06 2024-09-17T21:29:57.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339810+0000", 2024-09-17T21:29:57.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007705796 2024-09-17T21:29:57.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.339958+0000", 2024-09-17T21:29:57.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000147391 2024-09-17T21:29:57.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.340052+0000", 2024-09-17T21:29:57.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4313000000000003e-05 2024-09-17T21:29:57.667 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.667 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.667 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.667 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16178 3.a 3.150c2d0a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.668 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.332111+0000", 2024-09-17T21:29:57.668 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.2700933849999991, 2024-09-17T21:29:57.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079742430000000007, 2024-09-17T21:29:57.668 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.668 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.668 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:29:57.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.668 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16178 2024-09-17T21:29:57.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.669 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332111+0000", 2024-09-17T21:29:57.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332111+0000", 2024-09-17T21:29:57.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332112+0000", 2024-09-17T21:29:57.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.58e-07 2024-09-17T21:29:57.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332117+0000", 2024-09-17T21:29:57.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4929999999999998e-06 2024-09-17T21:29:57.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332125+0000", 2024-09-17T21:29:57.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2169999999999997e-06 2024-09-17T21:29:57.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332130+0000", 2024-09-17T21:29:57.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1630000000000004e-06 2024-09-17T21:29:57.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.340080+0000", 2024-09-17T21:29:57.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079505689999999993 2024-09-17T21:29:57.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.340086+0000", 2024-09-17T21:29:57.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3430000000000003e-06 2024-09-17T21:29:57.672 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.672 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.672 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.672 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16179 3.a 3.150c2d0a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.672 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.332136+0000", 2024-09-17T21:29:57.673 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.2700684580000008, 2024-09-17T21:29:57.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079794670000000005, 2024-09-17T21:29:57.673 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.673 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.673 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:29:57.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.673 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16179 2024-09-17T21:29:57.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.673 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332136+0000", 2024-09-17T21:29:57.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332136+0000", 2024-09-17T21:29:57.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332137+0000", 2024-09-17T21:29:57.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0900000000000002e-07 2024-09-17T21:29:57.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332139+0000", 2024-09-17T21:29:57.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0549999999999998e-06 2024-09-17T21:29:57.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332145+0000", 2024-09-17T21:29:57.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.088e-06 2024-09-17T21:29:57.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332150+0000", 2024-09-17T21:29:57.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6399999999999996e-06 2024-09-17T21:29:57.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.340111+0000", 2024-09-17T21:29:57.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079615510000000007 2024-09-17T21:29:57.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.340116+0000", 2024-09-17T21:29:57.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.724e-06 2024-09-17T21:29:57.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.677 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.677 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16180 3.a 3.150c2d0a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.677 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.332156+0000", 2024-09-17T21:29:57.677 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.2700490089999992, 2024-09-17T21:29:57.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079971169999999998, 2024-09-17T21:29:57.678 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.678 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.678 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:29:57.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.678 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16180 2024-09-17T21:29:57.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.678 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332156+0000", 2024-09-17T21:29:57.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332156+0000", 2024-09-17T21:29:57.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332156+0000", 2024-09-17T21:29:57.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:29:57.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332158+0000", 2024-09-17T21:29:57.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.846e-06 2024-09-17T21:29:57.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332165+0000", 2024-09-17T21:29:57.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9970000000000001e-06 2024-09-17T21:29:57.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.332170+0000", 2024-09-17T21:29:57.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7859999999999996e-06 2024-09-17T21:29:57.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.340144+0000", 2024-09-17T21:29:57.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079745760000000006 2024-09-17T21:29:57.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.340153+0000", 2024-09-17T21:29:57.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5329999999999994e-06 2024-09-17T21:29:57.682 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.682 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.682 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.682 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16468 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 610304~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.682 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.342505+0000", 2024-09-17T21:29:57.682 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.2596995290000006, 2024-09-17T21:29:57.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007360213, 2024-09-17T21:29:57.682 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.683 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.683 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.683 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.683 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.683 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.683 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16468 2024-09-17T21:29:57.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.683 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342505+0000", 2024-09-17T21:29:57.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342505+0000", 2024-09-17T21:29:57.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342506+0000", 2024-09-17T21:29:57.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5400000000000002e-07 2024-09-17T21:29:57.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342508+0000", 2024-09-17T21:29:57.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.88e-06 2024-09-17T21:29:57.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342513+0000", 2024-09-17T21:29:57.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5670000000000002e-06 2024-09-17T21:29:57.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342518+0000", 2024-09-17T21:29:57.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6619999999999996e-06 2024-09-17T21:29:57.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.349747+0000", 2024-09-17T21:29:57.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0072289060000000002 2024-09-17T21:29:57.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.349801+0000", 2024-09-17T21:29:57.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4326000000000001e-05 2024-09-17T21:29:57.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.349866+0000", 2024-09-17T21:29:57.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4517999999999995e-05 2024-09-17T21:29:57.687 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.687 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.687 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.687 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16469 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 624640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.687 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.342549+0000", 2024-09-17T21:29:57.688 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.259655639, 2024-09-17T21:29:57.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0074639520000000003, 2024-09-17T21:29:57.688 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.688 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.688 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.688 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.688 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.688 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.688 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16469 2024-09-17T21:29:57.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.688 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342549+0000", 2024-09-17T21:29:57.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342549+0000", 2024-09-17T21:29:57.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342550+0000", 2024-09-17T21:29:57.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9299999999999998e-07 2024-09-17T21:29:57.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342552+0000", 2024-09-17T21:29:57.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4509999999999999e-06 2024-09-17T21:29:57.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342559+0000", 2024-09-17T21:29:57.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2389999999999996e-06 2024-09-17T21:29:57.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342565+0000", 2024-09-17T21:29:57.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5849999999999999e-06 2024-09-17T21:29:57.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.349892+0000", 2024-09-17T21:29:57.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073271650000000001 2024-09-17T21:29:57.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.349949+0000", 2024-09-17T21:29:57.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6854999999999997e-05 2024-09-17T21:29:57.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350013+0000", 2024-09-17T21:29:57.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4064000000000002e-05 2024-09-17T21:29:57.692 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.692 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.692 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.693 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16470 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 636928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.693 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.342573+0000", 2024-09-17T21:29:57.693 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.2596318889999996, 2024-09-17T21:29:57.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075758229999999998, 2024-09-17T21:29:57.693 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.693 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.693 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.693 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.693 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.693 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.694 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16470 2024-09-17T21:29:57.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.694 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342573+0000", 2024-09-17T21:29:57.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342573+0000", 2024-09-17T21:29:57.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342573+0000", 2024-09-17T21:29:57.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6800000000000001e-07 2024-09-17T21:29:57.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342575+0000", 2024-09-17T21:29:57.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.984e-06 2024-09-17T21:29:57.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342581+0000", 2024-09-17T21:29:57.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8690000000000001e-06 2024-09-17T21:29:57.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342586+0000", 2024-09-17T21:29:57.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4889999999999997e-06 2024-09-17T21:29:57.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350046+0000", 2024-09-17T21:29:57.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0074604249999999997 2024-09-17T21:29:57.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350097+0000", 2024-09-17T21:29:57.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0464999999999999e-05 2024-09-17T21:29:57.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350149+0000", 2024-09-17T21:29:57.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2123000000000002e-05 2024-09-17T21:29:57.698 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.698 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.698 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.698 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16471 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 651264~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.698 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.342592+0000", 2024-09-17T21:29:57.698 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.2596128279999999, 2024-09-17T21:29:57.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007701856, 2024-09-17T21:29:57.698 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.698 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.699 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.699 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.699 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.699 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.699 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16471 2024-09-17T21:29:57.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.699 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342592+0000", 2024-09-17T21:29:57.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342592+0000", 2024-09-17T21:29:57.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342592+0000", 2024-09-17T21:29:57.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:29:57.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342594+0000", 2024-09-17T21:29:57.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.906e-06 2024-09-17T21:29:57.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342600+0000", 2024-09-17T21:29:57.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8039999999999998e-06 2024-09-17T21:29:57.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342605+0000", 2024-09-17T21:29:57.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4649999999999996e-06 2024-09-17T21:29:57.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350178+0000", 2024-09-17T21:29:57.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075734920000000002 2024-09-17T21:29:57.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350240+0000", 2024-09-17T21:29:57.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2352000000000002e-05 2024-09-17T21:29:57.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350294+0000", 2024-09-17T21:29:57.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3431999999999999e-05 2024-09-17T21:29:57.703 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.703 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.703 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.703 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:16472 3.a 3:50b430a8:::rbd_data.11202c1b6cb4.0000000000000011:4 [read 665600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:57.703 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.342611+0000", 2024-09-17T21:29:57.704 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.2595943290000005, 2024-09-17T21:29:57.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0078017429999999999, 2024-09-17T21:29:57.704 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.704 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.704 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:57.704 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.704 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.704 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.704 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16472 2024-09-17T21:29:57.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.704 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342611+0000", 2024-09-17T21:29:57.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342611+0000", 2024-09-17T21:29:57.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342611+0000", 2024-09-17T21:29:57.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:29:57.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342614+0000", 2024-09-17T21:29:57.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9510000000000002e-06 2024-09-17T21:29:57.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342620+0000", 2024-09-17T21:29:57.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1e-06 2024-09-17T21:29:57.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.342631+0000", 2024-09-17T21:29:57.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1467999999999999e-05 2024-09-17T21:29:57.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350320+0000", 2024-09-17T21:29:57.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007688388 2024-09-17T21:29:57.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350360+0000", 2024-09-17T21:29:57.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9987999999999997e-05 2024-09-17T21:29:57.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.350412+0000", 2024-09-17T21:29:57.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2437e-05 2024-09-17T21:29:57.708 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.708 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.708 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.709 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:36935 3.a 3:54c8759b:::rbd_data.1120e62eea89.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-17T21:29:57.709 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:29:57.709 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.64329568599999998, 2024-09-17T21:29:57.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020540151999999999, 2024-09-17T21:29:57.709 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:57.709 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:57.709 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:57.709 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:57.709 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:57.709 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:57.710 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 36935 2024-09-17T21:29:57.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.710 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:57.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:57.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:29:57.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:57.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:29:57.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:57.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:57.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958910+0000", 2024-09-17T21:29:57.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:29:57.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:57.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958918+0000", 2024-09-17T21:29:57.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6469999999999998e-06 2024-09-17T21:29:57.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:57.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958933+0000", 2024-09-17T21:29:57.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4791e-05 2024-09-17T21:29:57.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:57.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958940+0000", 2024-09-17T21:29:57.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-17T21:29:57.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:57.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958990+0000", 2024-09-17T21:29:57.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9759000000000003e-05 2024-09-17T21:29:57.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:57.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.959039+0000", 2024-09-17T21:29:57.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9487000000000002e-05 2024-09-17T21:29:57.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:29:57.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.962098+0000", 2024-09-17T21:29:57.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003059258 2024-09-17T21:29:57.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:57.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979325+0000", 2024-09-17T21:29:57.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017226841 2024-09-17T21:29:57.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:57.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979343+0000", 2024-09-17T21:29:57.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7486000000000002e-05 2024-09-17T21:29:57.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:57.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979401+0000", 2024-09-17T21:29:57.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7980999999999999e-05 2024-09-17T21:29:57.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:57.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:57.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:57.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979449+0000", 2024-09-17T21:29:57.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8639e-05 2024-09-17T21:29:57.715 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.715 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.716 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.716 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:57.716 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:57.716 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:57.718 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:29:57.718 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:57.715+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:57.818 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:57.816+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:57.863 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:57.863 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:29:57.863 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:29:57.864 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:57.872 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:29:57.889 INFO:tasks.ceph.mon.a.smithi129.stderr:2024-09-17T21:29:57.887+0000 7f70111af640 -1 mon.a@0(leader).osd e36 definitely_dead 0 2024-09-17T21:29:57.956 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:57.954+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:58.016 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:58.016 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:29:58.016 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:29:58.016 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:29:58.016 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:58.019 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:58.017+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:58.024 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:29:58.120 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:58.118+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:58.169 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:29:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:29:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:29:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:29:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:29:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:29:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 30.226961515999999, 2024-09-17T21:29:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61572000699999996, 2024-09-17T21:29:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:29:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:29:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T21:29:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:29:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:29:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939745+0000", 2024-09-17T21:29:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6899999999999999e-06 2024-09-17T21:29:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939753+0000", 2024-09-17T21:29:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1950000000000003e-06 2024-09-17T21:29:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939767+0000", 2024-09-17T21:29:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3429e-05 2024-09-17T21:29:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939776+0000", 2024-09-17T21:29:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:29:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939858+0000", 2024-09-17T21:29:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2496000000000004e-05 2024-09-17T21:29:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T21:29:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939868+0000", 2024-09-17T21:29:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1910000000000006e-06 2024-09-17T21:29:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940612+0000", 2024-09-17T21:29:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074429599999999998 2024-09-17T21:29:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:29:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940621+0000", 2024-09-17T21:29:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9139999999999997e-06 2024-09-17T21:29:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554006+0000", 2024-09-17T21:29:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61338505099999996 2024-09-17T21:29:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554166+0000", 2024-09-17T21:29:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015982600000000001 2024-09-17T21:29:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:29:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554296+0000", 2024-09-17T21:29:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013002399999999999 2024-09-17T21:29:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554936+0000", 2024-09-17T21:29:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064009400000000001 2024-09-17T21:29:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555389+0000", 2024-09-17T21:29:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045355200000000002 2024-09-17T21:29:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555432+0000", 2024-09-17T21:29:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2477e-05 2024-09-17T21:29:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555463+0000", 2024-09-17T21:29:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1612000000000002e-05 2024-09-17T21:29:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.178 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17527 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1005568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.178 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:29:58.178 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7617336849999994, 2024-09-17T21:29:58.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046452090000000001, 2024-09-17T21:29:58.178 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.178 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.178 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:58.178 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.178 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.178 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.178 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17527 2024-09-17T21:29:58.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.179 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:29:58.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:29:58.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405496+0000", 2024-09-17T21:29:58.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052498400000000004 2024-09-17T21:29:58.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405500+0000", 2024-09-17T21:29:58.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8410000000000002e-06 2024-09-17T21:29:58.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405511+0000", 2024-09-17T21:29:58.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0749e-05 2024-09-17T21:29:58.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405517+0000", 2024-09-17T21:29:58.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:29:58.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451275+0000", 2024-09-17T21:29:58.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045758134999999998 2024-09-17T21:29:58.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451355+0000", 2024-09-17T21:29:58.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9453e-05 2024-09-17T21:29:58.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451423+0000", 2024-09-17T21:29:58.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8782999999999995e-05 2024-09-17T21:29:58.182 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.182 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.182 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.182 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17530 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1046528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.182 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:29:58.182 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7611260340000001, 2024-09-17T21:29:58.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046376737000000001, 2024-09-17T21:29:58.182 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.183 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.183 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:58.183 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.183 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.183 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.183 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17530 2024-09-17T21:29:58.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.183 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:29:58.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:29:58.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:29:58.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3300000000000002e-07 2024-09-17T21:29:58.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405581+0000", 2024-09-17T21:29:58.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.074e-06 2024-09-17T21:29:58.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405587+0000", 2024-09-17T21:29:58.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7559999999999996e-06 2024-09-17T21:29:58.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405592+0000", 2024-09-17T21:29:58.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8860000000000003e-06 2024-09-17T21:29:58.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451813+0000", 2024-09-17T21:29:58.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046221119999999997 2024-09-17T21:29:58.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451891+0000", 2024-09-17T21:29:58.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7449999999999999e-05 2024-09-17T21:29:58.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451956+0000", 2024-09-17T21:29:58.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4918000000000004e-05 2024-09-17T21:29:58.186 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.186 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.186 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.187 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17615 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.187 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:29:58.187 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7564931969999993, 2024-09-17T21:29:58.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046741309000000002, 2024-09-17T21:29:58.187 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.187 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.187 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:58.187 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.187 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.187 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.187 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17615 2024-09-17T21:29:58.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.187 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:29:58.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:29:58.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:29:58.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:29:58.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410213+0000", 2024-09-17T21:29:58.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:29:58.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410217+0000", 2024-09-17T21:29:58.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8199999999999998e-06 2024-09-17T21:29:58.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410220+0000", 2024-09-17T21:29:58.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6900000000000001e-06 2024-09-17T21:29:58.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455264+0000", 2024-09-17T21:29:58.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045044002999999999 2024-09-17T21:29:58.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455338+0000", 2024-09-17T21:29:58.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4182000000000002e-05 2024-09-17T21:29:58.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.456953+0000", 2024-09-17T21:29:58.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016150089999999999 2024-09-17T21:29:58.191 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.191 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.191 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.191 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17616 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 872448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.191 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:29:58.191 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7564811450000004, 2024-09-17T21:29:58.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946039000000002, 2024-09-17T21:29:58.192 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.192 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.192 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:58.192 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.192 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.192 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.192 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17616 2024-09-17T21:29:58.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.192 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:29:58.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:29:58.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:29:58.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T21:29:58.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410225+0000", 2024-09-17T21:29:58.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3459999999999999e-06 2024-09-17T21:29:58.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410228+0000", 2024-09-17T21:29:58.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.083e-06 2024-09-17T21:29:58.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410232+0000", 2024-09-17T21:29:58.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4419999999999998e-06 2024-09-17T21:29:58.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457004+0000", 2024-09-17T21:29:58.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046771873999999998 2024-09-17T21:29:58.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457084+0000", 2024-09-17T21:29:58.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0506000000000005e-05 2024-09-17T21:29:58.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457170+0000", 2024-09-17T21:29:58.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5494000000000005e-05 2024-09-17T21:29:58.195 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.195 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.196 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.196 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17617 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.196 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:29:58.196 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7564697430000002, 2024-09-17T21:29:58.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047084168000000003, 2024-09-17T21:29:58.196 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.196 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.196 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:58.196 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.196 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.196 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.196 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17617 2024-09-17T21:29:58.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.197 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:29:58.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:29:58.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:29:58.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-17T21:29:58.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410237+0000", 2024-09-17T21:29:58.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1960000000000001e-06 2024-09-17T21:29:58.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410246+0000", 2024-09-17T21:29:58.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7880000000000001e-06 2024-09-17T21:29:58.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410250+0000", 2024-09-17T21:29:58.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.518e-06 2024-09-17T21:29:58.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457209+0000", 2024-09-17T21:29:58.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046959283999999997 2024-09-17T21:29:58.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457264+0000", 2024-09-17T21:29:58.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673999999999999e-05 2024-09-17T21:29:58.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457319+0000", 2024-09-17T21:29:58.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5497000000000001e-05 2024-09-17T21:29:58.200 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.200 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.200 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.200 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17618 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 899072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.201 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:29:58.201 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7564502990000008, 2024-09-17T21:29:58.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047204208999999997, 2024-09-17T21:29:58.201 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.201 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.201 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:58.201 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.201 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.201 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.201 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17618 2024-09-17T21:29:58.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.201 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:29:58.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:29:58.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:29:58.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8799999999999998e-07 2024-09-17T21:29:58.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410256+0000", 2024-09-17T21:29:58.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T21:29:58.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410260+0000", 2024-09-17T21:29:58.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6500000000000002e-06 2024-09-17T21:29:58.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410263+0000", 2024-09-17T21:29:58.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7089999999999998e-06 2024-09-17T21:29:58.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457355+0000", 2024-09-17T21:29:58.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047092756999999999 2024-09-17T21:29:58.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457407+0000", 2024-09-17T21:29:58.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1808999999999998e-05 2024-09-17T21:29:58.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457459+0000", 2024-09-17T21:29:58.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1684000000000002e-05 2024-09-17T21:29:58.205 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.205 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.205 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.205 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17619 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 913408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.205 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:29:58.205 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.756439061, 2024-09-17T21:29:58.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047293443999999997, 2024-09-17T21:29:58.205 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.205 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.206 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:58.206 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.206 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.206 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.206 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17619 2024-09-17T21:29:58.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.206 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:29:58.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:29:58.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:29:58.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:29:58.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410267+0000", 2024-09-17T21:29:58.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:29:58.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410270+0000", 2024-09-17T21:29:58.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.01e-06 2024-09-17T21:29:58.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410273+0000", 2024-09-17T21:29:58.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6410000000000002e-06 2024-09-17T21:29:58.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457473+0000", 2024-09-17T21:29:58.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047199829999999998 2024-09-17T21:29:58.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457517+0000", 2024-09-17T21:29:58.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4058999999999999e-05 2024-09-17T21:29:58.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457559+0000", 2024-09-17T21:29:58.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2404999999999998e-05 2024-09-17T21:29:58.209 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.210 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.210 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.210 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17620 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 927744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.210 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:29:58.210 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7564278889999994, 2024-09-17T21:29:58.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047402409, 2024-09-17T21:29:58.210 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.210 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.210 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:58.210 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.210 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.211 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.211 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17620 2024-09-17T21:29:58.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.211 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:29:58.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:29:58.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:29:58.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:29:58.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410278+0000", 2024-09-17T21:29:58.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1790000000000001e-06 2024-09-17T21:29:58.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410281+0000", 2024-09-17T21:29:58.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.061e-06 2024-09-17T21:29:58.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410284+0000", 2024-09-17T21:29:58.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.65e-06 2024-09-17T21:29:58.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457587+0000", 2024-09-17T21:29:58.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047303206 2024-09-17T21:29:58.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457633+0000", 2024-09-17T21:29:58.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6063999999999998e-05 2024-09-17T21:29:58.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457679+0000", 2024-09-17T21:29:58.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6007000000000002e-05 2024-09-17T21:29:58.214 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.214 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.214 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.215 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17621 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 940032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.215 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:29:58.215 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7564174729999991, 2024-09-17T21:29:58.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047486550000000002, 2024-09-17T21:29:58.215 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.215 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.215 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:58.215 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.215 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.215 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.215 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17621 2024-09-17T21:29:58.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.215 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:29:58.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:29:58.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410288+0000", 2024-09-17T21:29:58.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:29:58.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410289+0000", 2024-09-17T21:29:58.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1179999999999999e-06 2024-09-17T21:29:58.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410292+0000", 2024-09-17T21:29:58.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:29:58.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410294+0000", 2024-09-17T21:29:58.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T21:29:58.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457694+0000", 2024-09-17T21:29:58.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047399416999999999 2024-09-17T21:29:58.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457734+0000", 2024-09-17T21:29:58.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0528999999999998e-05 2024-09-17T21:29:58.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.219 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457774+0000", 2024-09-17T21:29:58.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9529e-05 2024-09-17T21:29:58.219 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.219 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.219 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.219 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17622 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.219 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:29:58.219 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7564072910000004, 2024-09-17T21:29:58.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047596619, 2024-09-17T21:29:58.220 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.220 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.220 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:58.220 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.220 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.220 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.220 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17622 2024-09-17T21:29:58.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.220 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:29:58.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:29:58.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:29:58.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:29:58.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410303+0000", 2024-09-17T21:29:58.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2979999999999999e-06 2024-09-17T21:29:58.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410307+0000", 2024-09-17T21:29:58.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:29:58.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410310+0000", 2024-09-17T21:29:58.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.146e-06 2024-09-17T21:29:58.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457801+0000", 2024-09-17T21:29:58.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047491322000000002 2024-09-17T21:29:58.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457850+0000", 2024-09-17T21:29:58.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8585000000000003e-05 2024-09-17T21:29:58.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.224 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457894+0000", 2024-09-17T21:29:58.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4455e-05 2024-09-17T21:29:58.224 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.224 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.224 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.224 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17623 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 968704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.224 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:29:58.224 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7563909070000001, 2024-09-17T21:29:58.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047689377999999998, 2024-09-17T21:29:58.224 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.224 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.225 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:58.225 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.225 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.225 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.225 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17623 2024-09-17T21:29:58.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.225 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:29:58.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:29:58.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:29:58.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:29:58.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410315+0000", 2024-09-17T21:29:58.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0499999999999999e-06 2024-09-17T21:29:58.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410319+0000", 2024-09-17T21:29:58.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.517e-06 2024-09-17T21:29:58.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410321+0000", 2024-09-17T21:29:58.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:29:58.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457922+0000", 2024-09-17T21:29:58.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047600638000000001 2024-09-17T21:29:58.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457964+0000", 2024-09-17T21:29:58.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2126000000000003e-05 2024-09-17T21:29:58.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458003+0000", 2024-09-17T21:29:58.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9138999999999999e-05 2024-09-17T21:29:58.228 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.228 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.229 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.229 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17624 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.229 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:29:58.229 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7563803230000001, 2024-09-17T21:29:58.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047798292999999999, 2024-09-17T21:29:58.229 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.229 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.229 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:58.229 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.229 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.229 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.230 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17624 2024-09-17T21:29:58.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.230 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:29:58.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:29:58.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:29:58.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:29:58.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410326+0000", 2024-09-17T21:29:58.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0610000000000001e-06 2024-09-17T21:29:58.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410329+0000", 2024-09-17T21:29:58.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0319999999999999e-06 2024-09-17T21:29:58.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410331+0000", 2024-09-17T21:29:58.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5749999999999999e-06 2024-09-17T21:29:58.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458033+0000", 2024-09-17T21:29:58.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047701371999999999 2024-09-17T21:29:58.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458079+0000", 2024-09-17T21:29:58.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:29:58.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458123+0000", 2024-09-17T21:29:58.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850999999999997e-05 2024-09-17T21:29:58.233 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.233 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.233 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.234 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17625 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.234 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:29:58.234 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7563701629999997, 2024-09-17T21:29:58.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047883051000000003, 2024-09-17T21:29:58.234 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.234 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.234 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:58.234 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.234 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.234 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.234 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17625 2024-09-17T21:29:58.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.234 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:29:58.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:29:58.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:29:58.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:29:58.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410336+0000", 2024-09-17T21:29:58.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1310000000000001e-06 2024-09-17T21:29:58.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410339+0000", 2024-09-17T21:29:58.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000001e-06 2024-09-17T21:29:58.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410342+0000", 2024-09-17T21:29:58.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7460000000000001e-06 2024-09-17T21:29:58.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458137+0000", 2024-09-17T21:29:58.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047794931999999998 2024-09-17T21:29:58.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458178+0000", 2024-09-17T21:29:58.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1013999999999997e-05 2024-09-17T21:29:58.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458218+0000", 2024-09-17T21:29:58.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9675999999999998e-05 2024-09-17T21:29:58.238 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.238 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.238 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.239 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17626 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1009664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.239 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:29:58.239 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7563595420000002, 2024-09-17T21:29:58.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047991601000000002, 2024-09-17T21:29:58.239 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.239 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.239 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:58.239 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.239 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.239 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.239 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17626 2024-09-17T21:29:58.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.239 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:29:58.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:29:58.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410346+0000", 2024-09-17T21:29:58.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:29:58.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410347+0000", 2024-09-17T21:29:58.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.114e-06 2024-09-17T21:29:58.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410350+0000", 2024-09-17T21:29:58.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.101e-06 2024-09-17T21:29:58.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410353+0000", 2024-09-17T21:29:58.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T21:29:58.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458245+0000", 2024-09-17T21:29:58.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0478921 2024-09-17T21:29:58.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458292+0000", 2024-09-17T21:29:58.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397999999999999e-05 2024-09-17T21:29:58.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458337+0000", 2024-09-17T21:29:58.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5221e-05 2024-09-17T21:29:58.243 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.243 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.243 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.243 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17627 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.244 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:29:58.244 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.756322441, 2024-09-17T21:29:58.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048047383999999999, 2024-09-17T21:29:58.244 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.244 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.244 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:58.244 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.244 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.244 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.244 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17627 2024-09-17T21:29:58.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.244 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:29:58.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:29:58.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410383+0000", 2024-09-17T21:29:58.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:29:58.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410385+0000", 2024-09-17T21:29:58.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:29:58.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410391+0000", 2024-09-17T21:29:58.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:29:58.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410395+0000", 2024-09-17T21:29:58.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6539999999999998e-06 2024-09-17T21:29:58.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458350+0000", 2024-09-17T21:29:58.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047954855999999997 2024-09-17T21:29:58.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458392+0000", 2024-09-17T21:29:58.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2094000000000001e-05 2024-09-17T21:29:58.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458430+0000", 2024-09-17T21:29:58.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7474000000000001e-05 2024-09-17T21:29:58.248 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.248 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.248 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.248 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17628 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.248 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:29:58.248 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7563028799999998, 2024-09-17T21:29:58.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048142676000000002, 2024-09-17T21:29:58.249 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.249 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.249 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:29:58.249 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.249 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.249 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.249 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17628 2024-09-17T21:29:58.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.252 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:29:58.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:29:58.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:29:58.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:29:58.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410404+0000", 2024-09-17T21:29:58.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-06 2024-09-17T21:29:58.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410411+0000", 2024-09-17T21:29:58.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6850000000000004e-06 2024-09-17T21:29:58.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410415+0000", 2024-09-17T21:29:58.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6750000000000001e-06 2024-09-17T21:29:58.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458457+0000", 2024-09-17T21:29:58.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048042045999999998 2024-09-17T21:29:58.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458501+0000", 2024-09-17T21:29:58.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3661000000000001e-05 2024-09-17T21:29:58.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458545+0000", 2024-09-17T21:29:58.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3529000000000003e-05 2024-09-17T21:29:58.255 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.255 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.255 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.255 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17687 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.255 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:29:58.255 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7538579270000003, 2024-09-17T21:29:58.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050670448, 2024-09-17T21:29:58.255 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.255 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.256 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:58.256 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.256 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.256 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.256 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17687 2024-09-17T21:29:58.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.256 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:29:58.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:29:58.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412848+0000", 2024-09-17T21:29:58.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9100000000000003e-07 2024-09-17T21:29:58.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413102+0000", 2024-09-17T21:29:58.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025415600000000001 2024-09-17T21:29:58.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413111+0000", 2024-09-17T21:29:58.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2010000000000005e-06 2024-09-17T21:29:58.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413116+0000", 2024-09-17T21:29:58.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:29:58.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458559+0000", 2024-09-17T21:29:58.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045442914000000001 2024-09-17T21:29:58.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458632+0000", 2024-09-17T21:29:58.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3207000000000006e-05 2024-09-17T21:29:58.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:29:58.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458785+0000", 2024-09-17T21:29:58.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015226700000000001 2024-09-17T21:29:58.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:58.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.461648+0000", 2024-09-17T21:29:58.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028633069999999998 2024-09-17T21:29:58.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:58.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463416+0000", 2024-09-17T21:29:58.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017680230000000001 2024-09-17T21:29:58.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:58.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463475+0000", 2024-09-17T21:29:58.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9333000000000003e-05 2024-09-17T21:29:58.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463517+0000", 2024-09-17T21:29:58.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2162000000000001e-05 2024-09-17T21:29:58.261 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.261 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.261 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.261 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17688 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [write 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.261 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:29:58.261 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7535790589999998, 2024-09-17T21:29:58.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050625098, 2024-09-17T21:29:58.261 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.261 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.261 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:58.262 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.262 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.262 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.262 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17688 2024-09-17T21:29:58.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.262 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:29:58.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:29:58.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:29:58.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:29:58.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413128+0000", 2024-09-17T21:29:58.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.813e-06 2024-09-17T21:29:58.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413132+0000", 2024-09-17T21:29:58.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:29:58.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413135+0000", 2024-09-17T21:29:58.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:29:58.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459756+0000", 2024-09-17T21:29:58.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046621293000000001 2024-09-17T21:29:58.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459816+0000", 2024-09-17T21:29:58.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9156000000000001e-05 2024-09-17T21:29:58.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:29:58.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459912+0000", 2024-09-17T21:29:58.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6302000000000001e-05 2024-09-17T21:29:58.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:58.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463231+0000", 2024-09-17T21:29:58.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033188290000000001 2024-09-17T21:29:58.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:58.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463701+0000", 2024-09-17T21:29:58.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047068199999999999 2024-09-17T21:29:58.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:58.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463728+0000", 2024-09-17T21:29:58.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6726000000000001e-05 2024-09-17T21:29:58.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463751+0000", 2024-09-17T21:29:58.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2824e-05 2024-09-17T21:29:58.267 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.267 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.267 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.267 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17691 3.d 3:b3a055f3:::rbd_data.11207744cc2c.0000000000000083:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.267 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:29:58.267 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7500341559999999, 2024-09-17T21:29:58.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049309370999999998, 2024-09-17T21:29:58.268 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:29:58.268 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:29:58.268 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:29:58.268 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:29:58.268 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:29:58.268 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.268 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17691 2024-09-17T21:29:58.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.268 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:29:58.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:29:58.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:29:58.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:29:58.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:29:58.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:29:58.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:29:58.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416672+0000", 2024-09-17T21:29:58.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.47e-07 2024-09-17T21:29:58.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:29:58.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416915+0000", 2024-09-17T21:29:58.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024344500000000001 2024-09-17T21:29:58.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:29:58.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416924+0000", 2024-09-17T21:29:58.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7739999999999993e-06 2024-09-17T21:29:58.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416929+0000", 2024-09-17T21:29:58.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6990000000000002e-06 2024-09-17T21:29:58.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:29:58.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460414+0000", 2024-09-17T21:29:58.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043485466 2024-09-17T21:29:58.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:29:58.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460540+0000", 2024-09-17T21:29:58.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012608899999999999 2024-09-17T21:29:58.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:29:58.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460746+0000", 2024-09-17T21:29:58.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000205577 2024-09-17T21:29:58.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:29:58.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463800+0000", 2024-09-17T21:29:58.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030542030000000001 2024-09-17T21:29:58.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:29:58.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465867+0000", 2024-09-17T21:29:58.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020665929999999998 2024-09-17T21:29:58.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:29:58.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465914+0000", 2024-09-17T21:29:58.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497000000000003e-05 2024-09-17T21:29:58.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:29:58.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:29:58.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:29:58.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465980+0000", 2024-09-17T21:29:58.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6081000000000006e-05 2024-09-17T21:29:58.273 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.273 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.273 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.273 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:29:58.273 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:29:58.273 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:29:58.275 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:29:58.275 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:58.272+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:58.376 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:58.373+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:29:58.380 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:58.380 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:58.381 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:29:58.381 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:58.389 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:29:58.477 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:29:58.475+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:29:58.532 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:58.532 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:58.532 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:29:58.532 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:29:58.533 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:58.533 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is unset 2024-09-17T21:29:58.539 INFO:teuthology.orchestra.run.smithi129.stderr:marked out osd.5. 2024-09-17T21:29:58.548 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:29:58.578 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:58.576+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:58.654 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:58.654 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:29:58.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:29:58.654 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:29:58.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.654 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5316 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 905216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.654 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.593733+0000", 2024-09-17T21:29:58.654 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.056885275999999, 2024-09-17T21:29:58.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044533852999999998, 2024-09-17T21:29:58.654 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.654 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.655 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.655 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.655 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.655 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.655 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5316 2024-09-17T21:29:58.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.655 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593733+0000", 2024-09-17T21:29:58.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593733+0000", 2024-09-17T21:29:58.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593733+0000", 2024-09-17T21:29:58.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8799999999999998e-07 2024-09-17T21:29:58.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593734+0000", 2024-09-17T21:29:58.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.249e-06 2024-09-17T21:29:58.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593738+0000", 2024-09-17T21:29:58.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2210000000000002e-06 2024-09-17T21:29:58.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593740+0000", 2024-09-17T21:29:58.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4600000000000002e-06 2024-09-17T21:29:58.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638172+0000", 2024-09-17T21:29:58.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044431598000000003 2024-09-17T21:29:58.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638213+0000", 2024-09-17T21:29:58.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1068000000000001e-05 2024-09-17T21:29:58.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638267+0000", 2024-09-17T21:29:58.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3869000000000002e-05 2024-09-17T21:29:58.660 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.660 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.660 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.661 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5317 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 1017856~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.661 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.593757+0000", 2024-09-17T21:29:58.661 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.056860722, 2024-09-17T21:29:58.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044707716000000002, 2024-09-17T21:29:58.661 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.661 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.661 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.661 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5317 2024-09-17T21:29:58.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.662 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593757+0000", 2024-09-17T21:29:58.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593757+0000", 2024-09-17T21:29:58.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593758+0000", 2024-09-17T21:29:58.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.79e-07 2024-09-17T21:29:58.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593759+0000", 2024-09-17T21:29:58.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.451e-06 2024-09-17T21:29:58.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593762+0000", 2024-09-17T21:29:58.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4489999999999999e-06 2024-09-17T21:29:58.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593765+0000", 2024-09-17T21:29:58.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:29:58.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638302+0000", 2024-09-17T21:29:58.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044536394 2024-09-17T21:29:58.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638376+0000", 2024-09-17T21:29:58.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4016000000000004e-05 2024-09-17T21:29:58.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638465+0000", 2024-09-17T21:29:58.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9136999999999997e-05 2024-09-17T21:29:58.666 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.666 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.666 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.666 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5318 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 1079296~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.666 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.593770+0000", 2024-09-17T21:29:58.666 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.056848062, 2024-09-17T21:29:58.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044817632000000003, 2024-09-17T21:29:58.666 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.666 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.667 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.667 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.667 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.667 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.667 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5318 2024-09-17T21:29:58.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.667 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593770+0000", 2024-09-17T21:29:58.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593770+0000", 2024-09-17T21:29:58.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593770+0000", 2024-09-17T21:29:58.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.29e-07 2024-09-17T21:29:58.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593771+0000", 2024-09-17T21:29:58.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0160000000000001e-06 2024-09-17T21:29:58.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593774+0000", 2024-09-17T21:29:58.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9569999999999998e-06 2024-09-17T21:29:58.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593777+0000", 2024-09-17T21:29:58.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4210000000000002e-06 2024-09-17T21:29:58.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638494+0000", 2024-09-17T21:29:58.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044717645 2024-09-17T21:29:58.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638533+0000", 2024-09-17T21:29:58.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9051999999999998e-05 2024-09-17T21:29:58.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638588+0000", 2024-09-17T21:29:58.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4311999999999998e-05 2024-09-17T21:29:58.671 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.671 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.671 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.671 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5319 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 1130496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.671 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.593780+0000", 2024-09-17T21:29:58.671 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.056838129000001, 2024-09-17T21:29:58.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045002081999999999, 2024-09-17T21:29:58.672 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.672 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.672 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.672 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.672 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.672 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.672 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5319 2024-09-17T21:29:58.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.672 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593780+0000", 2024-09-17T21:29:58.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593780+0000", 2024-09-17T21:29:58.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593780+0000", 2024-09-17T21:29:58.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2599999999999998e-07 2024-09-17T21:29:58.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593781+0000", 2024-09-17T21:29:58.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7599999999999996e-07 2024-09-17T21:29:58.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593784+0000", 2024-09-17T21:29:58.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9229999999999998e-06 2024-09-17T21:29:58.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593790+0000", 2024-09-17T21:29:58.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1840000000000004e-06 2024-09-17T21:29:58.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638622+0000", 2024-09-17T21:29:58.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044831862 2024-09-17T21:29:58.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638695+0000", 2024-09-17T21:29:58.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3032000000000001e-05 2024-09-17T21:29:58.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638782+0000", 2024-09-17T21:29:58.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6879000000000006e-05 2024-09-17T21:29:58.676 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.676 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.676 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.677 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5320 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 1191936~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.677 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.593794+0000", 2024-09-17T21:29:58.677 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.056824412999999, 2024-09-17T21:29:58.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045110798000000001, 2024-09-17T21:29:58.677 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.677 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.677 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.677 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5320 2024-09-17T21:29:58.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.677 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593794+0000", 2024-09-17T21:29:58.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593794+0000", 2024-09-17T21:29:58.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593794+0000", 2024-09-17T21:29:58.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4999999999999999e-07 2024-09-17T21:29:58.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593795+0000", 2024-09-17T21:29:58.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0550000000000001e-06 2024-09-17T21:29:58.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593798+0000", 2024-09-17T21:29:58.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0790000000000001e-06 2024-09-17T21:29:58.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593801+0000", 2024-09-17T21:29:58.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9409999999999999e-06 2024-09-17T21:29:58.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638810+0000", 2024-09-17T21:29:58.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045009277 2024-09-17T21:29:58.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638849+0000", 2024-09-17T21:29:58.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8980000000000003e-05 2024-09-17T21:29:58.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638904+0000", 2024-09-17T21:29:58.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5216000000000001e-05 2024-09-17T21:29:58.681 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.682 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.682 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.682 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5321 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 1304576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.682 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.593804+0000", 2024-09-17T21:29:58.682 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.056813868000001, 2024-09-17T21:29:58.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045293438999999998, 2024-09-17T21:29:58.682 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.682 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.682 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.682 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.683 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5321 2024-09-17T21:29:58.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.683 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593804+0000", 2024-09-17T21:29:58.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593804+0000", 2024-09-17T21:29:58.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593804+0000", 2024-09-17T21:29:58.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.36e-07 2024-09-17T21:29:58.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593805+0000", 2024-09-17T21:29:58.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9300000000000006e-07 2024-09-17T21:29:58.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593808+0000", 2024-09-17T21:29:58.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9610000000000001e-06 2024-09-17T21:29:58.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593811+0000", 2024-09-17T21:29:58.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.407e-06 2024-09-17T21:29:58.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638939+0000", 2024-09-17T21:29:58.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045128387999999998 2024-09-17T21:29:58.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.639010+0000", 2024-09-17T21:29:58.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0419999999999993e-05 2024-09-17T21:29:58.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.639098+0000", 2024-09-17T21:29:58.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8034000000000005e-05 2024-09-17T21:29:58.687 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.687 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.687 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.688 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5322 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 55296~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.688 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.593814+0000", 2024-09-17T21:29:58.688 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.056804103999999, 2024-09-17T21:29:58.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045414585, 2024-09-17T21:29:58.688 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.688 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.688 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.688 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.688 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.689 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.689 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5322 2024-09-17T21:29:58.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.689 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593814+0000", 2024-09-17T21:29:58.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593814+0000", 2024-09-17T21:29:58.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593814+0000", 2024-09-17T21:29:58.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2000000000000001e-07 2024-09-17T21:29:58.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593815+0000", 2024-09-17T21:29:58.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8199999999999998e-07 2024-09-17T21:29:58.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593818+0000", 2024-09-17T21:29:58.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9299999999999999e-06 2024-09-17T21:29:58.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593820+0000", 2024-09-17T21:29:58.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4320000000000002e-06 2024-09-17T21:29:58.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.639126+0000", 2024-09-17T21:29:58.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045305426000000003 2024-09-17T21:29:58.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.639173+0000", 2024-09-17T21:29:58.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7657999999999997e-05 2024-09-17T21:29:58.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.639229+0000", 2024-09-17T21:29:58.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5037000000000002e-05 2024-09-17T21:29:58.693 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.693 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.694 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.694 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5323 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 106496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.694 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.593823+0000", 2024-09-17T21:29:58.694 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.056794553, 2024-09-17T21:29:58.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045605916000000003, 2024-09-17T21:29:58.694 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.694 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.695 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.695 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.695 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.695 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.695 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5323 2024-09-17T21:29:58.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.695 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593823+0000", 2024-09-17T21:29:58.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593823+0000", 2024-09-17T21:29:58.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599270+0000", 2024-09-17T21:29:58.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0054463580000000001 2024-09-17T21:29:58.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599276+0000", 2024-09-17T21:29:58.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3620000000000002e-06 2024-09-17T21:29:58.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599287+0000", 2024-09-17T21:29:58.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0721e-05 2024-09-17T21:29:58.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599296+0000", 2024-09-17T21:29:58.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.969e-06 2024-09-17T21:29:58.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.639264+0000", 2024-09-17T21:29:58.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039967920999999997 2024-09-17T21:29:58.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.639337+0000", 2024-09-17T21:29:58.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3510999999999995e-05 2024-09-17T21:29:58.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.639429+0000", 2024-09-17T21:29:58.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2074e-05 2024-09-17T21:29:58.700 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.700 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.700 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.701 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5347 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 518144~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.701 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599747+0000", 2024-09-17T21:29:58.701 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.050871342000001, 2024-09-17T21:29:58.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044652727000000003, 2024-09-17T21:29:58.701 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.701 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.701 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.701 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.701 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.701 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.701 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5347 2024-09-17T21:29:58.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.701 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599747+0000", 2024-09-17T21:29:58.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599747+0000", 2024-09-17T21:29:58.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599747+0000", 2024-09-17T21:29:58.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:29:58.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599748+0000", 2024-09-17T21:29:58.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8400000000000002e-07 2024-09-17T21:29:58.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599752+0000", 2024-09-17T21:29:58.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.754e-06 2024-09-17T21:29:58.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599754+0000", 2024-09-17T21:29:58.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.34e-06 2024-09-17T21:29:58.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.644238+0000", 2024-09-17T21:29:58.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044484403999999998 2024-09-17T21:29:58.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.644310+0000", 2024-09-17T21:29:58.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1423999999999999e-05 2024-09-17T21:29:58.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.644399+0000", 2024-09-17T21:29:58.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9602e-05 2024-09-17T21:29:58.706 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.706 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.706 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.706 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5348 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 579584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.706 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599757+0000", 2024-09-17T21:29:58.706 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.050860548999999, 2024-09-17T21:29:58.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045142726000000001, 2024-09-17T21:29:58.706 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.706 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.706 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.706 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.706 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.707 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.707 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5348 2024-09-17T21:29:58.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.707 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599757+0000", 2024-09-17T21:29:58.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599757+0000", 2024-09-17T21:29:58.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599758+0000", 2024-09-17T21:29:58.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.16e-07 2024-09-17T21:29:58.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599765+0000", 2024-09-17T21:29:58.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.063e-06 2024-09-17T21:29:58.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599768+0000", 2024-09-17T21:29:58.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.455e-06 2024-09-17T21:29:58.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599771+0000", 2024-09-17T21:29:58.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.565e-06 2024-09-17T21:29:58.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.644438+0000", 2024-09-17T21:29:58.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044667017000000003 2024-09-17T21:29:58.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.644532+0000", 2024-09-17T21:29:58.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4238000000000001e-05 2024-09-17T21:29:58.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.644900+0000", 2024-09-17T21:29:58.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000368172 2024-09-17T21:29:58.710 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.710 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.710 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.711 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5349 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 630784~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.711 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599774+0000", 2024-09-17T21:29:58.711 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.050843816, 2024-09-17T21:29:58.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045306261, 2024-09-17T21:29:58.711 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.711 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.711 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.711 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.711 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.711 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.711 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5349 2024-09-17T21:29:58.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.711 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599774+0000", 2024-09-17T21:29:58.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599774+0000", 2024-09-17T21:29:58.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599774+0000", 2024-09-17T21:29:58.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:29:58.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599775+0000", 2024-09-17T21:29:58.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0240000000000001e-06 2024-09-17T21:29:58.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599778+0000", 2024-09-17T21:29:58.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9469999999999999e-06 2024-09-17T21:29:58.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599781+0000", 2024-09-17T21:29:58.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4080000000000001e-06 2024-09-17T21:29:58.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.644922+0000", 2024-09-17T21:29:58.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045141463 2024-09-17T21:29:58.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.644993+0000", 2024-09-17T21:29:58.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1109e-05 2024-09-17T21:29:58.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645080+0000", 2024-09-17T21:29:58.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7089000000000006e-05 2024-09-17T21:29:58.715 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.715 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.715 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.715 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5350 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 692224~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.715 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599784+0000", 2024-09-17T21:29:58.715 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.050834046, 2024-09-17T21:29:58.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045533090999999998, 2024-09-17T21:29:58.715 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.715 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.715 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.716 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.716 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.716 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.716 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5350 2024-09-17T21:29:58.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.716 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599784+0000", 2024-09-17T21:29:58.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599784+0000", 2024-09-17T21:29:58.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599784+0000", 2024-09-17T21:29:58.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:29:58.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599785+0000", 2024-09-17T21:29:58.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2699999999999998e-07 2024-09-17T21:29:58.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599789+0000", 2024-09-17T21:29:58.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7189999999999999e-06 2024-09-17T21:29:58.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599796+0000", 2024-09-17T21:29:58.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3849999999999996e-06 2024-09-17T21:29:58.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645122+0000", 2024-09-17T21:29:58.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045325987999999998 2024-09-17T21:29:58.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645210+0000", 2024-09-17T21:29:58.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7905000000000006e-05 2024-09-17T21:29:58.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645317+0000", 2024-09-17T21:29:58.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106953 2024-09-17T21:29:58.719 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.719 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.719 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.720 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5351 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 804864~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.720 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599802+0000", 2024-09-17T21:29:58.720 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.050815681, 2024-09-17T21:29:58.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.04571066, 2024-09-17T21:29:58.720 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.720 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.720 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.720 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.720 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.720 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.720 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5351 2024-09-17T21:29:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599802+0000", 2024-09-17T21:29:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599802+0000", 2024-09-17T21:29:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599803+0000", 2024-09-17T21:29:58.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:29:58.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599809+0000", 2024-09-17T21:29:58.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0599999999999996e-06 2024-09-17T21:29:58.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599815+0000", 2024-09-17T21:29:58.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6499999999999999e-06 2024-09-17T21:29:58.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599820+0000", 2024-09-17T21:29:58.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6630000000000001e-06 2024-09-17T21:29:58.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645354+0000", 2024-09-17T21:29:58.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045533853999999999 2024-09-17T21:29:58.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645424+0000", 2024-09-17T21:29:58.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0191000000000006e-05 2024-09-17T21:29:58.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645513+0000", 2024-09-17T21:29:58.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8863999999999995e-05 2024-09-17T21:29:58.724 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.724 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.724 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.724 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5352 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 866304~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.724 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599827+0000", 2024-09-17T21:29:58.724 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.050791354999999, 2024-09-17T21:29:58.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045890574000000003, 2024-09-17T21:29:58.725 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.725 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.725 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.725 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.725 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.725 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.725 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5352 2024-09-17T21:29:58.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.725 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599827+0000", 2024-09-17T21:29:58.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599827+0000", 2024-09-17T21:29:58.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599827+0000", 2024-09-17T21:29:58.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6100000000000002e-07 2024-09-17T21:29:58.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599829+0000", 2024-09-17T21:29:58.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.663e-06 2024-09-17T21:29:58.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599834+0000", 2024-09-17T21:29:58.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4269999999999999e-06 2024-09-17T21:29:58.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599838+0000", 2024-09-17T21:29:58.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1880000000000004e-06 2024-09-17T21:29:58.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645554+0000", 2024-09-17T21:29:58.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045715649999999997 2024-09-17T21:29:58.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645634+0000", 2024-09-17T21:29:58.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0276000000000002e-05 2024-09-17T21:29:58.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645717+0000", 2024-09-17T21:29:58.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3009000000000004e-05 2024-09-17T21:29:58.728 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.729 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.729 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.729 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5353 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 917504~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.729 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599857+0000", 2024-09-17T21:29:58.729 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.050760973999999, 2024-09-17T21:29:58.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046056769999999997, 2024-09-17T21:29:58.729 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.729 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.729 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.729 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.729 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.730 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.730 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5353 2024-09-17T21:29:58.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.730 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599857+0000", 2024-09-17T21:29:58.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599857+0000", 2024-09-17T21:29:58.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599858+0000", 2024-09-17T21:29:58.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2600000000000002e-07 2024-09-17T21:29:58.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599860+0000", 2024-09-17T21:29:58.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.131e-06 2024-09-17T21:29:58.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599866+0000", 2024-09-17T21:29:58.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1789999999999996e-06 2024-09-17T21:29:58.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599871+0000", 2024-09-17T21:29:58.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9969999999999998e-06 2024-09-17T21:29:58.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645750+0000", 2024-09-17T21:29:58.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045879511999999997 2024-09-17T21:29:58.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645824+0000", 2024-09-17T21:29:58.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3738000000000004e-05 2024-09-17T21:29:58.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645914+0000", 2024-09-17T21:29:58.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9686999999999997e-05 2024-09-17T21:29:58.733 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.733 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.733 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.733 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5355 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 978944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.734 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599878+0000", 2024-09-17T21:29:58.734 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.050740122000001, 2024-09-17T21:29:58.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046157286999999998, 2024-09-17T21:29:58.734 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.734 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.734 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.734 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.734 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.734 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.734 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5355 2024-09-17T21:29:58.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.734 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599878+0000", 2024-09-17T21:29:58.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599878+0000", 2024-09-17T21:29:58.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599878+0000", 2024-09-17T21:29:58.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.27e-07 2024-09-17T21:29:58.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599880+0000", 2024-09-17T21:29:58.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9180000000000001e-06 2024-09-17T21:29:58.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599886+0000", 2024-09-17T21:29:58.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0839999999999997e-06 2024-09-17T21:29:58.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599891+0000", 2024-09-17T21:29:58.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3290000000000004e-06 2024-09-17T21:29:58.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645927+0000", 2024-09-17T21:29:58.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046036430000000003 2024-09-17T21:29:58.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645982+0000", 2024-09-17T21:29:58.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4697999999999997e-05 2024-09-17T21:29:58.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646035+0000", 2024-09-17T21:29:58.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3400999999999999e-05 2024-09-17T21:29:58.738 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.738 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.738 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.738 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5357 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 1091584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.738 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599897+0000", 2024-09-17T21:29:58.738 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.050720558, 2024-09-17T21:29:58.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046338788999999998, 2024-09-17T21:29:58.738 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.738 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.739 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.739 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.739 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.739 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.739 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5357 2024-09-17T21:29:58.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.739 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599897+0000", 2024-09-17T21:29:58.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599897+0000", 2024-09-17T21:29:58.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599898+0000", 2024-09-17T21:29:58.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:29:58.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599903+0000", 2024-09-17T21:29:58.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:29:58.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599912+0000", 2024-09-17T21:29:58.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0049999999999993e-06 2024-09-17T21:29:58.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599924+0000", 2024-09-17T21:29:58.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1539000000000001e-05 2024-09-17T21:29:58.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646067+0000", 2024-09-17T21:29:58.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046143048999999998 2024-09-17T21:29:58.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646141+0000", 2024-09-17T21:29:58.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3955000000000006e-05 2024-09-17T21:29:58.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646236+0000", 2024-09-17T21:29:58.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5484999999999998e-05 2024-09-17T21:29:58.742 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.742 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.742 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.743 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5359 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 1153024~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.743 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599932+0000", 2024-09-17T21:29:58.743 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.050686464, 2024-09-17T21:29:58.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046420520999999999, 2024-09-17T21:29:58.743 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.743 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.743 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.743 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.743 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.743 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.743 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5359 2024-09-17T21:29:58.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.744 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599932+0000", 2024-09-17T21:29:58.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599932+0000", 2024-09-17T21:29:58.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599932+0000", 2024-09-17T21:29:58.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6600000000000002e-07 2024-09-17T21:29:58.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599934+0000", 2024-09-17T21:29:58.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3329999999999999e-06 2024-09-17T21:29:58.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599941+0000", 2024-09-17T21:29:58.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7399999999999998e-06 2024-09-17T21:29:58.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599946+0000", 2024-09-17T21:29:58.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8450000000000002e-06 2024-09-17T21:29:58.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646263+0000", 2024-09-17T21:29:58.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046316619000000003 2024-09-17T21:29:58.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646301+0000", 2024-09-17T21:29:58.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8379e-05 2024-09-17T21:29:58.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646352+0000", 2024-09-17T21:29:58.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1138999999999999e-05 2024-09-17T21:29:58.747 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.747 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.747 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.747 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5361 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 1204224~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.747 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599953+0000", 2024-09-17T21:29:58.748 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.050665159999999, 2024-09-17T21:29:58.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046597168000000001, 2024-09-17T21:29:58.748 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.748 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.748 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.748 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.748 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.748 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.748 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5361 2024-09-17T21:29:58.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.748 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599953+0000", 2024-09-17T21:29:58.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599953+0000", 2024-09-17T21:29:58.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599953+0000", 2024-09-17T21:29:58.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1399999999999997e-07 2024-09-17T21:29:58.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599955+0000", 2024-09-17T21:29:58.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0729999999999999e-06 2024-09-17T21:29:58.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599962+0000", 2024-09-17T21:29:58.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4690000000000002e-06 2024-09-17T21:29:58.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599970+0000", 2024-09-17T21:29:58.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8399999999999995e-06 2024-09-17T21:29:58.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646385+0000", 2024-09-17T21:29:58.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046415005000000002 2024-09-17T21:29:58.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646459+0000", 2024-09-17T21:29:58.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4214000000000005e-05 2024-09-17T21:29:58.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646550+0000", 2024-09-17T21:29:58.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1153e-05 2024-09-17T21:29:58.752 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.752 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.752 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.752 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5364 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 1265664~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:58.752 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599975+0000", 2024-09-17T21:29:58.752 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.050642848000001, 2024-09-17T21:29:58.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046685196999999998, 2024-09-17T21:29:58.752 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:58.752 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:58.752 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:58.753 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:58.753 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:58.753 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:58.753 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5364 2024-09-17T21:29:58.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.753 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:58.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:58.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599975+0000", 2024-09-17T21:29:58.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:58.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599975+0000", 2024-09-17T21:29:58.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:58.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:58.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.605220+0000", 2024-09-17T21:29:58.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0052443769999999997 2024-09-17T21:29:58.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:58.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.605226+0000", 2024-09-17T21:29:58.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3729999999999998e-06 2024-09-17T21:29:58.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:58.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.605239+0000", 2024-09-17T21:29:58.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2575000000000001e-05 2024-09-17T21:29:58.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:58.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.605246+0000", 2024-09-17T21:29:58.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7859999999999997e-06 2024-09-17T21:29:58.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:58.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646574+0000", 2024-09-17T21:29:58.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041328107000000003 2024-09-17T21:29:58.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:58.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646610+0000", 2024-09-17T21:29:58.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5512000000000002e-05 2024-09-17T21:29:58.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:58.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:58.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:58.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646660+0000", 2024-09-17T21:29:58.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0467000000000003e-05 2024-09-17T21:29:58.756 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.756 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.756 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.756 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:58.756 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:58.756 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:58.758 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:29:58.758 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:29:58.755+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:29:58.859 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:58.857+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:58.861 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:58.862 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:58.862 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:29:58.862 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:58.871 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:29:58.876 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:29:58.960 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:58.958+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:58.976 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:29:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:29:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:29:58.976 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:58.985 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:29:59.036 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:29:59.036 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:29:59.061 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:59.058+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:59.091 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:29:59.091 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:29:59.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:29:59.091 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:29:59.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.092 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5493 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.092 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571177+0000", 2024-09-17T21:29:59.092 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.517149587, 2024-09-17T21:29:59.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043037758000000002, 2024-09-17T21:29:59.092 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.092 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.092 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.092 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.092 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.092 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.092 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5493 2024-09-17T21:29:59.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.093 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571177+0000", 2024-09-17T21:29:59.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571177+0000", 2024-09-17T21:29:59.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571178+0000", 2024-09-17T21:29:59.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8800000000000003e-07 2024-09-17T21:29:59.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571180+0000", 2024-09-17T21:29:59.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.17e-06 2024-09-17T21:29:59.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571187+0000", 2024-09-17T21:29:59.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.345e-06 2024-09-17T21:29:59.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571192+0000", 2024-09-17T21:29:59.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8300000000000003e-06 2024-09-17T21:29:59.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.613913+0000", 2024-09-17T21:29:59.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042721071999999999 2024-09-17T21:29:59.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614000+0000", 2024-09-17T21:29:59.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7027000000000006e-05 2024-09-17T21:29:59.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614215+0000", 2024-09-17T21:29:59.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482599999999999 2024-09-17T21:29:59.096 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.096 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.096 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.096 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5495 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.097 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571198+0000", 2024-09-17T21:29:59.097 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.517128379000001, 2024-09-17T21:29:59.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043378574000000003, 2024-09-17T21:29:59.097 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.097 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.097 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.097 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.097 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.097 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.097 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5495 2024-09-17T21:29:59.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.097 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571198+0000", 2024-09-17T21:29:59.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571198+0000", 2024-09-17T21:29:59.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571199+0000", 2024-09-17T21:29:59.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:29:59.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571200+0000", 2024-09-17T21:29:59.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7120000000000001e-06 2024-09-17T21:29:59.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571207+0000", 2024-09-17T21:29:59.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7159999999999997e-06 2024-09-17T21:29:59.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571212+0000", 2024-09-17T21:29:59.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5410000000000002e-06 2024-09-17T21:29:59.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614255+0000", 2024-09-17T21:29:59.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043043658999999998 2024-09-17T21:29:59.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614330+0000", 2024-09-17T21:29:59.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4640000000000004e-05 2024-09-17T21:29:59.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614577+0000", 2024-09-17T21:29:59.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024695400000000002 2024-09-17T21:29:59.101 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.101 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.101 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.101 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5497 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 329728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.101 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571218+0000", 2024-09-17T21:29:59.101 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.517109017999999, 2024-09-17T21:29:59.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043663621999999999, 2024-09-17T21:29:59.102 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.102 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.102 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.102 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.102 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.102 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.102 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5497 2024-09-17T21:29:59.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.102 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571218+0000", 2024-09-17T21:29:59.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571218+0000", 2024-09-17T21:29:59.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571218+0000", 2024-09-17T21:29:59.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6199999999999999e-07 2024-09-17T21:29:59.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571220+0000", 2024-09-17T21:29:59.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.655e-06 2024-09-17T21:29:59.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571227+0000", 2024-09-17T21:29:59.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9829999999999999e-06 2024-09-17T21:29:59.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571231+0000", 2024-09-17T21:29:59.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5310000000000003e-06 2024-09-17T21:29:59.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614621+0000", 2024-09-17T21:29:59.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043389469 2024-09-17T21:29:59.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614699+0000", 2024-09-17T21:29:59.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8490000000000002e-05 2024-09-17T21:29:59.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614881+0000", 2024-09-17T21:29:59.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000182132 2024-09-17T21:29:59.106 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.106 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.106 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.106 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5499 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 342016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.106 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571237+0000", 2024-09-17T21:29:59.106 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.517089417999999, 2024-09-17T21:29:59.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043955813000000003, 2024-09-17T21:29:59.106 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.106 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.106 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.106 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.107 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.107 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.107 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5499 2024-09-17T21:29:59.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.107 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571237+0000", 2024-09-17T21:29:59.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571237+0000", 2024-09-17T21:29:59.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571238+0000", 2024-09-17T21:29:59.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:29:59.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571239+0000", 2024-09-17T21:29:59.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:29:59.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571246+0000", 2024-09-17T21:29:59.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8390000000000001e-06 2024-09-17T21:29:59.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571251+0000", 2024-09-17T21:29:59.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4349999999999999e-06 2024-09-17T21:29:59.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614924+0000", 2024-09-17T21:29:59.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043673099999999999 2024-09-17T21:29:59.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614983+0000", 2024-09-17T21:29:59.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8794e-05 2024-09-17T21:29:59.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615193+0000", 2024-09-17T21:29:59.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000210545 2024-09-17T21:29:59.110 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.110 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.110 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.111 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5501 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 356352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.111 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571257+0000", 2024-09-17T21:29:59.111 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.517070185, 2024-09-17T21:29:59.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044214865999999999, 2024-09-17T21:29:59.111 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.111 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.111 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.111 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.111 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.111 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.111 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5501 2024-09-17T21:29:59.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.111 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571257+0000", 2024-09-17T21:29:59.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571257+0000", 2024-09-17T21:29:59.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571257+0000", 2024-09-17T21:29:59.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:29:59.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571259+0000", 2024-09-17T21:29:59.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9460000000000001e-06 2024-09-17T21:29:59.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571266+0000", 2024-09-17T21:29:59.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0439999999999998e-06 2024-09-17T21:29:59.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571271+0000", 2024-09-17T21:29:59.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6500000000000004e-06 2024-09-17T21:29:59.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615238+0000", 2024-09-17T21:29:59.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043967570999999997 2024-09-17T21:29:59.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615293+0000", 2024-09-17T21:29:59.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4857e-05 2024-09-17T21:29:59.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615471+0000", 2024-09-17T21:29:59.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00017844900000000001 2024-09-17T21:29:59.115 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.115 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.115 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.115 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5503 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 370688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.115 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571276+0000", 2024-09-17T21:29:59.115 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.517050306, 2024-09-17T21:29:59.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044418055999999997, 2024-09-17T21:29:59.115 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.115 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.116 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.116 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.116 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.116 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.116 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5503 2024-09-17T21:29:59.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.116 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571276+0000", 2024-09-17T21:29:59.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571276+0000", 2024-09-17T21:29:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571277+0000", 2024-09-17T21:29:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.46e-07 2024-09-17T21:29:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571286+0000", 2024-09-17T21:29:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8810000000000002e-06 2024-09-17T21:29:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571293+0000", 2024-09-17T21:29:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5499999999999997e-06 2024-09-17T21:29:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571303+0000", 2024-09-17T21:29:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5279999999999992e-06 2024-09-17T21:29:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615514+0000", 2024-09-17T21:29:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044211617000000002 2024-09-17T21:29:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615593+0000", 2024-09-17T21:29:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8768000000000002e-05 2024-09-17T21:29:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615694+0000", 2024-09-17T21:29:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101366 2024-09-17T21:29:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5548 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 194560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571310+0000", 2024-09-17T21:29:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.517017217999999, 2024-09-17T21:29:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044804920999999998, 2024-09-17T21:29:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5548 2024-09-17T21:29:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571310+0000", 2024-09-17T21:29:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571310+0000", 2024-09-17T21:29:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571310+0000", 2024-09-17T21:29:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8799999999999998e-07 2024-09-17T21:29:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571312+0000", 2024-09-17T21:29:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8920000000000001e-06 2024-09-17T21:29:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571319+0000", 2024-09-17T21:29:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9360000000000002e-06 2024-09-17T21:29:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571323+0000", 2024-09-17T21:29:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6739999999999996e-06 2024-09-17T21:29:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615725+0000", 2024-09-17T21:29:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044401949000000003 2024-09-17T21:29:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615801+0000", 2024-09-17T21:29:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5150999999999999e-05 2024-09-17T21:29:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616114+0000", 2024-09-17T21:29:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00031393099999999999 2024-09-17T21:29:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5551 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 208896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571329+0000", 2024-09-17T21:29:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.516997297, 2024-09-17T21:29:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045083356999999998, 2024-09-17T21:29:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5551 2024-09-17T21:29:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571329+0000", 2024-09-17T21:29:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571329+0000", 2024-09-17T21:29:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571330+0000", 2024-09-17T21:29:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5499999999999999e-07 2024-09-17T21:29:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571331+0000", 2024-09-17T21:29:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.691e-06 2024-09-17T21:29:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571338+0000", 2024-09-17T21:29:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-17T21:29:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571343+0000", 2024-09-17T21:29:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6260000000000003e-06 2024-09-17T21:29:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616174+0000", 2024-09-17T21:29:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044831322 2024-09-17T21:29:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616224+0000", 2024-09-17T21:29:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9950000000000001e-05 2024-09-17T21:29:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616413+0000", 2024-09-17T21:29:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018850400000000001 2024-09-17T21:29:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5553 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571349+0000", 2024-09-17T21:29:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.516977767, 2024-09-17T21:29:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045359177000000001, 2024-09-17T21:29:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5553 2024-09-17T21:29:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571349+0000", 2024-09-17T21:29:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571349+0000", 2024-09-17T21:29:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571349+0000", 2024-09-17T21:29:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.15e-07 2024-09-17T21:29:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571351+0000", 2024-09-17T21:29:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6750000000000001e-06 2024-09-17T21:29:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571360+0000", 2024-09-17T21:29:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6270000000000006e-06 2024-09-17T21:29:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571364+0000", 2024-09-17T21:29:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7029999999999997e-06 2024-09-17T21:29:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616459+0000", 2024-09-17T21:29:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045094672000000002 2024-09-17T21:29:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616503+0000", 2024-09-17T21:29:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4032000000000001e-05 2024-09-17T21:29:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616708+0000", 2024-09-17T21:29:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020515299999999999 2024-09-17T21:29:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5555 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 235520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571371+0000", 2024-09-17T21:29:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.516956187, 2024-09-17T21:29:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045673013999999998, 2024-09-17T21:29:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5555 2024-09-17T21:29:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571371+0000", 2024-09-17T21:29:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571371+0000", 2024-09-17T21:29:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571371+0000", 2024-09-17T21:29:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9400000000000001e-07 2024-09-17T21:29:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571373+0000", 2024-09-17T21:29:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.714e-06 2024-09-17T21:29:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571379+0000", 2024-09-17T21:29:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0449999999999997e-06 2024-09-17T21:29:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571383+0000", 2024-09-17T21:29:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.963e-06 2024-09-17T21:29:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616752+0000", 2024-09-17T21:29:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045369481000000003 2024-09-17T21:29:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616828+0000", 2024-09-17T21:29:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5961e-05 2024-09-17T21:29:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617044+0000", 2024-09-17T21:29:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021545599999999999 2024-09-17T21:29:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5556 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571392+0000", 2024-09-17T21:29:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.516934328, 2024-09-17T21:29:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045971310000000001, 2024-09-17T21:29:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5556 2024-09-17T21:29:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571392+0000", 2024-09-17T21:29:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571392+0000", 2024-09-17T21:29:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571393+0000", 2024-09-17T21:29:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1399999999999997e-07 2024-09-17T21:29:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571395+0000", 2024-09-17T21:29:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.956e-06 2024-09-17T21:29:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571401+0000", 2024-09-17T21:29:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9089999999999997e-06 2024-09-17T21:29:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571405+0000", 2024-09-17T21:29:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6410000000000001e-06 2024-09-17T21:29:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617070+0000", 2024-09-17T21:29:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045664874000000001 2024-09-17T21:29:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617130+0000", 2024-09-17T21:29:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9756999999999997e-05 2024-09-17T21:29:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617364+0000", 2024-09-17T21:29:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000233759 2024-09-17T21:29:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5559 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 264192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571411+0000", 2024-09-17T21:29:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.516915319000001, 2024-09-17T21:29:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046332908999999999, 2024-09-17T21:29:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5559 2024-09-17T21:29:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571411+0000", 2024-09-17T21:29:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571411+0000", 2024-09-17T21:29:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571412+0000", 2024-09-17T21:29:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.65e-07 2024-09-17T21:29:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571419+0000", 2024-09-17T21:29:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2899999999999997e-06 2024-09-17T21:29:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571425+0000", 2024-09-17T21:29:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3400000000000003e-06 2024-09-17T21:29:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571430+0000", 2024-09-17T21:29:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4499999999999997e-06 2024-09-17T21:29:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617455+0000", 2024-09-17T21:29:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046024661000000001 2024-09-17T21:29:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617532+0000", 2024-09-17T21:29:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7712000000000004e-05 2024-09-17T21:29:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617744+0000", 2024-09-17T21:29:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000212091 2024-09-17T21:29:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5561 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 276480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571436+0000", 2024-09-17T21:29:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.51689053, 2024-09-17T21:29:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046615325999999999, 2024-09-17T21:29:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5561 2024-09-17T21:29:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571436+0000", 2024-09-17T21:29:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571436+0000", 2024-09-17T21:29:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571437+0000", 2024-09-17T21:29:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:29:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571439+0000", 2024-09-17T21:29:59.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9130000000000001e-06 2024-09-17T21:29:59.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571444+0000", 2024-09-17T21:29:59.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5110000000000003e-06 2024-09-17T21:29:59.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571448+0000", 2024-09-17T21:29:59.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3749999999999996e-06 2024-09-17T21:29:59.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617775+0000", 2024-09-17T21:29:59.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046326668000000001 2024-09-17T21:29:59.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617851+0000", 2024-09-17T21:29:59.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6192000000000005e-05 2024-09-17T21:29:59.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618052+0000", 2024-09-17T21:29:59.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020022900000000001 2024-09-17T21:29:59.151 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5563 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 290816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571454+0000", 2024-09-17T21:29:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.516872253000001, 2024-09-17T21:29:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046975959999999997, 2024-09-17T21:29:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5563 2024-09-17T21:29:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571454+0000", 2024-09-17T21:29:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571454+0000", 2024-09-17T21:29:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571455+0000", 2024-09-17T21:29:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:29:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571456+0000", 2024-09-17T21:29:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.64e-06 2024-09-17T21:29:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571462+0000", 2024-09-17T21:29:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6230000000000002e-06 2024-09-17T21:29:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571467+0000", 2024-09-17T21:29:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5199999999999999e-06 2024-09-17T21:29:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618091+0000", 2024-09-17T21:29:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046624489999999998 2024-09-17T21:29:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618172+0000", 2024-09-17T21:29:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1271000000000002e-05 2024-09-17T21:29:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618430+0000", 2024-09-17T21:29:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025807199999999998 2024-09-17T21:29:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5566 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571472+0000", 2024-09-17T21:29:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.51685436, 2024-09-17T21:29:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047328261000000003, 2024-09-17T21:29:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5566 2024-09-17T21:29:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571472+0000", 2024-09-17T21:29:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571472+0000", 2024-09-17T21:29:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571473+0000", 2024-09-17T21:29:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:29:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571474+0000", 2024-09-17T21:29:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.668e-06 2024-09-17T21:29:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571480+0000", 2024-09-17T21:29:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6899999999999997e-06 2024-09-17T21:29:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571489+0000", 2024-09-17T21:29:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8589999999999994e-06 2024-09-17T21:29:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618481+0000", 2024-09-17T21:29:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046992468000000003 2024-09-17T21:29:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618563+0000", 2024-09-17T21:29:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1464000000000005e-05 2024-09-17T21:29:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618801+0000", 2024-09-17T21:29:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000237762 2024-09-17T21:29:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5568 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 317440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571495+0000", 2024-09-17T21:29:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.516831872999999, 2024-09-17T21:29:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047753012999999997, 2024-09-17T21:29:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5568 2024-09-17T21:29:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571495+0000", 2024-09-17T21:29:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571495+0000", 2024-09-17T21:29:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571495+0000", 2024-09-17T21:29:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.89e-07 2024-09-17T21:29:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571497+0000", 2024-09-17T21:29:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.739e-06 2024-09-17T21:29:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571503+0000", 2024-09-17T21:29:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:29:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571507+0000", 2024-09-17T21:29:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3560000000000003e-06 2024-09-17T21:29:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618877+0000", 2024-09-17T21:29:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047370019999999999 2024-09-17T21:29:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618964+0000", 2024-09-17T21:29:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6615999999999999e-05 2024-09-17T21:29:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619248+0000", 2024-09-17T21:29:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000284154 2024-09-17T21:29:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5570 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 331776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571513+0000", 2024-09-17T21:29:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.516814010999999, 2024-09-17T21:29:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048061883999999999, 2024-09-17T21:29:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5570 2024-09-17T21:29:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571513+0000", 2024-09-17T21:29:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571513+0000", 2024-09-17T21:29:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571513+0000", 2024-09-17T21:29:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4700000000000002e-07 2024-09-17T21:29:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571518+0000", 2024-09-17T21:29:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.242e-06 2024-09-17T21:29:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571874+0000", 2024-09-17T21:29:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00035530799999999998 2024-09-17T21:29:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571884+0000", 2024-09-17T21:29:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0733e-05 2024-09-17T21:29:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619283+0000", 2024-09-17T21:29:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047398218999999998 2024-09-17T21:29:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619361+0000", 2024-09-17T21:29:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8660999999999998e-05 2024-09-17T21:29:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619575+0000", 2024-09-17T21:29:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021337400000000001 2024-09-17T21:29:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5572 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 346112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571897+0000", 2024-09-17T21:29:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.516430059999999, 2024-09-17T21:29:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048019328999999999, 2024-09-17T21:29:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5572 2024-09-17T21:29:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571897+0000", 2024-09-17T21:29:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571897+0000", 2024-09-17T21:29:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571897+0000", 2024-09-17T21:29:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8100000000000002e-07 2024-09-17T21:29:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571901+0000", 2024-09-17T21:29:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4369999999999999e-06 2024-09-17T21:29:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571909+0000", 2024-09-17T21:29:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.841e-06 2024-09-17T21:29:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571915+0000", 2024-09-17T21:29:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5130000000000002e-06 2024-09-17T21:29:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619619+0000", 2024-09-17T21:29:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047704203000000001 2024-09-17T21:29:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619698+0000", 2024-09-17T21:29:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8656e-05 2024-09-17T21:29:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619916+0000", 2024-09-17T21:29:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021789800000000001 2024-09-17T21:29:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5574 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 358400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571923+0000", 2024-09-17T21:29:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.51640403, 2024-09-17T21:29:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048336208999999998, 2024-09-17T21:29:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5574 2024-09-17T21:29:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571923+0000", 2024-09-17T21:29:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571923+0000", 2024-09-17T21:29:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571923+0000", 2024-09-17T21:29:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9900000000000001e-07 2024-09-17T21:29:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571925+0000", 2024-09-17T21:29:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.978e-06 2024-09-17T21:29:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571931+0000", 2024-09-17T21:29:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2530000000000001e-06 2024-09-17T21:29:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571936+0000", 2024-09-17T21:29:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6700000000000002e-06 2024-09-17T21:29:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619957+0000", 2024-09-17T21:29:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048020500000000001 2024-09-17T21:29:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.620037+0000", 2024-09-17T21:29:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0016000000000001e-05 2024-09-17T21:29:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.620259+0000", 2024-09-17T21:29:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022239300000000001 2024-09-17T21:29:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5576 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:29:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571944+0000", 2024-09-17T21:29:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.516382904, 2024-09-17T21:29:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048513909000000001, 2024-09-17T21:29:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:29:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:29:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:29:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:29:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:29:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:29:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5576 2024-09-17T21:29:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:29:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:29:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571944+0000", 2024-09-17T21:29:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:29:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571944+0000", 2024-09-17T21:29:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:29:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:29:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571952+0000", 2024-09-17T21:29:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7729999999999999e-06 2024-09-17T21:29:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:29:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571954+0000", 2024-09-17T21:29:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8310000000000001e-06 2024-09-17T21:29:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:29:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571964+0000", 2024-09-17T21:29:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4450000000000002e-06 2024-09-17T21:29:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:29:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571976+0000", 2024-09-17T21:29:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1793999999999999e-05 2024-09-17T21:29:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:29:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.620299+0000", 2024-09-17T21:29:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048323558000000003 2024-09-17T21:29:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:29:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.620373+0000", 2024-09-17T21:29:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4262000000000002e-05 2024-09-17T21:29:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:29:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:29:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:29:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.620458+0000", 2024-09-17T21:29:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4246000000000006e-05 2024-09-17T21:29:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:29:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:29:59.183 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:29:59.185 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:29:59.185 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:29:59.182+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:29:59.255 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:29:59.256 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:29:59.273 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:29:59.273 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:29:59.285 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:59.283+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:59.289 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:29:59.289 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:29:59.306 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:29:59.306 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:29:59.323 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:29:59.323 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:29:59.333 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:59.333 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:59.340 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:29:59.340 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:29:59.342 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:29:59.357 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:29:59.357 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:29:59.409 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:59.407+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:59.460 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:59.460 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:59.469 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:29:59.510 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:59.508+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:29:59.588 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:29:59.586+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:29:59.628 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:59.628 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:59.636 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:29:59.689 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:29:59.686+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:29:59.739 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:59.739 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:59.748 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:29:59.790 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:29:59.788+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:29:59.890 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:29:59.890 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:29:59.891 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:29:59.888+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:29:59.899 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:29:59.991 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:29:59.989+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:00.042 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:00.042 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:00.050 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:30:00.094 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:00.092+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:00.168 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:00.169 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:00.178 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:30:00.193 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:00.191+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:00.294 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:00.292+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:00.324 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:00.325 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:00.334 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:30:00.395 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:00.393+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:00.441 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:00.441 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:00.450 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:30:00.496 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:00.494+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:00.555 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:30:00.599 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:00.597+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:00.609 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:00.609 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:00.619 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:30:00.701 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:00.698+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:30:00.724 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:00.724 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:00.735 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:30:00.755 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-17T21:30:00.802 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:00.799+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:00.841 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:00.841 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:00.850 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:30:00.934 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:00.932+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:00.996 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:00.996 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:01.005 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:30:01.005 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:01.002+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:01.008 INFO:teuthology.orchestra.run.smithi129.stdout:1458 read 0x98169ca thru 0x9825cac (0xf2e3 bytes) 2024-09-17T21:30:01.013 INFO:teuthology.orchestra.run.smithi129.stdout:1462 trunc from 0xe002bf5 to 0xb91e671 2024-09-17T21:30:01.034 INFO:teuthology.orchestra.run.smithi129.stdout:1463 read 0xecde8e thru 0xed9986 (0xbaf9 bytes) 2024-09-17T21:30:01.034 INFO:teuthology.orchestra.run.smithi129.stdout:1464 read 0x3cb5a40 thru 0x3cc06e6 (0xaca7 bytes) 2024-09-17T21:30:01.035 INFO:teuthology.orchestra.run.smithi129.stdout:1465 write 0x9e8152 thru 0x9f1fd0 (0x9e7f bytes) 2024-09-17T21:30:01.039 INFO:teuthology.orchestra.run.smithi129.stdout:1466 write 0x570be2b thru 0x570f89c (0x3a72 bytes) 2024-09-17T21:30:01.043 INFO:teuthology.orchestra.run.smithi129.stdout:1467 writesame 0x8c45018 thru 0x8c490de data_size 0x17(0x40c7 bytes) 2024-09-17T21:30:01.048 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:01.060 INFO:teuthology.orchestra.run.smithi129.stdout:1469 trunc from 0xb91e671 to 0x105c9a9 2024-09-17T21:30:01.079 INFO:teuthology.orchestra.run.smithi129.stdout:1470 punch from 0xb9d850 to 0xba9e09, (0xc5b9 bytes) 2024-09-17T21:30:01.094 INFO:teuthology.orchestra.run.smithi129.stdout:1472 trunc from 0x105c9a9 to 0xd79b4be 2024-09-17T21:30:01.100 INFO:teuthology.orchestra.run.smithi129.stdout:1473 write 0xaee10a4 thru 0xaee28c5 (0x1822 bytes) 2024-09-17T21:30:01.103 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:01.103 INFO:teuthology.orchestra.run.smithi129.stdout:1476 read 0x89396bd thru 0x89476d7 (0xe01b bytes) 2024-09-17T21:30:01.103 INFO:teuthology.orchestra.run.smithi129.stdout:1477 read 0x3db6bd4 thru 0x3dbac5d (0x408a bytes) 2024-09-17T21:30:01.103 INFO:teuthology.orchestra.run.smithi129.stdout:1478 write 0xbf299f6 thru 0xbf2b6be (0x1cc9 bytes) 2024-09-17T21:30:01.105 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:01.103+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:01.106 INFO:teuthology.orchestra.run.smithi129.stdout:1479 flatten 2024-09-17T21:30:01.127 INFO:teuthology.orchestra.run.smithi129.stdout:1481 trunc from 0xd79b4be to 0x75e2eec 2024-09-17T21:30:01.138 INFO:teuthology.orchestra.run.smithi129.stdout:1482 punch from 0x4fafae to 0x50879d, (0xd7ef bytes) 2024-09-17T21:30:01.138 INFO:teuthology.orchestra.run.smithi129.stdout:1483 clone 9 order 21 su 8192 sc 8 2024-09-17T21:30:01.159 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:01.159 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:01.168 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:30:01.207 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:01.204+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:01.272 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:01.272 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:01.281 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:30:01.325 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:01.323+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:01.369 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:30:01.386 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:01.387 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:01.395 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:30:01.413 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:01.411+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:30:01.491 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:01.491 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:01.507 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:30:01.508 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:30:01.509 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:01.507+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:01.523 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:01.523 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:01.539 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:01.539 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:01.540 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:01.540 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:01.549 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:30:01.559 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:01.559 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:01.562 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is unset 2024-09-17T21:30:01.577 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:01.578 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:01.596 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:01.596 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:01.609 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:01.607+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:30:01.614 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:01.614 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:01.698 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:01.698 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:01.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:01.698 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:01.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.698 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:30:01.698 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:01.698 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.755138997000003, 2024-09-17T21:30:01.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61572000699999996, 2024-09-17T21:30:01.699 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.699 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.699 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:01.699 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.699 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:30:01.699 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:30:01.699 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T21:30:01.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.699 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:01.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:01.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939745+0000", 2024-09-17T21:30:01.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6899999999999999e-06 2024-09-17T21:30:01.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939753+0000", 2024-09-17T21:30:01.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1950000000000003e-06 2024-09-17T21:30:01.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939767+0000", 2024-09-17T21:30:01.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3429e-05 2024-09-17T21:30:01.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939776+0000", 2024-09-17T21:30:01.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:30:01.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939858+0000", 2024-09-17T21:30:01.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2496000000000004e-05 2024-09-17T21:30:01.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T21:30:01.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939868+0000", 2024-09-17T21:30:01.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1910000000000006e-06 2024-09-17T21:30:01.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940612+0000", 2024-09-17T21:30:01.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074429599999999998 2024-09-17T21:30:01.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:30:01.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940621+0000", 2024-09-17T21:30:01.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9139999999999997e-06 2024-09-17T21:30:01.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554006+0000", 2024-09-17T21:30:01.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61338505099999996 2024-09-17T21:30:01.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554166+0000", 2024-09-17T21:30:01.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015982600000000001 2024-09-17T21:30:01.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:30:01.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554296+0000", 2024-09-17T21:30:01.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013002399999999999 2024-09-17T21:30:01.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:01.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554936+0000", 2024-09-17T21:30:01.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064009400000000001 2024-09-17T21:30:01.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:01.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555389+0000", 2024-09-17T21:30:01.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045355200000000002 2024-09-17T21:30:01.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:01.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555432+0000", 2024-09-17T21:30:01.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2477e-05 2024-09-17T21:30:01.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555463+0000", 2024-09-17T21:30:01.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1612000000000002e-05 2024-09-17T21:30:01.705 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.705 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.705 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.706 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17527 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1005568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.706 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:01.706 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.289911166, 2024-09-17T21:30:01.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046452090000000001, 2024-09-17T21:30:01.706 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.706 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.706 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:01.706 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.706 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.706 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.706 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17527 2024-09-17T21:30:01.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.706 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:01.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:01.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405496+0000", 2024-09-17T21:30:01.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052498400000000004 2024-09-17T21:30:01.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405500+0000", 2024-09-17T21:30:01.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8410000000000002e-06 2024-09-17T21:30:01.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405511+0000", 2024-09-17T21:30:01.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0749e-05 2024-09-17T21:30:01.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405517+0000", 2024-09-17T21:30:01.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:30:01.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451275+0000", 2024-09-17T21:30:01.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045758134999999998 2024-09-17T21:30:01.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451355+0000", 2024-09-17T21:30:01.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9453e-05 2024-09-17T21:30:01.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451423+0000", 2024-09-17T21:30:01.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8782999999999995e-05 2024-09-17T21:30:01.710 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.710 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.710 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.710 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17530 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1046528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.710 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:01.710 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.289303515, 2024-09-17T21:30:01.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046376737000000001, 2024-09-17T21:30:01.710 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.710 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.710 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:01.711 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.711 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.711 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.711 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17530 2024-09-17T21:30:01.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.711 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:01.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:01.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:01.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3300000000000002e-07 2024-09-17T21:30:01.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405581+0000", 2024-09-17T21:30:01.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.074e-06 2024-09-17T21:30:01.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405587+0000", 2024-09-17T21:30:01.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7559999999999996e-06 2024-09-17T21:30:01.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405592+0000", 2024-09-17T21:30:01.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8860000000000003e-06 2024-09-17T21:30:01.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451813+0000", 2024-09-17T21:30:01.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046221119999999997 2024-09-17T21:30:01.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451891+0000", 2024-09-17T21:30:01.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7449999999999999e-05 2024-09-17T21:30:01.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451956+0000", 2024-09-17T21:30:01.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4918000000000004e-05 2024-09-17T21:30:01.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.714 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.715 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17615 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.715 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:01.715 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.284670677999999, 2024-09-17T21:30:01.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046741309000000002, 2024-09-17T21:30:01.715 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.715 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.715 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:01.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.715 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17615 2024-09-17T21:30:01.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.715 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:01.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:01.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:01.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:30:01.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410213+0000", 2024-09-17T21:30:01.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:30:01.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410217+0000", 2024-09-17T21:30:01.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8199999999999998e-06 2024-09-17T21:30:01.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410220+0000", 2024-09-17T21:30:01.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6900000000000001e-06 2024-09-17T21:30:01.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455264+0000", 2024-09-17T21:30:01.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045044002999999999 2024-09-17T21:30:01.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455338+0000", 2024-09-17T21:30:01.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4182000000000002e-05 2024-09-17T21:30:01.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.456953+0000", 2024-09-17T21:30:01.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016150089999999999 2024-09-17T21:30:01.719 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.719 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.719 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.719 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17616 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 872448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.719 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:01.719 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.284658626000001, 2024-09-17T21:30:01.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946039000000002, 2024-09-17T21:30:01.719 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.719 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.719 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:01.719 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.719 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.720 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.720 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17616 2024-09-17T21:30:01.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.720 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:01.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:01.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:01.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T21:30:01.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410225+0000", 2024-09-17T21:30:01.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3459999999999999e-06 2024-09-17T21:30:01.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410228+0000", 2024-09-17T21:30:01.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.083e-06 2024-09-17T21:30:01.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410232+0000", 2024-09-17T21:30:01.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4419999999999998e-06 2024-09-17T21:30:01.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457004+0000", 2024-09-17T21:30:01.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046771873999999998 2024-09-17T21:30:01.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457084+0000", 2024-09-17T21:30:01.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0506000000000005e-05 2024-09-17T21:30:01.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457170+0000", 2024-09-17T21:30:01.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5494000000000005e-05 2024-09-17T21:30:01.723 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.723 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.723 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.723 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17617 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.723 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:01.723 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.284647224, 2024-09-17T21:30:01.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047084168000000003, 2024-09-17T21:30:01.723 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.724 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.724 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:01.724 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.724 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.724 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.724 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17617 2024-09-17T21:30:01.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.724 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:01.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:01.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:01.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-17T21:30:01.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410237+0000", 2024-09-17T21:30:01.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1960000000000001e-06 2024-09-17T21:30:01.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410246+0000", 2024-09-17T21:30:01.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7880000000000001e-06 2024-09-17T21:30:01.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410250+0000", 2024-09-17T21:30:01.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.518e-06 2024-09-17T21:30:01.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457209+0000", 2024-09-17T21:30:01.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046959283999999997 2024-09-17T21:30:01.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457264+0000", 2024-09-17T21:30:01.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673999999999999e-05 2024-09-17T21:30:01.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457319+0000", 2024-09-17T21:30:01.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5497000000000001e-05 2024-09-17T21:30:01.727 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.727 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.727 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.728 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17618 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 899072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.728 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:01.728 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.284627779999999, 2024-09-17T21:30:01.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047204208999999997, 2024-09-17T21:30:01.728 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.728 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.728 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:01.728 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.728 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.728 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.728 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17618 2024-09-17T21:30:01.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.729 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:01.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:01.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:01.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8799999999999998e-07 2024-09-17T21:30:01.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410256+0000", 2024-09-17T21:30:01.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T21:30:01.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410260+0000", 2024-09-17T21:30:01.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6500000000000002e-06 2024-09-17T21:30:01.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410263+0000", 2024-09-17T21:30:01.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7089999999999998e-06 2024-09-17T21:30:01.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457355+0000", 2024-09-17T21:30:01.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047092756999999999 2024-09-17T21:30:01.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457407+0000", 2024-09-17T21:30:01.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1808999999999998e-05 2024-09-17T21:30:01.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457459+0000", 2024-09-17T21:30:01.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1684000000000002e-05 2024-09-17T21:30:01.732 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.732 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.732 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.733 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17619 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 913408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.733 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:01.733 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.284616542, 2024-09-17T21:30:01.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047293443999999997, 2024-09-17T21:30:01.733 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.733 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.733 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:01.733 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.733 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.733 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.733 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17619 2024-09-17T21:30:01.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.733 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:01.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:01.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:01.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:30:01.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410267+0000", 2024-09-17T21:30:01.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:30:01.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410270+0000", 2024-09-17T21:30:01.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.01e-06 2024-09-17T21:30:01.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410273+0000", 2024-09-17T21:30:01.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6410000000000002e-06 2024-09-17T21:30:01.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457473+0000", 2024-09-17T21:30:01.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047199829999999998 2024-09-17T21:30:01.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457517+0000", 2024-09-17T21:30:01.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4058999999999999e-05 2024-09-17T21:30:01.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457559+0000", 2024-09-17T21:30:01.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2404999999999998e-05 2024-09-17T21:30:01.737 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.737 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.737 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.737 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17620 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 927744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.737 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:01.737 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.28460537, 2024-09-17T21:30:01.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047402409, 2024-09-17T21:30:01.737 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.737 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.737 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:01.738 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.738 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.738 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.738 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17620 2024-09-17T21:30:01.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.738 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:01.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:01.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:01.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:30:01.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410278+0000", 2024-09-17T21:30:01.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1790000000000001e-06 2024-09-17T21:30:01.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410281+0000", 2024-09-17T21:30:01.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.061e-06 2024-09-17T21:30:01.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410284+0000", 2024-09-17T21:30:01.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.65e-06 2024-09-17T21:30:01.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457587+0000", 2024-09-17T21:30:01.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047303206 2024-09-17T21:30:01.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457633+0000", 2024-09-17T21:30:01.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6063999999999998e-05 2024-09-17T21:30:01.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457679+0000", 2024-09-17T21:30:01.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6007000000000002e-05 2024-09-17T21:30:01.741 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.741 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.741 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.741 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17621 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 940032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.741 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:01.742 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.284594953999999, 2024-09-17T21:30:01.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047486550000000002, 2024-09-17T21:30:01.742 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.742 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.742 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:01.742 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.742 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.742 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.742 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17621 2024-09-17T21:30:01.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.742 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:01.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:01.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410288+0000", 2024-09-17T21:30:01.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:30:01.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410289+0000", 2024-09-17T21:30:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1179999999999999e-06 2024-09-17T21:30:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410292+0000", 2024-09-17T21:30:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:30:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410294+0000", 2024-09-17T21:30:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T21:30:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457694+0000", 2024-09-17T21:30:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047399416999999999 2024-09-17T21:30:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457734+0000", 2024-09-17T21:30:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0528999999999998e-05 2024-09-17T21:30:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457774+0000", 2024-09-17T21:30:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9529e-05 2024-09-17T21:30:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17622 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.284584772000001, 2024-09-17T21:30:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047596619, 2024-09-17T21:30:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17622 2024-09-17T21:30:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410303+0000", 2024-09-17T21:30:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2979999999999999e-06 2024-09-17T21:30:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410307+0000", 2024-09-17T21:30:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:30:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410310+0000", 2024-09-17T21:30:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.146e-06 2024-09-17T21:30:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457801+0000", 2024-09-17T21:30:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047491322000000002 2024-09-17T21:30:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457850+0000", 2024-09-17T21:30:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8585000000000003e-05 2024-09-17T21:30:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457894+0000", 2024-09-17T21:30:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4455e-05 2024-09-17T21:30:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17623 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 968704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.284568388, 2024-09-17T21:30:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047689377999999998, 2024-09-17T21:30:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17623 2024-09-17T21:30:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:30:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410315+0000", 2024-09-17T21:30:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0499999999999999e-06 2024-09-17T21:30:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410319+0000", 2024-09-17T21:30:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.517e-06 2024-09-17T21:30:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410321+0000", 2024-09-17T21:30:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:30:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457922+0000", 2024-09-17T21:30:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047600638000000001 2024-09-17T21:30:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457964+0000", 2024-09-17T21:30:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2126000000000003e-05 2024-09-17T21:30:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458003+0000", 2024-09-17T21:30:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9138999999999999e-05 2024-09-17T21:30:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17624 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.284557804, 2024-09-17T21:30:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047798292999999999, 2024-09-17T21:30:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.756 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.756 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17624 2024-09-17T21:30:01.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.756 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:01.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:01.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:01.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:30:01.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410326+0000", 2024-09-17T21:30:01.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0610000000000001e-06 2024-09-17T21:30:01.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410329+0000", 2024-09-17T21:30:01.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0319999999999999e-06 2024-09-17T21:30:01.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410331+0000", 2024-09-17T21:30:01.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5749999999999999e-06 2024-09-17T21:30:01.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458033+0000", 2024-09-17T21:30:01.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047701371999999999 2024-09-17T21:30:01.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458079+0000", 2024-09-17T21:30:01.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:30:01.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458123+0000", 2024-09-17T21:30:01.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850999999999997e-05 2024-09-17T21:30:01.759 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.759 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.759 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.760 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17625 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.760 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:01.760 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.284547644, 2024-09-17T21:30:01.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047883051000000003, 2024-09-17T21:30:01.760 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.760 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.760 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:01.760 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.760 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.760 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.760 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17625 2024-09-17T21:30:01.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.760 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:01.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:01.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:01.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:01.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410336+0000", 2024-09-17T21:30:01.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1310000000000001e-06 2024-09-17T21:30:01.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410339+0000", 2024-09-17T21:30:01.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000001e-06 2024-09-17T21:30:01.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410342+0000", 2024-09-17T21:30:01.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7460000000000001e-06 2024-09-17T21:30:01.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458137+0000", 2024-09-17T21:30:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047794931999999998 2024-09-17T21:30:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458178+0000", 2024-09-17T21:30:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1013999999999997e-05 2024-09-17T21:30:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458218+0000", 2024-09-17T21:30:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9675999999999998e-05 2024-09-17T21:30:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.764 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.764 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.764 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17626 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1009664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.764 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:01.764 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.284537023, 2024-09-17T21:30:01.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047991601000000002, 2024-09-17T21:30:01.764 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.764 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.764 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:01.764 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.764 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17626 2024-09-17T21:30:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410346+0000", 2024-09-17T21:30:01.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:30:01.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410347+0000", 2024-09-17T21:30:01.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.114e-06 2024-09-17T21:30:01.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410350+0000", 2024-09-17T21:30:01.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.101e-06 2024-09-17T21:30:01.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410353+0000", 2024-09-17T21:30:01.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T21:30:01.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458245+0000", 2024-09-17T21:30:01.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0478921 2024-09-17T21:30:01.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458292+0000", 2024-09-17T21:30:01.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397999999999999e-05 2024-09-17T21:30:01.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458337+0000", 2024-09-17T21:30:01.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5221e-05 2024-09-17T21:30:01.768 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.768 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.768 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.768 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17627 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.768 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:01.768 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.284499922, 2024-09-17T21:30:01.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048047383999999999, 2024-09-17T21:30:01.769 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.769 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.769 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:01.769 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.769 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.769 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.769 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17627 2024-09-17T21:30:01.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.769 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:01.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:01.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410383+0000", 2024-09-17T21:30:01.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:30:01.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410385+0000", 2024-09-17T21:30:01.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:30:01.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410391+0000", 2024-09-17T21:30:01.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:30:01.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410395+0000", 2024-09-17T21:30:01.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6539999999999998e-06 2024-09-17T21:30:01.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458350+0000", 2024-09-17T21:30:01.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047954855999999997 2024-09-17T21:30:01.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458392+0000", 2024-09-17T21:30:01.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2094000000000001e-05 2024-09-17T21:30:01.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458430+0000", 2024-09-17T21:30:01.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7474000000000001e-05 2024-09-17T21:30:01.772 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.772 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.772 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.773 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17628 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.773 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:01.773 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.284480361, 2024-09-17T21:30:01.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048142676000000002, 2024-09-17T21:30:01.773 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.773 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.773 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:01.773 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.773 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.773 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.773 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17628 2024-09-17T21:30:01.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.773 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:01.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:01.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:01.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:30:01.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410404+0000", 2024-09-17T21:30:01.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-06 2024-09-17T21:30:01.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410411+0000", 2024-09-17T21:30:01.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6850000000000004e-06 2024-09-17T21:30:01.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410415+0000", 2024-09-17T21:30:01.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6750000000000001e-06 2024-09-17T21:30:01.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458457+0000", 2024-09-17T21:30:01.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048042045999999998 2024-09-17T21:30:01.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458501+0000", 2024-09-17T21:30:01.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3661000000000001e-05 2024-09-17T21:30:01.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458545+0000", 2024-09-17T21:30:01.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3529000000000003e-05 2024-09-17T21:30:01.776 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.777 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.777 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.777 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17687 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.777 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:01.777 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.282035408, 2024-09-17T21:30:01.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050670448, 2024-09-17T21:30:01.777 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.777 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.777 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:01.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.778 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.778 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.778 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17687 2024-09-17T21:30:01.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.778 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:01.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:01.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412848+0000", 2024-09-17T21:30:01.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9100000000000003e-07 2024-09-17T21:30:01.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413102+0000", 2024-09-17T21:30:01.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025415600000000001 2024-09-17T21:30:01.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413111+0000", 2024-09-17T21:30:01.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2010000000000005e-06 2024-09-17T21:30:01.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413116+0000", 2024-09-17T21:30:01.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:30:01.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458559+0000", 2024-09-17T21:30:01.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045442914000000001 2024-09-17T21:30:01.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458632+0000", 2024-09-17T21:30:01.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3207000000000006e-05 2024-09-17T21:30:01.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:01.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458785+0000", 2024-09-17T21:30:01.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015226700000000001 2024-09-17T21:30:01.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:01.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.461648+0000", 2024-09-17T21:30:01.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028633069999999998 2024-09-17T21:30:01.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:01.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463416+0000", 2024-09-17T21:30:01.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017680230000000001 2024-09-17T21:30:01.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:01.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463475+0000", 2024-09-17T21:30:01.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9333000000000003e-05 2024-09-17T21:30:01.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463517+0000", 2024-09-17T21:30:01.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2162000000000001e-05 2024-09-17T21:30:01.783 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.783 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.783 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.783 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17688 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [write 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.783 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:01.783 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.28175654, 2024-09-17T21:30:01.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050625098, 2024-09-17T21:30:01.783 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.783 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.783 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:01.783 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.783 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.783 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.784 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17688 2024-09-17T21:30:01.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.784 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:01.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:01.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:01.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:30:01.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413128+0000", 2024-09-17T21:30:01.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.813e-06 2024-09-17T21:30:01.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413132+0000", 2024-09-17T21:30:01.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:30:01.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413135+0000", 2024-09-17T21:30:01.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:30:01.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459756+0000", 2024-09-17T21:30:01.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046621293000000001 2024-09-17T21:30:01.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459816+0000", 2024-09-17T21:30:01.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9156000000000001e-05 2024-09-17T21:30:01.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:01.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459912+0000", 2024-09-17T21:30:01.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6302000000000001e-05 2024-09-17T21:30:01.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:01.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463231+0000", 2024-09-17T21:30:01.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033188290000000001 2024-09-17T21:30:01.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:01.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463701+0000", 2024-09-17T21:30:01.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047068199999999999 2024-09-17T21:30:01.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:01.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463728+0000", 2024-09-17T21:30:01.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6726000000000001e-05 2024-09-17T21:30:01.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463751+0000", 2024-09-17T21:30:01.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2824e-05 2024-09-17T21:30:01.788 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.788 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.788 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.789 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17691 3.d 3:b3a055f3:::rbd_data.11207744cc2c.0000000000000083:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:01.789 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:01.789 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.278211637, 2024-09-17T21:30:01.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049309370999999998, 2024-09-17T21:30:01.789 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:01.789 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:01.789 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:01.789 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:01.789 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:01.789 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.789 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17691 2024-09-17T21:30:01.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.790 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:01.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:01.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:01.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:01.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:01.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:01.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:01.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416672+0000", 2024-09-17T21:30:01.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.47e-07 2024-09-17T21:30:01.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:01.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416915+0000", 2024-09-17T21:30:01.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024344500000000001 2024-09-17T21:30:01.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:01.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416924+0000", 2024-09-17T21:30:01.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7739999999999993e-06 2024-09-17T21:30:01.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416929+0000", 2024-09-17T21:30:01.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6990000000000002e-06 2024-09-17T21:30:01.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:01.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460414+0000", 2024-09-17T21:30:01.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043485466 2024-09-17T21:30:01.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:01.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460540+0000", 2024-09-17T21:30:01.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012608899999999999 2024-09-17T21:30:01.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:01.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460746+0000", 2024-09-17T21:30:01.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000205577 2024-09-17T21:30:01.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:01.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463800+0000", 2024-09-17T21:30:01.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030542030000000001 2024-09-17T21:30:01.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:01.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465867+0000", 2024-09-17T21:30:01.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020665929999999998 2024-09-17T21:30:01.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:01.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465914+0000", 2024-09-17T21:30:01.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497000000000003e-05 2024-09-17T21:30:01.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:01.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:01.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:01.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465980+0000", 2024-09-17T21:30:01.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6081000000000006e-05 2024-09-17T21:30:01.795 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.795 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.795 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.795 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:01.795 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:01.795 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:01.796 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:30:01.797 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:01.794+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:30:01.898 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:01.895+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:30:01.900 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:01.900 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:01.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:01.901 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:37817 3.17 3.b15a1d57 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:30:01.901 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:01.901 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.20371409500000001, 2024-09-17T21:30:01.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.203734902, 2024-09-17T21:30:01.901 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:01.901 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:01.901 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "waiting for peered", 2024-09-17T21:30:01.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:01.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:01.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:01.901 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37817 2024-09-17T21:30:01.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:01.902 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:01.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:01.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:01.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:01.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:01.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:01.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:01.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:01.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:01.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:01.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:01.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:01.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:01.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695457+0000", 2024-09-17T21:30:01.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5799999999999998e-07 2024-09-17T21:30:01.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:01.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:01.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:01.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695461+0000", 2024-09-17T21:30:01.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1110000000000001e-06 2024-09-17T21:30:01.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:01.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:01.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:01.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695471+0000", 2024-09-17T21:30:01.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0441e-05 2024-09-17T21:30:01.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:01.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:01.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:01.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695477+0000", 2024-09-17T21:30:01.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7599999999999999e-06 2024-09-17T21:30:01.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:01.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:01.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:01.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695533+0000", 2024-09-17T21:30:01.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6570999999999998e-05 2024-09-17T21:30:01.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:01.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:01.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:30:01.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695543+0000", 2024-09-17T21:30:01.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9979999999999998e-06 2024-09-17T21:30:01.905 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:01.905 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:01.905 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:01.905 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:01.905 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:30:01.905 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 1 2024-09-17T21:30:01.905 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:01.909 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:30:01.999 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:01.997+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:02.051 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:02.052 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:02.052 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:02.052 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:02.052 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:02.060 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:30:02.100 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:02.098+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:02.164 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:02.164 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:02.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:02.164 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:02.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.164 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5316 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 905216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.164 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.593733+0000", 2024-09-17T21:30:02.164 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.567238334000001, 2024-09-17T21:30:02.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044533852999999998, 2024-09-17T21:30:02.165 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.165 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.165 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.165 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.165 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.165 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.165 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5316 2024-09-17T21:30:02.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.165 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593733+0000", 2024-09-17T21:30:02.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593733+0000", 2024-09-17T21:30:02.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593733+0000", 2024-09-17T21:30:02.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8799999999999998e-07 2024-09-17T21:30:02.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593734+0000", 2024-09-17T21:30:02.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.249e-06 2024-09-17T21:30:02.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593738+0000", 2024-09-17T21:30:02.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2210000000000002e-06 2024-09-17T21:30:02.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593740+0000", 2024-09-17T21:30:02.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4600000000000002e-06 2024-09-17T21:30:02.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638172+0000", 2024-09-17T21:30:02.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044431598000000003 2024-09-17T21:30:02.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638213+0000", 2024-09-17T21:30:02.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1068000000000001e-05 2024-09-17T21:30:02.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638267+0000", 2024-09-17T21:30:02.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3869000000000002e-05 2024-09-17T21:30:02.168 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.168 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.169 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.169 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5317 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 1017856~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.169 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.593757+0000", 2024-09-17T21:30:02.169 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.567213779999999, 2024-09-17T21:30:02.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044707716000000002, 2024-09-17T21:30:02.169 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.169 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.169 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.169 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.169 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.169 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.170 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5317 2024-09-17T21:30:02.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.170 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593757+0000", 2024-09-17T21:30:02.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593757+0000", 2024-09-17T21:30:02.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593758+0000", 2024-09-17T21:30:02.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.79e-07 2024-09-17T21:30:02.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593759+0000", 2024-09-17T21:30:02.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.451e-06 2024-09-17T21:30:02.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593762+0000", 2024-09-17T21:30:02.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4489999999999999e-06 2024-09-17T21:30:02.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593765+0000", 2024-09-17T21:30:02.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:30:02.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638302+0000", 2024-09-17T21:30:02.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044536394 2024-09-17T21:30:02.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638376+0000", 2024-09-17T21:30:02.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4016000000000004e-05 2024-09-17T21:30:02.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638465+0000", 2024-09-17T21:30:02.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9136999999999997e-05 2024-09-17T21:30:02.173 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.173 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.173 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.173 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5318 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 1079296~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.173 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.593770+0000", 2024-09-17T21:30:02.173 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.56720112, 2024-09-17T21:30:02.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044817632000000003, 2024-09-17T21:30:02.174 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.174 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.174 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.174 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.174 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.174 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.174 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5318 2024-09-17T21:30:02.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.174 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593770+0000", 2024-09-17T21:30:02.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593770+0000", 2024-09-17T21:30:02.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593770+0000", 2024-09-17T21:30:02.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.29e-07 2024-09-17T21:30:02.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593771+0000", 2024-09-17T21:30:02.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0160000000000001e-06 2024-09-17T21:30:02.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593774+0000", 2024-09-17T21:30:02.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9569999999999998e-06 2024-09-17T21:30:02.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593777+0000", 2024-09-17T21:30:02.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4210000000000002e-06 2024-09-17T21:30:02.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638494+0000", 2024-09-17T21:30:02.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044717645 2024-09-17T21:30:02.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638533+0000", 2024-09-17T21:30:02.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9051999999999998e-05 2024-09-17T21:30:02.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638588+0000", 2024-09-17T21:30:02.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4311999999999998e-05 2024-09-17T21:30:02.177 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.177 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.177 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.178 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5319 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 1130496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.178 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.593780+0000", 2024-09-17T21:30:02.178 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.567191187000001, 2024-09-17T21:30:02.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045002081999999999, 2024-09-17T21:30:02.178 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.178 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.178 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.178 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.178 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.178 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.178 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5319 2024-09-17T21:30:02.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.178 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593780+0000", 2024-09-17T21:30:02.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593780+0000", 2024-09-17T21:30:02.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593780+0000", 2024-09-17T21:30:02.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2599999999999998e-07 2024-09-17T21:30:02.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593781+0000", 2024-09-17T21:30:02.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7599999999999996e-07 2024-09-17T21:30:02.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593784+0000", 2024-09-17T21:30:02.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9229999999999998e-06 2024-09-17T21:30:02.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593790+0000", 2024-09-17T21:30:02.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1840000000000004e-06 2024-09-17T21:30:02.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638622+0000", 2024-09-17T21:30:02.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044831862 2024-09-17T21:30:02.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638695+0000", 2024-09-17T21:30:02.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3032000000000001e-05 2024-09-17T21:30:02.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638782+0000", 2024-09-17T21:30:02.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6879000000000006e-05 2024-09-17T21:30:02.182 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.182 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.182 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.182 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5320 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 1191936~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.182 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.593794+0000", 2024-09-17T21:30:02.182 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.567177471000001, 2024-09-17T21:30:02.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045110798000000001, 2024-09-17T21:30:02.183 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.183 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.183 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.183 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.183 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.183 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.183 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5320 2024-09-17T21:30:02.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.183 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593794+0000", 2024-09-17T21:30:02.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593794+0000", 2024-09-17T21:30:02.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593794+0000", 2024-09-17T21:30:02.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4999999999999999e-07 2024-09-17T21:30:02.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593795+0000", 2024-09-17T21:30:02.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0550000000000001e-06 2024-09-17T21:30:02.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593798+0000", 2024-09-17T21:30:02.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0790000000000001e-06 2024-09-17T21:30:02.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593801+0000", 2024-09-17T21:30:02.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9409999999999999e-06 2024-09-17T21:30:02.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638810+0000", 2024-09-17T21:30:02.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045009277 2024-09-17T21:30:02.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638849+0000", 2024-09-17T21:30:02.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8980000000000003e-05 2024-09-17T21:30:02.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638904+0000", 2024-09-17T21:30:02.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5216000000000001e-05 2024-09-17T21:30:02.186 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.186 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.186 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.187 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5321 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 1304576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.187 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.593804+0000", 2024-09-17T21:30:02.187 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.567166926000001, 2024-09-17T21:30:02.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045293438999999998, 2024-09-17T21:30:02.187 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.187 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.187 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.187 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5321 2024-09-17T21:30:02.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.188 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593804+0000", 2024-09-17T21:30:02.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593804+0000", 2024-09-17T21:30:02.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593804+0000", 2024-09-17T21:30:02.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.36e-07 2024-09-17T21:30:02.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593805+0000", 2024-09-17T21:30:02.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9300000000000006e-07 2024-09-17T21:30:02.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593808+0000", 2024-09-17T21:30:02.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9610000000000001e-06 2024-09-17T21:30:02.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593811+0000", 2024-09-17T21:30:02.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.407e-06 2024-09-17T21:30:02.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.638939+0000", 2024-09-17T21:30:02.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045128387999999998 2024-09-17T21:30:02.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.639010+0000", 2024-09-17T21:30:02.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0419999999999993e-05 2024-09-17T21:30:02.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.639098+0000", 2024-09-17T21:30:02.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8034000000000005e-05 2024-09-17T21:30:02.191 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.191 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.191 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.191 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5322 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 55296~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.191 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.593814+0000", 2024-09-17T21:30:02.191 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.567157162000001, 2024-09-17T21:30:02.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045414585, 2024-09-17T21:30:02.191 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.191 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.191 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.192 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.192 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.192 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.192 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5322 2024-09-17T21:30:02.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.192 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593814+0000", 2024-09-17T21:30:02.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593814+0000", 2024-09-17T21:30:02.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593814+0000", 2024-09-17T21:30:02.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2000000000000001e-07 2024-09-17T21:30:02.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593815+0000", 2024-09-17T21:30:02.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8199999999999998e-07 2024-09-17T21:30:02.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593818+0000", 2024-09-17T21:30:02.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9299999999999999e-06 2024-09-17T21:30:02.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593820+0000", 2024-09-17T21:30:02.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4320000000000002e-06 2024-09-17T21:30:02.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.639126+0000", 2024-09-17T21:30:02.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045305426000000003 2024-09-17T21:30:02.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.639173+0000", 2024-09-17T21:30:02.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7657999999999997e-05 2024-09-17T21:30:02.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.639229+0000", 2024-09-17T21:30:02.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5037000000000002e-05 2024-09-17T21:30:02.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.195 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.196 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.196 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5323 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 106496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.196 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.593823+0000", 2024-09-17T21:30:02.196 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.567147610999999, 2024-09-17T21:30:02.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045605916000000003, 2024-09-17T21:30:02.196 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.196 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.196 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.196 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.196 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.196 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.196 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5323 2024-09-17T21:30:02.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.197 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593823+0000", 2024-09-17T21:30:02.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.593823+0000", 2024-09-17T21:30:02.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599270+0000", 2024-09-17T21:30:02.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0054463580000000001 2024-09-17T21:30:02.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599276+0000", 2024-09-17T21:30:02.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3620000000000002e-06 2024-09-17T21:30:02.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599287+0000", 2024-09-17T21:30:02.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0721e-05 2024-09-17T21:30:02.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599296+0000", 2024-09-17T21:30:02.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.969e-06 2024-09-17T21:30:02.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.639264+0000", 2024-09-17T21:30:02.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039967920999999997 2024-09-17T21:30:02.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.639337+0000", 2024-09-17T21:30:02.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3510999999999995e-05 2024-09-17T21:30:02.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.200 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.639429+0000", 2024-09-17T21:30:02.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2074e-05 2024-09-17T21:30:02.200 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.200 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.200 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.200 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5347 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 518144~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.200 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599747+0000", 2024-09-17T21:30:02.200 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.5612244, 2024-09-17T21:30:02.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044652727000000003, 2024-09-17T21:30:02.200 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.200 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.201 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.201 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.201 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.201 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.201 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5347 2024-09-17T21:30:02.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.201 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.201 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599747+0000", 2024-09-17T21:30:02.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599747+0000", 2024-09-17T21:30:02.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599747+0000", 2024-09-17T21:30:02.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:30:02.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599748+0000", 2024-09-17T21:30:02.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8400000000000002e-07 2024-09-17T21:30:02.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599752+0000", 2024-09-17T21:30:02.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.754e-06 2024-09-17T21:30:02.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599754+0000", 2024-09-17T21:30:02.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.34e-06 2024-09-17T21:30:02.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.644238+0000", 2024-09-17T21:30:02.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044484403999999998 2024-09-17T21:30:02.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.644310+0000", 2024-09-17T21:30:02.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1423999999999999e-05 2024-09-17T21:30:02.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.644399+0000", 2024-09-17T21:30:02.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9602e-05 2024-09-17T21:30:02.204 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.204 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.204 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.204 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5348 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 579584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.205 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599757+0000", 2024-09-17T21:30:02.205 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.561213606999999, 2024-09-17T21:30:02.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045142726000000001, 2024-09-17T21:30:02.205 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.205 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.205 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.205 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.205 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.205 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.205 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5348 2024-09-17T21:30:02.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.205 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.205 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599757+0000", 2024-09-17T21:30:02.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599757+0000", 2024-09-17T21:30:02.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599758+0000", 2024-09-17T21:30:02.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.16e-07 2024-09-17T21:30:02.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599765+0000", 2024-09-17T21:30:02.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.063e-06 2024-09-17T21:30:02.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599768+0000", 2024-09-17T21:30:02.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.455e-06 2024-09-17T21:30:02.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599771+0000", 2024-09-17T21:30:02.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.565e-06 2024-09-17T21:30:02.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.644438+0000", 2024-09-17T21:30:02.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044667017000000003 2024-09-17T21:30:02.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.644532+0000", 2024-09-17T21:30:02.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4238000000000001e-05 2024-09-17T21:30:02.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.644900+0000", 2024-09-17T21:30:02.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000368172 2024-09-17T21:30:02.209 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.209 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.209 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.209 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5349 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 630784~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.209 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599774+0000", 2024-09-17T21:30:02.209 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.561196874, 2024-09-17T21:30:02.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045306261, 2024-09-17T21:30:02.209 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.209 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.209 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.209 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.210 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5349 2024-09-17T21:30:02.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.210 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599774+0000", 2024-09-17T21:30:02.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599774+0000", 2024-09-17T21:30:02.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599774+0000", 2024-09-17T21:30:02.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:30:02.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599775+0000", 2024-09-17T21:30:02.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0240000000000001e-06 2024-09-17T21:30:02.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599778+0000", 2024-09-17T21:30:02.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9469999999999999e-06 2024-09-17T21:30:02.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599781+0000", 2024-09-17T21:30:02.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4080000000000001e-06 2024-09-17T21:30:02.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.644922+0000", 2024-09-17T21:30:02.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045141463 2024-09-17T21:30:02.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.644993+0000", 2024-09-17T21:30:02.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1109e-05 2024-09-17T21:30:02.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645080+0000", 2024-09-17T21:30:02.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7089000000000006e-05 2024-09-17T21:30:02.213 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.213 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.213 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.213 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5350 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 692224~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.213 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599784+0000", 2024-09-17T21:30:02.213 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.561187104, 2024-09-17T21:30:02.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045533090999999998, 2024-09-17T21:30:02.214 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.214 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.214 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.214 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.214 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.214 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.214 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5350 2024-09-17T21:30:02.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.214 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599784+0000", 2024-09-17T21:30:02.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599784+0000", 2024-09-17T21:30:02.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599784+0000", 2024-09-17T21:30:02.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:02.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599785+0000", 2024-09-17T21:30:02.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2699999999999998e-07 2024-09-17T21:30:02.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599789+0000", 2024-09-17T21:30:02.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7189999999999999e-06 2024-09-17T21:30:02.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599796+0000", 2024-09-17T21:30:02.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3849999999999996e-06 2024-09-17T21:30:02.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645122+0000", 2024-09-17T21:30:02.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045325987999999998 2024-09-17T21:30:02.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645210+0000", 2024-09-17T21:30:02.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7905000000000006e-05 2024-09-17T21:30:02.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645317+0000", 2024-09-17T21:30:02.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106953 2024-09-17T21:30:02.217 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.217 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.218 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.218 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5351 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 804864~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.218 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599802+0000", 2024-09-17T21:30:02.218 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.561168738999999, 2024-09-17T21:30:02.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.04571066, 2024-09-17T21:30:02.218 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.218 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.218 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.218 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.218 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.218 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.218 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5351 2024-09-17T21:30:02.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.219 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599802+0000", 2024-09-17T21:30:02.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599802+0000", 2024-09-17T21:30:02.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599803+0000", 2024-09-17T21:30:02.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:02.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599809+0000", 2024-09-17T21:30:02.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0599999999999996e-06 2024-09-17T21:30:02.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599815+0000", 2024-09-17T21:30:02.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6499999999999999e-06 2024-09-17T21:30:02.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599820+0000", 2024-09-17T21:30:02.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6630000000000001e-06 2024-09-17T21:30:02.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645354+0000", 2024-09-17T21:30:02.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045533853999999999 2024-09-17T21:30:02.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645424+0000", 2024-09-17T21:30:02.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0191000000000006e-05 2024-09-17T21:30:02.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645513+0000", 2024-09-17T21:30:02.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8863999999999995e-05 2024-09-17T21:30:02.222 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.222 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.222 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.222 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5352 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 866304~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.222 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599827+0000", 2024-09-17T21:30:02.222 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.561144412999999, 2024-09-17T21:30:02.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045890574000000003, 2024-09-17T21:30:02.222 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.222 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.223 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.223 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.223 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.223 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.223 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5352 2024-09-17T21:30:02.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.223 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599827+0000", 2024-09-17T21:30:02.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599827+0000", 2024-09-17T21:30:02.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599827+0000", 2024-09-17T21:30:02.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6100000000000002e-07 2024-09-17T21:30:02.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599829+0000", 2024-09-17T21:30:02.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.663e-06 2024-09-17T21:30:02.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599834+0000", 2024-09-17T21:30:02.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4269999999999999e-06 2024-09-17T21:30:02.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599838+0000", 2024-09-17T21:30:02.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1880000000000004e-06 2024-09-17T21:30:02.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645554+0000", 2024-09-17T21:30:02.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045715649999999997 2024-09-17T21:30:02.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645634+0000", 2024-09-17T21:30:02.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0276000000000002e-05 2024-09-17T21:30:02.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645717+0000", 2024-09-17T21:30:02.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3009000000000004e-05 2024-09-17T21:30:02.226 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.226 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.226 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.227 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5353 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 917504~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.227 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599857+0000", 2024-09-17T21:30:02.227 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.561114032000001, 2024-09-17T21:30:02.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046056769999999997, 2024-09-17T21:30:02.227 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.227 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.227 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.227 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.227 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.227 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.227 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5353 2024-09-17T21:30:02.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.227 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599857+0000", 2024-09-17T21:30:02.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599857+0000", 2024-09-17T21:30:02.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599858+0000", 2024-09-17T21:30:02.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2600000000000002e-07 2024-09-17T21:30:02.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599860+0000", 2024-09-17T21:30:02.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.131e-06 2024-09-17T21:30:02.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599866+0000", 2024-09-17T21:30:02.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1789999999999996e-06 2024-09-17T21:30:02.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599871+0000", 2024-09-17T21:30:02.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9969999999999998e-06 2024-09-17T21:30:02.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645750+0000", 2024-09-17T21:30:02.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045879511999999997 2024-09-17T21:30:02.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645824+0000", 2024-09-17T21:30:02.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3738000000000004e-05 2024-09-17T21:30:02.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645914+0000", 2024-09-17T21:30:02.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9686999999999997e-05 2024-09-17T21:30:02.231 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.231 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.231 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.231 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5355 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 978944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.231 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599878+0000", 2024-09-17T21:30:02.231 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.56109318, 2024-09-17T21:30:02.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046157286999999998, 2024-09-17T21:30:02.231 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.232 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.232 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.232 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.232 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.232 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.232 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5355 2024-09-17T21:30:02.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.232 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599878+0000", 2024-09-17T21:30:02.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599878+0000", 2024-09-17T21:30:02.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599878+0000", 2024-09-17T21:30:02.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.27e-07 2024-09-17T21:30:02.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599880+0000", 2024-09-17T21:30:02.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9180000000000001e-06 2024-09-17T21:30:02.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599886+0000", 2024-09-17T21:30:02.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0839999999999997e-06 2024-09-17T21:30:02.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599891+0000", 2024-09-17T21:30:02.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3290000000000004e-06 2024-09-17T21:30:02.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645927+0000", 2024-09-17T21:30:02.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046036430000000003 2024-09-17T21:30:02.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.645982+0000", 2024-09-17T21:30:02.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4697999999999997e-05 2024-09-17T21:30:02.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646035+0000", 2024-09-17T21:30:02.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3400999999999999e-05 2024-09-17T21:30:02.235 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.235 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.236 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.236 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5357 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 1091584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.236 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599897+0000", 2024-09-17T21:30:02.236 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.561073616, 2024-09-17T21:30:02.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046338788999999998, 2024-09-17T21:30:02.236 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.236 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.236 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.236 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.236 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.236 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.236 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5357 2024-09-17T21:30:02.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.237 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599897+0000", 2024-09-17T21:30:02.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599897+0000", 2024-09-17T21:30:02.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599898+0000", 2024-09-17T21:30:02.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:30:02.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599903+0000", 2024-09-17T21:30:02.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:30:02.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599912+0000", 2024-09-17T21:30:02.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0049999999999993e-06 2024-09-17T21:30:02.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599924+0000", 2024-09-17T21:30:02.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1539000000000001e-05 2024-09-17T21:30:02.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646067+0000", 2024-09-17T21:30:02.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046143048999999998 2024-09-17T21:30:02.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646141+0000", 2024-09-17T21:30:02.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3955000000000006e-05 2024-09-17T21:30:02.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646236+0000", 2024-09-17T21:30:02.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5484999999999998e-05 2024-09-17T21:30:02.240 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.240 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.240 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.240 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5359 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 1153024~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.240 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599932+0000", 2024-09-17T21:30:02.240 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.561039522, 2024-09-17T21:30:02.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046420520999999999, 2024-09-17T21:30:02.240 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.241 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.241 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.241 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.241 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.241 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.241 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5359 2024-09-17T21:30:02.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.241 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599932+0000", 2024-09-17T21:30:02.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599932+0000", 2024-09-17T21:30:02.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599932+0000", 2024-09-17T21:30:02.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6600000000000002e-07 2024-09-17T21:30:02.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599934+0000", 2024-09-17T21:30:02.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3329999999999999e-06 2024-09-17T21:30:02.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599941+0000", 2024-09-17T21:30:02.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7399999999999998e-06 2024-09-17T21:30:02.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599946+0000", 2024-09-17T21:30:02.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8450000000000002e-06 2024-09-17T21:30:02.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646263+0000", 2024-09-17T21:30:02.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046316619000000003 2024-09-17T21:30:02.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646301+0000", 2024-09-17T21:30:02.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8379e-05 2024-09-17T21:30:02.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646352+0000", 2024-09-17T21:30:02.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1138999999999999e-05 2024-09-17T21:30:02.244 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.244 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.244 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.245 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5361 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 1204224~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.245 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599953+0000", 2024-09-17T21:30:02.245 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.561018217999999, 2024-09-17T21:30:02.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046597168000000001, 2024-09-17T21:30:02.245 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.245 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.245 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.245 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.245 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.245 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.245 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5361 2024-09-17T21:30:02.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.245 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599953+0000", 2024-09-17T21:30:02.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599953+0000", 2024-09-17T21:30:02.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599953+0000", 2024-09-17T21:30:02.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1399999999999997e-07 2024-09-17T21:30:02.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599955+0000", 2024-09-17T21:30:02.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0729999999999999e-06 2024-09-17T21:30:02.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599962+0000", 2024-09-17T21:30:02.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4690000000000002e-06 2024-09-17T21:30:02.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599970+0000", 2024-09-17T21:30:02.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8399999999999995e-06 2024-09-17T21:30:02.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646385+0000", 2024-09-17T21:30:02.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046415005000000002 2024-09-17T21:30:02.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646459+0000", 2024-09-17T21:30:02.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4214000000000005e-05 2024-09-17T21:30:02.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646550+0000", 2024-09-17T21:30:02.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1153e-05 2024-09-17T21:30:02.249 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.249 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.249 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.249 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5364 3.1b 3:dc08ecff:::rbd_data.1120555f8258.0000000000000000:3 [read 1265664~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.249 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.599975+0000", 2024-09-17T21:30:02.249 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.560995906, 2024-09-17T21:30:02.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046685196999999998, 2024-09-17T21:30:02.249 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.249 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.250 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.250 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.250 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.250 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.250 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5364 2024-09-17T21:30:02.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.250 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599975+0000", 2024-09-17T21:30:02.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.599975+0000", 2024-09-17T21:30:02.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.605220+0000", 2024-09-17T21:30:02.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0052443769999999997 2024-09-17T21:30:02.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.605226+0000", 2024-09-17T21:30:02.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3729999999999998e-06 2024-09-17T21:30:02.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.605239+0000", 2024-09-17T21:30:02.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2575000000000001e-05 2024-09-17T21:30:02.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.605246+0000", 2024-09-17T21:30:02.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7859999999999997e-06 2024-09-17T21:30:02.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646574+0000", 2024-09-17T21:30:02.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041328107000000003 2024-09-17T21:30:02.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646610+0000", 2024-09-17T21:30:02.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5512000000000002e-05 2024-09-17T21:30:02.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.646660+0000", 2024-09-17T21:30:02.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0467000000000003e-05 2024-09-17T21:30:02.253 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.253 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.253 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.254 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.254 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.254 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:02.255 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:30:02.255 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:02.253+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:02.356 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:02.354+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:02.400 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:02.400 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:02.400 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:02.400 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:02.409 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:30:02.457 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:02.455+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:02.553 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:02.553 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:02.553 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:02.553 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:02.553 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:02.560 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:02.558+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:02.564 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:30:02.568 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-17T21:30:02.661 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:02.659+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:02.684 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:02.684 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:02.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:02.685 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:02.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.685 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5493 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.685 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571177+0000", 2024-09-17T21:30:02.685 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.110498034999999, 2024-09-17T21:30:02.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043037758000000002, 2024-09-17T21:30:02.685 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.685 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.685 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.685 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.685 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.685 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.685 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5493 2024-09-17T21:30:02.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.686 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571177+0000", 2024-09-17T21:30:02.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571177+0000", 2024-09-17T21:30:02.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571178+0000", 2024-09-17T21:30:02.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8800000000000003e-07 2024-09-17T21:30:02.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571180+0000", 2024-09-17T21:30:02.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.17e-06 2024-09-17T21:30:02.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571187+0000", 2024-09-17T21:30:02.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.345e-06 2024-09-17T21:30:02.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571192+0000", 2024-09-17T21:30:02.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8300000000000003e-06 2024-09-17T21:30:02.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.613913+0000", 2024-09-17T21:30:02.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042721071999999999 2024-09-17T21:30:02.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614000+0000", 2024-09-17T21:30:02.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7027000000000006e-05 2024-09-17T21:30:02.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614215+0000", 2024-09-17T21:30:02.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482599999999999 2024-09-17T21:30:02.689 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.689 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.689 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.689 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5495 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.689 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571198+0000", 2024-09-17T21:30:02.689 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.110476826999999, 2024-09-17T21:30:02.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043378574000000003, 2024-09-17T21:30:02.689 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5495 2024-09-17T21:30:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571198+0000", 2024-09-17T21:30:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571198+0000", 2024-09-17T21:30:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571199+0000", 2024-09-17T21:30:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:30:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571200+0000", 2024-09-17T21:30:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7120000000000001e-06 2024-09-17T21:30:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571207+0000", 2024-09-17T21:30:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7159999999999997e-06 2024-09-17T21:30:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571212+0000", 2024-09-17T21:30:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5410000000000002e-06 2024-09-17T21:30:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614255+0000", 2024-09-17T21:30:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043043658999999998 2024-09-17T21:30:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614330+0000", 2024-09-17T21:30:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4640000000000004e-05 2024-09-17T21:30:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614577+0000", 2024-09-17T21:30:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024695400000000002 2024-09-17T21:30:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5497 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 329728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571218+0000", 2024-09-17T21:30:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.110457466, 2024-09-17T21:30:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043663621999999999, 2024-09-17T21:30:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5497 2024-09-17T21:30:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571218+0000", 2024-09-17T21:30:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571218+0000", 2024-09-17T21:30:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571218+0000", 2024-09-17T21:30:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6199999999999999e-07 2024-09-17T21:30:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571220+0000", 2024-09-17T21:30:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.655e-06 2024-09-17T21:30:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571227+0000", 2024-09-17T21:30:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9829999999999999e-06 2024-09-17T21:30:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571231+0000", 2024-09-17T21:30:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5310000000000003e-06 2024-09-17T21:30:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614621+0000", 2024-09-17T21:30:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043389469 2024-09-17T21:30:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614699+0000", 2024-09-17T21:30:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8490000000000002e-05 2024-09-17T21:30:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614881+0000", 2024-09-17T21:30:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000182132 2024-09-17T21:30:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5499 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 342016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571237+0000", 2024-09-17T21:30:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.110437866, 2024-09-17T21:30:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043955813000000003, 2024-09-17T21:30:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5499 2024-09-17T21:30:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571237+0000", 2024-09-17T21:30:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571237+0000", 2024-09-17T21:30:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571238+0000", 2024-09-17T21:30:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:30:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571239+0000", 2024-09-17T21:30:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:30:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571246+0000", 2024-09-17T21:30:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8390000000000001e-06 2024-09-17T21:30:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571251+0000", 2024-09-17T21:30:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4349999999999999e-06 2024-09-17T21:30:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614924+0000", 2024-09-17T21:30:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043673099999999999 2024-09-17T21:30:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614983+0000", 2024-09-17T21:30:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8794e-05 2024-09-17T21:30:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615193+0000", 2024-09-17T21:30:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000210545 2024-09-17T21:30:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5501 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 356352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571257+0000", 2024-09-17T21:30:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.110418633, 2024-09-17T21:30:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044214865999999999, 2024-09-17T21:30:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5501 2024-09-17T21:30:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571257+0000", 2024-09-17T21:30:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571257+0000", 2024-09-17T21:30:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571257+0000", 2024-09-17T21:30:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:30:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571259+0000", 2024-09-17T21:30:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9460000000000001e-06 2024-09-17T21:30:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571266+0000", 2024-09-17T21:30:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0439999999999998e-06 2024-09-17T21:30:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571271+0000", 2024-09-17T21:30:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6500000000000004e-06 2024-09-17T21:30:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615238+0000", 2024-09-17T21:30:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043967570999999997 2024-09-17T21:30:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615293+0000", 2024-09-17T21:30:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4857e-05 2024-09-17T21:30:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615471+0000", 2024-09-17T21:30:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00017844900000000001 2024-09-17T21:30:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5503 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 370688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571276+0000", 2024-09-17T21:30:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.110398754, 2024-09-17T21:30:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044418055999999997, 2024-09-17T21:30:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5503 2024-09-17T21:30:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571276+0000", 2024-09-17T21:30:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571276+0000", 2024-09-17T21:30:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571277+0000", 2024-09-17T21:30:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.46e-07 2024-09-17T21:30:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571286+0000", 2024-09-17T21:30:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8810000000000002e-06 2024-09-17T21:30:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571293+0000", 2024-09-17T21:30:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5499999999999997e-06 2024-09-17T21:30:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571303+0000", 2024-09-17T21:30:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5279999999999992e-06 2024-09-17T21:30:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615514+0000", 2024-09-17T21:30:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044211617000000002 2024-09-17T21:30:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615593+0000", 2024-09-17T21:30:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8768000000000002e-05 2024-09-17T21:30:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615694+0000", 2024-09-17T21:30:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101366 2024-09-17T21:30:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5548 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 194560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571310+0000", 2024-09-17T21:30:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.110365666, 2024-09-17T21:30:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044804920999999998, 2024-09-17T21:30:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5548 2024-09-17T21:30:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571310+0000", 2024-09-17T21:30:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571310+0000", 2024-09-17T21:30:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571310+0000", 2024-09-17T21:30:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8799999999999998e-07 2024-09-17T21:30:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571312+0000", 2024-09-17T21:30:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8920000000000001e-06 2024-09-17T21:30:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571319+0000", 2024-09-17T21:30:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9360000000000002e-06 2024-09-17T21:30:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571323+0000", 2024-09-17T21:30:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6739999999999996e-06 2024-09-17T21:30:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615725+0000", 2024-09-17T21:30:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044401949000000003 2024-09-17T21:30:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615801+0000", 2024-09-17T21:30:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5150999999999999e-05 2024-09-17T21:30:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616114+0000", 2024-09-17T21:30:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00031393099999999999 2024-09-17T21:30:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5551 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 208896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571329+0000", 2024-09-17T21:30:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.110345745, 2024-09-17T21:30:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045083356999999998, 2024-09-17T21:30:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5551 2024-09-17T21:30:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571329+0000", 2024-09-17T21:30:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571329+0000", 2024-09-17T21:30:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571330+0000", 2024-09-17T21:30:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5499999999999999e-07 2024-09-17T21:30:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571331+0000", 2024-09-17T21:30:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.691e-06 2024-09-17T21:30:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571338+0000", 2024-09-17T21:30:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-17T21:30:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571343+0000", 2024-09-17T21:30:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6260000000000003e-06 2024-09-17T21:30:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616174+0000", 2024-09-17T21:30:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044831322 2024-09-17T21:30:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616224+0000", 2024-09-17T21:30:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9950000000000001e-05 2024-09-17T21:30:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616413+0000", 2024-09-17T21:30:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018850400000000001 2024-09-17T21:30:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5553 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571349+0000", 2024-09-17T21:30:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.110326215000001, 2024-09-17T21:30:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045359177000000001, 2024-09-17T21:30:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5553 2024-09-17T21:30:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571349+0000", 2024-09-17T21:30:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571349+0000", 2024-09-17T21:30:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571349+0000", 2024-09-17T21:30:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.15e-07 2024-09-17T21:30:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571351+0000", 2024-09-17T21:30:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6750000000000001e-06 2024-09-17T21:30:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571360+0000", 2024-09-17T21:30:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6270000000000006e-06 2024-09-17T21:30:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571364+0000", 2024-09-17T21:30:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7029999999999997e-06 2024-09-17T21:30:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616459+0000", 2024-09-17T21:30:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045094672000000002 2024-09-17T21:30:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616503+0000", 2024-09-17T21:30:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4032000000000001e-05 2024-09-17T21:30:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616708+0000", 2024-09-17T21:30:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020515299999999999 2024-09-17T21:30:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5555 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 235520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571371+0000", 2024-09-17T21:30:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.110304635, 2024-09-17T21:30:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045673013999999998, 2024-09-17T21:30:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5555 2024-09-17T21:30:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571371+0000", 2024-09-17T21:30:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571371+0000", 2024-09-17T21:30:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571371+0000", 2024-09-17T21:30:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9400000000000001e-07 2024-09-17T21:30:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571373+0000", 2024-09-17T21:30:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.714e-06 2024-09-17T21:30:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571379+0000", 2024-09-17T21:30:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0449999999999997e-06 2024-09-17T21:30:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571383+0000", 2024-09-17T21:30:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.963e-06 2024-09-17T21:30:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616752+0000", 2024-09-17T21:30:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045369481000000003 2024-09-17T21:30:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616828+0000", 2024-09-17T21:30:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5961e-05 2024-09-17T21:30:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617044+0000", 2024-09-17T21:30:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021545599999999999 2024-09-17T21:30:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5556 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571392+0000", 2024-09-17T21:30:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.110282776, 2024-09-17T21:30:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045971310000000001, 2024-09-17T21:30:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5556 2024-09-17T21:30:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571392+0000", 2024-09-17T21:30:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571392+0000", 2024-09-17T21:30:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571393+0000", 2024-09-17T21:30:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1399999999999997e-07 2024-09-17T21:30:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571395+0000", 2024-09-17T21:30:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.956e-06 2024-09-17T21:30:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571401+0000", 2024-09-17T21:30:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9089999999999997e-06 2024-09-17T21:30:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571405+0000", 2024-09-17T21:30:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6410000000000001e-06 2024-09-17T21:30:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617070+0000", 2024-09-17T21:30:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045664874000000001 2024-09-17T21:30:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617130+0000", 2024-09-17T21:30:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9756999999999997e-05 2024-09-17T21:30:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617364+0000", 2024-09-17T21:30:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000233759 2024-09-17T21:30:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5559 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 264192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571411+0000", 2024-09-17T21:30:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.110263766999999, 2024-09-17T21:30:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046332908999999999, 2024-09-17T21:30:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5559 2024-09-17T21:30:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571411+0000", 2024-09-17T21:30:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571411+0000", 2024-09-17T21:30:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571412+0000", 2024-09-17T21:30:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.65e-07 2024-09-17T21:30:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571419+0000", 2024-09-17T21:30:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2899999999999997e-06 2024-09-17T21:30:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571425+0000", 2024-09-17T21:30:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3400000000000003e-06 2024-09-17T21:30:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571430+0000", 2024-09-17T21:30:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4499999999999997e-06 2024-09-17T21:30:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617455+0000", 2024-09-17T21:30:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046024661000000001 2024-09-17T21:30:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617532+0000", 2024-09-17T21:30:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7712000000000004e-05 2024-09-17T21:30:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617744+0000", 2024-09-17T21:30:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000212091 2024-09-17T21:30:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5561 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 276480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571436+0000", 2024-09-17T21:30:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.110238978, 2024-09-17T21:30:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046615325999999999, 2024-09-17T21:30:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5561 2024-09-17T21:30:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571436+0000", 2024-09-17T21:30:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571436+0000", 2024-09-17T21:30:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571437+0000", 2024-09-17T21:30:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:30:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571439+0000", 2024-09-17T21:30:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9130000000000001e-06 2024-09-17T21:30:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571444+0000", 2024-09-17T21:30:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5110000000000003e-06 2024-09-17T21:30:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571448+0000", 2024-09-17T21:30:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3749999999999996e-06 2024-09-17T21:30:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617775+0000", 2024-09-17T21:30:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046326668000000001 2024-09-17T21:30:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617851+0000", 2024-09-17T21:30:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6192000000000005e-05 2024-09-17T21:30:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618052+0000", 2024-09-17T21:30:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020022900000000001 2024-09-17T21:30:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5563 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 290816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571454+0000", 2024-09-17T21:30:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.110220700999999, 2024-09-17T21:30:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046975959999999997, 2024-09-17T21:30:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5563 2024-09-17T21:30:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571454+0000", 2024-09-17T21:30:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571454+0000", 2024-09-17T21:30:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571455+0000", 2024-09-17T21:30:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:30:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571456+0000", 2024-09-17T21:30:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.64e-06 2024-09-17T21:30:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571462+0000", 2024-09-17T21:30:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6230000000000002e-06 2024-09-17T21:30:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571467+0000", 2024-09-17T21:30:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5199999999999999e-06 2024-09-17T21:30:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618091+0000", 2024-09-17T21:30:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046624489999999998 2024-09-17T21:30:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618172+0000", 2024-09-17T21:30:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1271000000000002e-05 2024-09-17T21:30:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618430+0000", 2024-09-17T21:30:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025807199999999998 2024-09-17T21:30:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5566 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571472+0000", 2024-09-17T21:30:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.110202808, 2024-09-17T21:30:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047328261000000003, 2024-09-17T21:30:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5566 2024-09-17T21:30:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571472+0000", 2024-09-17T21:30:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571472+0000", 2024-09-17T21:30:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571473+0000", 2024-09-17T21:30:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:30:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571474+0000", 2024-09-17T21:30:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.668e-06 2024-09-17T21:30:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571480+0000", 2024-09-17T21:30:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6899999999999997e-06 2024-09-17T21:30:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571489+0000", 2024-09-17T21:30:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8589999999999994e-06 2024-09-17T21:30:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618481+0000", 2024-09-17T21:30:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046992468000000003 2024-09-17T21:30:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618563+0000", 2024-09-17T21:30:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1464000000000005e-05 2024-09-17T21:30:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618801+0000", 2024-09-17T21:30:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000237762 2024-09-17T21:30:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5568 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 317440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571495+0000", 2024-09-17T21:30:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.110180321, 2024-09-17T21:30:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047753012999999997, 2024-09-17T21:30:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5568 2024-09-17T21:30:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571495+0000", 2024-09-17T21:30:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571495+0000", 2024-09-17T21:30:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571495+0000", 2024-09-17T21:30:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.89e-07 2024-09-17T21:30:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571497+0000", 2024-09-17T21:30:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.739e-06 2024-09-17T21:30:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571503+0000", 2024-09-17T21:30:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:30:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571507+0000", 2024-09-17T21:30:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3560000000000003e-06 2024-09-17T21:30:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618877+0000", 2024-09-17T21:30:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047370019999999999 2024-09-17T21:30:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618964+0000", 2024-09-17T21:30:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6615999999999999e-05 2024-09-17T21:30:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619248+0000", 2024-09-17T21:30:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000284154 2024-09-17T21:30:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5570 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 331776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571513+0000", 2024-09-17T21:30:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.110162459, 2024-09-17T21:30:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048061883999999999, 2024-09-17T21:30:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5570 2024-09-17T21:30:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571513+0000", 2024-09-17T21:30:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571513+0000", 2024-09-17T21:30:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571513+0000", 2024-09-17T21:30:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4700000000000002e-07 2024-09-17T21:30:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571518+0000", 2024-09-17T21:30:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.242e-06 2024-09-17T21:30:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571874+0000", 2024-09-17T21:30:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00035530799999999998 2024-09-17T21:30:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571884+0000", 2024-09-17T21:30:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0733e-05 2024-09-17T21:30:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619283+0000", 2024-09-17T21:30:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047398218999999998 2024-09-17T21:30:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619361+0000", 2024-09-17T21:30:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8660999999999998e-05 2024-09-17T21:30:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619575+0000", 2024-09-17T21:30:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021337400000000001 2024-09-17T21:30:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5572 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 346112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571897+0000", 2024-09-17T21:30:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.109778508, 2024-09-17T21:30:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048019328999999999, 2024-09-17T21:30:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5572 2024-09-17T21:30:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571897+0000", 2024-09-17T21:30:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571897+0000", 2024-09-17T21:30:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571897+0000", 2024-09-17T21:30:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8100000000000002e-07 2024-09-17T21:30:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571901+0000", 2024-09-17T21:30:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4369999999999999e-06 2024-09-17T21:30:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571909+0000", 2024-09-17T21:30:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.841e-06 2024-09-17T21:30:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571915+0000", 2024-09-17T21:30:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5130000000000002e-06 2024-09-17T21:30:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619619+0000", 2024-09-17T21:30:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047704203000000001 2024-09-17T21:30:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619698+0000", 2024-09-17T21:30:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8656e-05 2024-09-17T21:30:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619916+0000", 2024-09-17T21:30:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021789800000000001 2024-09-17T21:30:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5574 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 358400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571923+0000", 2024-09-17T21:30:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.109752478000001, 2024-09-17T21:30:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048336208999999998, 2024-09-17T21:30:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5574 2024-09-17T21:30:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571923+0000", 2024-09-17T21:30:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571923+0000", 2024-09-17T21:30:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571923+0000", 2024-09-17T21:30:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9900000000000001e-07 2024-09-17T21:30:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571925+0000", 2024-09-17T21:30:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.978e-06 2024-09-17T21:30:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571931+0000", 2024-09-17T21:30:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2530000000000001e-06 2024-09-17T21:30:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571936+0000", 2024-09-17T21:30:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6700000000000002e-06 2024-09-17T21:30:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619957+0000", 2024-09-17T21:30:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048020500000000001 2024-09-17T21:30:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.620037+0000", 2024-09-17T21:30:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0016000000000001e-05 2024-09-17T21:30:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.620259+0000", 2024-09-17T21:30:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022239300000000001 2024-09-17T21:30:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5576 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571944+0000", 2024-09-17T21:30:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.109731352000001, 2024-09-17T21:30:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048513909000000001, 2024-09-17T21:30:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5576 2024-09-17T21:30:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571944+0000", 2024-09-17T21:30:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571944+0000", 2024-09-17T21:30:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571952+0000", 2024-09-17T21:30:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7729999999999999e-06 2024-09-17T21:30:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571954+0000", 2024-09-17T21:30:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8310000000000001e-06 2024-09-17T21:30:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571964+0000", 2024-09-17T21:30:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4450000000000002e-06 2024-09-17T21:30:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571976+0000", 2024-09-17T21:30:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1793999999999999e-05 2024-09-17T21:30:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.620299+0000", 2024-09-17T21:30:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048323558000000003 2024-09-17T21:30:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.620373+0000", 2024-09-17T21:30:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4262000000000002e-05 2024-09-17T21:30:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.620458+0000", 2024-09-17T21:30:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4246000000000006e-05 2024-09-17T21:30:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:02.775 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:02.777 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:30:02.777 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:02.774+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:30:02.828 INFO:teuthology.orchestra.run.smithi129.stdout:1484 write 0xa7197e8 thru 0xa71a383 (0xb9c bytes) 2024-09-17T21:30:02.842 INFO:teuthology.orchestra.run.smithi129.stdout:1485 write 0xe296ab3 thru 0xe299077 (0x25c5 bytes) 2024-09-17T21:30:02.849 INFO:teuthology.orchestra.run.smithi129.stdout:1486 writesame 0x81cf963 thru 0x81d4827 data_size 0x25(0x4ec5 bytes) 2024-09-17T21:30:02.872 INFO:teuthology.orchestra.run.smithi129.stdout:1487 trunc from 0xe299078 to 0x46abef4 2024-09-17T21:30:02.877 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:02.875+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:02.896 INFO:teuthology.orchestra.run.smithi129.stdout:1488 write 0x881d7d thru 0x884398 (0x261c bytes) 2024-09-17T21:30:02.925 INFO:teuthology.orchestra.run.smithi129.stdout:1490 writesame 0x28424ea thru 0x28518a7 data_size 0x7(0xf3be bytes) 2024-09-17T21:30:02.944 INFO:teuthology.orchestra.run.smithi129.stdout:1492 read 0x28c371 thru 0x29417c (0x7e0c bytes) 2024-09-17T21:30:02.949 INFO:teuthology.orchestra.run.smithi129.stdout:1493 trunc from 0x46abef4 to 0x1b20533 2024-09-17T21:30:02.949 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:02.949 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:02.949 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:02.949 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:02.962 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:30:03.002 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:03.000+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:03.004 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:03.004 INFO:teuthology.orchestra.run.smithi129.stdout:1496 read 0x1492e29 thru 0x149d6ed (0xa8c5 bytes) 2024-09-17T21:30:03.018 INFO:teuthology.orchestra.run.smithi129.stdout:1497 writesame 0x8129e41 thru 0x812d0dd data_size 0x7(0x329d bytes) 2024-09-17T21:30:03.018 INFO:teuthology.orchestra.run.smithi129.stdout:1498 punch from 0x7459689 to 0x745e08c, (0x4a03 bytes) 2024-09-17T21:30:03.018 INFO:teuthology.orchestra.run.smithi129.stdout:1499 write 0x3c9cf33 thru 0x3ca7dc0 (0xae8e bytes) 2024-09-17T21:30:03.038 INFO:teuthology.orchestra.run.smithi129.stdout:1500 write 0xc8c6d06 thru 0xc8cebbc (0x7eb7 bytes) 2024-09-17T21:30:03.044 INFO:teuthology.orchestra.run.smithi129.stdout:1501 write 0xbeb40d6 thru 0xbeb9d25 (0x5c50 bytes) 2024-09-17T21:30:03.049 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:03.049 INFO:teuthology.orchestra.run.smithi129.stdout:1504 punch from 0x4349d86 to 0x4356949, (0xcbc3 bytes) 2024-09-17T21:30:03.049 INFO:teuthology.orchestra.run.smithi129.stdout:1505 punch from 0xa2f9f25 to 0xa2fde08, (0x3ee3 bytes) 2024-09-17T21:30:03.056 INFO:teuthology.orchestra.run.smithi129.stdout:1506 writesame 0xdde6052 thru 0xddf436c data_size 0x1(0xe31b bytes) 2024-09-17T21:30:03.056 INFO:teuthology.orchestra.run.smithi129.stdout:1507 write 0x841062b thru 0x841655f (0x5f35 bytes) 2024-09-17T21:30:03.061 INFO:teuthology.orchestra.run.smithi129.stdout:1508 write 0xbd08339 thru 0xbd10580 (0x8248 bytes) 2024-09-17T21:30:03.070 INFO:teuthology.orchestra.run.smithi129.stdout:1509 punch from 0x387925a to 0x387bc52, (0x29f8 bytes) 2024-09-17T21:30:03.070 INFO:teuthology.orchestra.run.smithi129.stdout:1510 write 0x1bd572d thru 0x1be16e6 (0xbfba bytes) 2024-09-17T21:30:03.076 INFO:teuthology.orchestra.run.smithi129.stdout:1511 read 0x262bd18 thru 0x26333e6 (0x76cf bytes) 2024-09-17T21:30:03.076 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:03.076 INFO:teuthology.orchestra.run.smithi129.stdout:1513 punch from 0xae16ade to 0xae1d7bb, (0x6cdd bytes) 2024-09-17T21:30:03.076 INFO:teuthology.orchestra.run.smithi129.stdout:1514 read 0x60e7dc9 thru 0x60f2b64 (0xad9c bytes) 2024-09-17T21:30:03.076 INFO:teuthology.orchestra.run.smithi129.stdout:1515 write 0x5692cef thru 0x569decd (0xb1df bytes) 2024-09-17T21:30:03.079 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:03.077+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:03.087 INFO:teuthology.orchestra.run.smithi129.stdout:1516 read 0xbd3cbaa thru 0xbd3cdfd (0x254 bytes) 2024-09-17T21:30:03.087 INFO:teuthology.orchestra.run.smithi129.stdout:1517 write 0xd095f0d thru 0xd0a1c5f (0xbd53 bytes) 2024-09-17T21:30:03.098 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:03.098 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:03.098 INFO:teuthology.orchestra.run.smithi129.stdout:1522 write 0x7a359f6 thru 0x7a36c51 (0x125c bytes) 2024-09-17T21:30:03.103 INFO:teuthology.orchestra.run.smithi129.stdout:1523 read 0xb2ee08c thru 0xb2f29b7 (0x492c bytes) 2024-09-17T21:30:03.103 INFO:teuthology.orchestra.run.smithi129.stdout:1524 write 0x8cd2e99 thru 0x8ce134f (0xe4b7 bytes) 2024-09-17T21:30:03.112 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:03.113 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:03.113 INFO:teuthology.orchestra.run.smithi129.stdout:1527 writesame 0x31ab91b thru 0x31ac2cd data_size 0x1(0x9b3 bytes) 2024-09-17T21:30:03.116 INFO:teuthology.orchestra.run.smithi129.stdout:1530 writesame 0x48a226a thru 0x48a38bd data_size 0x2(0x1654 bytes) 2024-09-17T21:30:03.121 INFO:teuthology.orchestra.run.smithi129.stdout:1533 clone 10 order 22 su 2097152 sc 10 2024-09-17T21:30:03.128 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:03.128 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:03.128 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:03.128 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:03.128 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:03.137 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:30:03.180 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:03.178+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:03.281 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:03.279+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:03.282 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:14839 3.9 e31/25 3:96c42965:::rbd_data.11207744cc2c.0000000000000062:head v 31'6, mlcod=30'4)", 2024-09-17T21:30:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.297020+0000", 2024-09-17T21:30:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.981986488, 2024-09-17T21:30:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0014376460000000001, 2024-09-17T21:30:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297020+0000", 2024-09-17T21:30:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297020+0000", 2024-09-17T21:30:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297021+0000", 2024-09-17T21:30:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2300000000000001e-06 2024-09-17T21:30:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297217+0000", 2024-09-17T21:30:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019523500000000001 2024-09-17T21:30:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297233+0000", 2024-09-17T21:30:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6055000000000001e-05 2024-09-17T21:30:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297242+0000", 2024-09-17T21:30:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0939999999999997e-06 2024-09-17T21:30:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297294+0000", 2024-09-17T21:30:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2456000000000001e-05 2024-09-17T21:30:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297308+0000", 2024-09-17T21:30:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4051e-05 2024-09-17T21:30:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.298407+0000", 2024-09-17T21:30:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001098829 2024-09-17T21:30:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.298458+0000", 2024-09-17T21:30:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0695999999999997e-05 2024-09-17T21:30:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:14841 3.9 e31/25 3:96c42965:::rbd_data.11207744cc2c.0000000000000062:head v 31'8, mlcod=30'4)", 2024-09-17T21:30:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.297265+0000", 2024-09-17T21:30:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.981741354, 2024-09-17T21:30:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0016100859999999999, 2024-09-17T21:30:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297265+0000", 2024-09-17T21:30:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297265+0000", 2024-09-17T21:30:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297266+0000", 2024-09-17T21:30:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7e-07 2024-09-17T21:30:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297278+0000", 2024-09-17T21:30:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2259000000000001e-05 2024-09-17T21:30:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297285+0000", 2024-09-17T21:30:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1049999999999997e-06 2024-09-17T21:30:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297291+0000", 2024-09-17T21:30:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3029999999999999e-06 2024-09-17T21:30:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297823+0000", 2024-09-17T21:30:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053223799999999998 2024-09-17T21:30:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.297847+0000", 2024-09-17T21:30:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4063e-05 2024-09-17T21:30:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.298833+0000", 2024-09-17T21:30:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00098568500000000003 2024-09-17T21:30:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.298875+0000", 2024-09-17T21:30:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2963000000000002e-05 2024-09-17T21:30:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:20796 3.4 e32/25 3:219b1041:::rbd_data.11207744cc2c.00000000000000fc:head v 32'5, mlcod=29'3)", 2024-09-17T21:30:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.644732+0000", 2024-09-17T21:30:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.634274346, 2024-09-17T21:30:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002629045, 2024-09-17T21:30:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.644732+0000", 2024-09-17T21:30:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.644732+0000", 2024-09-17T21:30:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.644733+0000", 2024-09-17T21:30:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.27e-07 2024-09-17T21:30:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.644757+0000", 2024-09-17T21:30:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4097999999999999e-05 2024-09-17T21:30:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.644770+0000", 2024-09-17T21:30:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3275000000000001e-05 2024-09-17T21:30:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.644779+0000", 2024-09-17T21:30:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8950000000000004e-06 2024-09-17T21:30:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.644844+0000", 2024-09-17T21:30:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5087999999999998e-05 2024-09-17T21:30:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.644878+0000", 2024-09-17T21:30:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4214999999999997e-05 2024-09-17T21:30:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.647315+0000", 2024-09-17T21:30:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0024362149999999998 2024-09-17T21:30:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.647361+0000", 2024-09-17T21:30:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6931999999999997e-05 2024-09-17T21:30:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:20970 3.9 e32/25 3:9272546e:::rbd_data.11207744cc2c.00000000000000fe:head v 32'9, mlcod=31'6)", 2024-09-17T21:30:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.653977+0000", 2024-09-17T21:30:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.625029796, 2024-09-17T21:30:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001730353, 2024-09-17T21:30:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.653977+0000", 2024-09-17T21:30:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.653977+0000", 2024-09-17T21:30:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.653977+0000", 2024-09-17T21:30:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:30:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.654039+0000", 2024-09-17T21:30:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1467e-05 2024-09-17T21:30:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.654047+0000", 2024-09-17T21:30:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1259999999999998e-06 2024-09-17T21:30:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.654056+0000", 2024-09-17T21:30:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8999999999999995e-06 2024-09-17T21:30:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.654107+0000", 2024-09-17T21:30:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1561000000000003e-05 2024-09-17T21:30:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.654131+0000", 2024-09-17T21:30:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3516999999999999e-05 2024-09-17T21:30:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.655647+0000", 2024-09-17T21:30:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015161720000000001 2024-09-17T21:30:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.655707+0000", 2024-09-17T21:30:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0238999999999997e-05 2024-09-17T21:30:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:34521 3.4 e34/25 3:24c07f74:::rbd_object_map.112071f51ee8:head v 34'23, mlcod=34'21)", 2024-09-17T21:30:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:52.552854+0000", 2024-09-17T21:30:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.726152399, 2024-09-17T21:30:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0023850989999999999, 2024-09-17T21:30:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.552854+0000", 2024-09-17T21:30:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.552854+0000", 2024-09-17T21:30:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.552855+0000", 2024-09-17T21:30:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1149999999999999e-06 2024-09-17T21:30:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.552866+0000", 2024-09-17T21:30:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0934000000000001e-05 2024-09-17T21:30:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.552881+0000", 2024-09-17T21:30:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4428e-05 2024-09-17T21:30:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.552889+0000", 2024-09-17T21:30:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8590000000000005e-06 2024-09-17T21:30:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.552940+0000", 2024-09-17T21:30:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0992000000000002e-05 2024-09-17T21:30:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.552956+0000", 2024-09-17T21:30:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6206999999999999e-05 2024-09-17T21:30:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.555191+0000", 2024-09-17T21:30:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022347360000000002 2024-09-17T21:30:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.555239+0000", 2024-09-17T21:30:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8828000000000001e-05 2024-09-17T21:30:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:34524 3.4 e34/25 3:24c07f74:::rbd_object_map.112071f51ee8:head v 34'24, mlcod=34'22)", 2024-09-17T21:30:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:52.557952+0000", 2024-09-17T21:30:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.721054817000001, 2024-09-17T21:30:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015876969999999999, 2024-09-17T21:30:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.557952+0000", 2024-09-17T21:30:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.557952+0000", 2024-09-17T21:30:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.557953+0000", 2024-09-17T21:30:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.13e-06 2024-09-17T21:30:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.557960+0000", 2024-09-17T21:30:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3119999999999996e-06 2024-09-17T21:30:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.557973+0000", 2024-09-17T21:30:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2999999999999999e-05 2024-09-17T21:30:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.557980+0000", 2024-09-17T21:30:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0079999999999996e-06 2024-09-17T21:30:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.558034+0000", 2024-09-17T21:30:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3528e-05 2024-09-17T21:30:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.558054+0000", 2024-09-17T21:30:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0290999999999999e-05 2024-09-17T21:30:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.559508+0000", 2024-09-17T21:30:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0014540359999999999 2024-09-17T21:30:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.559540+0000", 2024-09-17T21:30:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1392e-05 2024-09-17T21:30:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:34547 3.4 e34/25 3:24c07f74:::rbd_object_map.112071f51ee8:head v 34'29, mlcod=34'27)", 2024-09-17T21:30:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:52.585792+0000", 2024-09-17T21:30:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.693214434, 2024-09-17T21:30:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0032798810000000001, 2024-09-17T21:30:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.585792+0000", 2024-09-17T21:30:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.585792+0000", 2024-09-17T21:30:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.585793+0000", 2024-09-17T21:30:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8899999999999998e-07 2024-09-17T21:30:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.585799+0000", 2024-09-17T21:30:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2929999999999997e-06 2024-09-17T21:30:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.585812+0000", 2024-09-17T21:30:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2276000000000001e-05 2024-09-17T21:30:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.585819+0000", 2024-09-17T21:30:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9979999999999997e-06 2024-09-17T21:30:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.585868+0000", 2024-09-17T21:30:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9750999999999999e-05 2024-09-17T21:30:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.585891+0000", 2024-09-17T21:30:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2149000000000001e-05 2024-09-17T21:30:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.589040+0000", 2024-09-17T21:30:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0031496929999999999 2024-09-17T21:30:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.589072+0000", 2024-09-17T21:30:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1931999999999999e-05 2024-09-17T21:30:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:34564 3.4 e34/25 3:24c07f74:::rbd_object_map.112071f51ee8:head v 34'34, mlcod=34'31)", 2024-09-17T21:30:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:52.606446+0000", 2024-09-17T21:30:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.672560679, 2024-09-17T21:30:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002569965, 2024-09-17T21:30:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.606446+0000", 2024-09-17T21:30:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.606446+0000", 2024-09-17T21:30:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.606447+0000", 2024-09-17T21:30:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.75e-07 2024-09-17T21:30:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.606451+0000", 2024-09-17T21:30:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6789999999999996e-06 2024-09-17T21:30:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.606459+0000", 2024-09-17T21:30:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1759999999999993e-06 2024-09-17T21:30:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.606464+0000", 2024-09-17T21:30:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:30:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.606505+0000", 2024-09-17T21:30:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0720000000000003e-05 2024-09-17T21:30:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.606518+0000", 2024-09-17T21:30:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2870999999999999e-05 2024-09-17T21:30:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.608964+0000", 2024-09-17T21:30:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0024465440000000001 2024-09-17T21:30:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.609016+0000", 2024-09-17T21:30:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1822e-05 2024-09-17T21:30:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:34637 3.4 e34/25 3:24c07f74:::rbd_object_map.112071f51ee8:head v 34'47, mlcod=34'44)", 2024-09-17T21:30:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:52.670033+0000", 2024-09-17T21:30:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.608973894, 2024-09-17T21:30:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0027378770000000001, 2024-09-17T21:30:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.670033+0000", 2024-09-17T21:30:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.670033+0000", 2024-09-17T21:30:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.670033+0000", 2024-09-17T21:30:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9299999999999998e-07 2024-09-17T21:30:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.670038+0000", 2024-09-17T21:30:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.882e-06 2024-09-17T21:30:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.670049+0000", 2024-09-17T21:30:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0208e-05 2024-09-17T21:30:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.670055+0000", 2024-09-17T21:30:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1809999999999998e-06 2024-09-17T21:30:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.670097+0000", 2024-09-17T21:30:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1998e-05 2024-09-17T21:30:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.670115+0000", 2024-09-17T21:30:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8192000000000001e-05 2024-09-17T21:30:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.672723+0000", 2024-09-17T21:30:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0026077180000000002 2024-09-17T21:30:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.672771+0000", 2024-09-17T21:30:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8105000000000001e-05 2024-09-17T21:30:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:34638 3.4 e34/25 3:24c07f74:::rbd_object_map.112071f51ee8:head v 34'48, mlcod=34'44)", 2024-09-17T21:30:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:52.670803+0000", 2024-09-17T21:30:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.608203657000001, 2024-09-17T21:30:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022839100000000001, 2024-09-17T21:30:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.670803+0000", 2024-09-17T21:30:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.670803+0000", 2024-09-17T21:30:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.670804+0000", 2024-09-17T21:30:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0100000000000005e-07 2024-09-17T21:30:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.670810+0000", 2024-09-17T21:30:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0519999999999998e-06 2024-09-17T21:30:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.670820+0000", 2024-09-17T21:30:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8479999999999996e-06 2024-09-17T21:30:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.670826+0000", 2024-09-17T21:30:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.049e-06 2024-09-17T21:30:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.670876+0000", 2024-09-17T21:30:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0284000000000002e-05 2024-09-17T21:30:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.670894+0000", 2024-09-17T21:30:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8434e-05 2024-09-17T21:30:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.673061+0000", 2024-09-17T21:30:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002167048 2024-09-17T21:30:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.673087+0000", 2024-09-17T21:30:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5593999999999999e-05 2024-09-17T21:30:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:34639 3.4 e34/25 3:24c07f74:::rbd_object_map.112071f51ee8:head v 34'49, mlcod=34'44)", 2024-09-17T21:30:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:52.671381+0000", 2024-09-17T21:30:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.607625577, 2024-09-17T21:30:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0017213490000000001, 2024-09-17T21:30:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.331 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.331 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.331 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.671381+0000", 2024-09-17T21:30:03.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.671381+0000", 2024-09-17T21:30:03.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.671382+0000", 2024-09-17T21:30:03.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7999999999999995e-07 2024-09-17T21:30:03.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.671386+0000", 2024-09-17T21:30:03.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5940000000000004e-06 2024-09-17T21:30:03.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.671396+0000", 2024-09-17T21:30:03.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3640000000000005e-06 2024-09-17T21:30:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.671404+0000", 2024-09-17T21:30:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7530000000000007e-06 2024-09-17T21:30:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.671443+0000", 2024-09-17T21:30:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8260999999999998e-05 2024-09-17T21:30:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.671462+0000", 2024-09-17T21:30:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9026999999999999e-05 2024-09-17T21:30:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.673089+0000", 2024-09-17T21:30:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0016272509999999999 2024-09-17T21:30:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.673103+0000", 2024-09-17T21:30:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3519e-05 2024-09-17T21:30:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:34777 3.4 e34/25 3:24c07f74:::rbd_object_map.112071f51ee8:head v 34'72, mlcod=34'70)", 2024-09-17T21:30:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:52.802976+0000", 2024-09-17T21:30:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.476030467999999, 2024-09-17T21:30:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002293228, 2024-09-17T21:30:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.802976+0000", 2024-09-17T21:30:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.802976+0000", 2024-09-17T21:30:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.802977+0000", 2024-09-17T21:30:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0500000000000002e-07 2024-09-17T21:30:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.802983+0000", 2024-09-17T21:30:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2380000000000002e-06 2024-09-17T21:30:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.802993+0000", 2024-09-17T21:30:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6709999999999994e-06 2024-09-17T21:30:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.802998+0000", 2024-09-17T21:30:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2920000000000003e-06 2024-09-17T21:30:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.803040+0000", 2024-09-17T21:30:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1569000000000001e-05 2024-09-17T21:30:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.803060+0000", 2024-09-17T21:30:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0154999999999999e-05 2024-09-17T21:30:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.805232+0000", 2024-09-17T21:30:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021721750000000001 2024-09-17T21:30:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.805269+0000", 2024-09-17T21:30:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7222999999999999e-05 2024-09-17T21:30:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:37873 3.0 e41/25 3:01af0035:::rbd_data.1120ab27bfd0.0000000000000074:head v 41'14, mlcod=36'12)", 2024-09-17T21:30:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:02.846191+0000", 2024-09-17T21:30:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.43281577399999999, 2024-09-17T21:30:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0023132550000000002, 2024-09-17T21:30:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.846191+0000", 2024-09-17T21:30:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.846191+0000", 2024-09-17T21:30:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.846191+0000", 2024-09-17T21:30:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9299999999999999e-07 2024-09-17T21:30:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.846197+0000", 2024-09-17T21:30:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6969999999999998e-06 2024-09-17T21:30:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.846210+0000", 2024-09-17T21:30:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2921999999999999e-05 2024-09-17T21:30:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.846216+0000", 2024-09-17T21:30:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1879999999999999e-06 2024-09-17T21:30:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.846272+0000", 2024-09-17T21:30:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5940000000000003e-05 2024-09-17T21:30:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.846295+0000", 2024-09-17T21:30:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3291000000000001e-05 2024-09-17T21:30:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.848470+0000", 2024-09-17T21:30:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021748430000000001 2024-09-17T21:30:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.848504+0000", 2024-09-17T21:30:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4180999999999997e-05 2024-09-17T21:30:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:37887 3.1f e41/37 3:fc790e8e:::rbd_data.1120ab27bfd0.0000000000000054:head v 41'27, mlcod=39'25)", 2024-09-17T21:30:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:02.873463+0000", 2024-09-17T21:30:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.40554387400000003, 2024-09-17T21:30:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0029774240000000002, 2024-09-17T21:30:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.873463+0000", 2024-09-17T21:30:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.873463+0000", 2024-09-17T21:30:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.873464+0000", 2024-09-17T21:30:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1399999999999995e-07 2024-09-17T21:30:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.873471+0000", 2024-09-17T21:30:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0310000000000001e-06 2024-09-17T21:30:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.873485+0000", 2024-09-17T21:30:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4552999999999999e-05 2024-09-17T21:30:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.873493+0000", 2024-09-17T21:30:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3039999999999999e-06 2024-09-17T21:30:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.873560+0000", 2024-09-17T21:30:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7136000000000002e-05 2024-09-17T21:30:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.873574+0000", 2024-09-17T21:30:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4459e-05 2024-09-17T21:30:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.876404+0000", 2024-09-17T21:30:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0028292870000000002 2024-09-17T21:30:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.876440+0000", 2024-09-17T21:30:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6739999999999997e-05 2024-09-17T21:30:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:38400 3.0 e41/25 3:050aa3f8:::rbd_data.1120ab27bfd0.0000000000000001:head v 41'16, mlcod=41'14)", 2024-09-17T21:30:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:02.918196+0000", 2024-09-17T21:30:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.360810466, 2024-09-17T21:30:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0028826490000000001, 2024-09-17T21:30:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.918196+0000", 2024-09-17T21:30:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.918196+0000", 2024-09-17T21:30:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.918198+0000", 2024-09-17T21:30:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.928e-06 2024-09-17T21:30:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.919127+0000", 2024-09-17T21:30:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00092894600000000005 2024-09-17T21:30:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.919152+0000", 2024-09-17T21:30:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4674000000000001e-05 2024-09-17T21:30:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.919163+0000", 2024-09-17T21:30:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1613999999999999e-05 2024-09-17T21:30:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.919223+0000", 2024-09-17T21:30:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9682999999999998e-05 2024-09-17T21:30:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.919247+0000", 2024-09-17T21:30:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3583000000000001e-05 2024-09-17T21:30:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.921040+0000", 2024-09-17T21:30:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0017930559999999999 2024-09-17T21:30:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.921079+0000", 2024-09-17T21:30:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9165000000000002e-05 2024-09-17T21:30:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:38413 3.5 e41/37 3:a30261b3:::rbd_data.1120ab27bfd0.0000000000000012:head v 41'43, mlcod=35'41)", 2024-09-17T21:30:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:02.933250+0000", 2024-09-17T21:30:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.34575655599999999, 2024-09-17T21:30:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001608228, 2024-09-17T21:30:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.933250+0000", 2024-09-17T21:30:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.933250+0000", 2024-09-17T21:30:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.933250+0000", 2024-09-17T21:30:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8e-07 2024-09-17T21:30:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.933265+0000", 2024-09-17T21:30:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5053e-05 2024-09-17T21:30:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.933274+0000", 2024-09-17T21:30:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0000000000000002e-06 2024-09-17T21:30:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.933286+0000", 2024-09-17T21:30:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1618e-05 2024-09-17T21:30:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.933343+0000", 2024-09-17T21:30:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6952e-05 2024-09-17T21:30:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.933365+0000", 2024-09-17T21:30:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2507e-05 2024-09-17T21:30:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.934817+0000", 2024-09-17T21:30:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0014514879999999999 2024-09-17T21:30:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.934858+0000", 2024-09-17T21:30:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1430000000000001e-05 2024-09-17T21:30:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.359 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:39515 3.0 e41/25 3:05e410ba:::rbd_data.1120ab27bfd0.000000000000001b:head v 41'17, mlcod=41'15)", 2024-09-17T21:30:03.359 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:03.022357+0000", 2024-09-17T21:30:03.359 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.25664991599999998, 2024-09-17T21:30:03.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00139222, 2024-09-17T21:30:03.359 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.359 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.359 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.359 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.022357+0000", 2024-09-17T21:30:03.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.022357+0000", 2024-09-17T21:30:03.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.022358+0000", 2024-09-17T21:30:03.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5030000000000001e-06 2024-09-17T21:30:03.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.022384+0000", 2024-09-17T21:30:03.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5286000000000002e-05 2024-09-17T21:30:03.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.022405+0000", 2024-09-17T21:30:03.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0948e-05 2024-09-17T21:30:03.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.022414+0000", 2024-09-17T21:30:03.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:30:03.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.022481+0000", 2024-09-17T21:30:03.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7737999999999994e-05 2024-09-17T21:30:03.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.022506+0000", 2024-09-17T21:30:03.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4834e-05 2024-09-17T21:30:03.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.023708+0000", 2024-09-17T21:30:03.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001202011 2024-09-17T21:30:03.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.023749+0000", 2024-09-17T21:30:03.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.074e-05 2024-09-17T21:30:03.363 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.363 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.363 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.363 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:39575 3.1f e41/37 3:f8f84c17:::rbd_data.1120ab27bfd0.000000000000000f:head v 41'29, mlcod=41'27)", 2024-09-17T21:30:03.363 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:03.069373+0000", 2024-09-17T21:30:03.363 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.209633715, 2024-09-17T21:30:03.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001748882, 2024-09-17T21:30:03.363 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.363 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.363 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.363 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.069373+0000", 2024-09-17T21:30:03.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.069373+0000", 2024-09-17T21:30:03.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.069374+0000", 2024-09-17T21:30:03.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.37e-06 2024-09-17T21:30:03.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.069392+0000", 2024-09-17T21:30:03.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7972e-05 2024-09-17T21:30:03.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.069412+0000", 2024-09-17T21:30:03.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9307999999999999e-05 2024-09-17T21:30:03.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.069420+0000", 2024-09-17T21:30:03.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6279999999999994e-06 2024-09-17T21:30:03.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.069475+0000", 2024-09-17T21:30:03.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4290999999999999e-05 2024-09-17T21:30:03.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.069494+0000", 2024-09-17T21:30:03.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9273999999999999e-05 2024-09-17T21:30:03.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.071084+0000", 2024-09-17T21:30:03.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015897680000000001 2024-09-17T21:30:03.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.071122+0000", 2024-09-17T21:30:03.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8271e-05 2024-09-17T21:30:03.367 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.367 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.367 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.367 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:39570 3.5 e41/37 3:a3db80ce:::rbd_data.1120ab27bfd0.000000000000000a:head v 41'46, mlcod=41'44)", 2024-09-17T21:30:03.367 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:03.069450+0000", 2024-09-17T21:30:03.367 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.20955695999999999, 2024-09-17T21:30:03.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001348473, 2024-09-17T21:30:03.368 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.368 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.368 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.368 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.069450+0000", 2024-09-17T21:30:03.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.069450+0000", 2024-09-17T21:30:03.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.069450+0000", 2024-09-17T21:30:03.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:30:03.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.069460+0000", 2024-09-17T21:30:03.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.302e-06 2024-09-17T21:30:03.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.069466+0000", 2024-09-17T21:30:03.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7329999999999997e-06 2024-09-17T21:30:03.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.069471+0000", 2024-09-17T21:30:03.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4789999999999998e-06 2024-09-17T21:30:03.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.069551+0000", 2024-09-17T21:30:03.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9754999999999998e-05 2024-09-17T21:30:03.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.069567+0000", 2024-09-17T21:30:03.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6770999999999999e-05 2024-09-17T21:30:03.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.070758+0000", 2024-09-17T21:30:03.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0011906670000000001 2024-09-17T21:30:03.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.070798+0000", 2024-09-17T21:30:03.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0306999999999999e-05 2024-09-17T21:30:03.372 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.372 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.372 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.372 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:39605 3.0 e41/25 3:00fe526a:::rbd_data.1120ab27bfd0.000000000000003a:head v 41'19, mlcod=41'17)", 2024-09-17T21:30:03.372 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:03.099263+0000", 2024-09-17T21:30:03.372 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.17974420699999999, 2024-09-17T21:30:03.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00240858, 2024-09-17T21:30:03.372 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.372 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.372 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:03.373 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.099263+0000", 2024-09-17T21:30:03.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.099263+0000", 2024-09-17T21:30:03.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.099263+0000", 2024-09-17T21:30:03.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8599999999999997e-07 2024-09-17T21:30:03.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.099271+0000", 2024-09-17T21:30:03.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1539999999999996e-06 2024-09-17T21:30:03.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.099284+0000", 2024-09-17T21:30:03.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.323e-05 2024-09-17T21:30:03.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.099291+0000", 2024-09-17T21:30:03.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.306e-06 2024-09-17T21:30:03.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.099345+0000", 2024-09-17T21:30:03.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3452999999999998e-05 2024-09-17T21:30:03.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.099359+0000", 2024-09-17T21:30:03.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4518000000000001e-05 2024-09-17T21:30:03.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:03.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.101637+0000", 2024-09-17T21:30:03.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022783149999999999 2024-09-17T21:30:03.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.101671+0000", 2024-09-17T21:30:03.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3717999999999998e-05 2024-09-17T21:30:03.376 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.376 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.376 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.376 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.376 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.377 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:03.378 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:30:03.421 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:03.419+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:03.481 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:03.482 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:03.482 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:03.482 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:03.482 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:03.479+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:30:03.490 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:30:03.558 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T21:30:03.558 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:30:03.559 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-17T21:30:03.559 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T21:30:03.559 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T21:30:03.559 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-17T21:30:03.576 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:30:03.620 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:03.618+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:03.627 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:30:03.633 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone9 from 0xddf436d (overlap 0x1b20533) to 0x0 2024-09-17T21:30:03.653 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:03.653 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:03.653 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:03.653 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:03.653 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:03.670 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:30:03.683 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:03.681+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:03.686 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:03.686 INFO:teuthology.orchestra.run.smithi171.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T21:30:03.686 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:03.760 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-17T21:30:03.784 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:03.782+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:03.810 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:03.811 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:03.829 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:30:03.829 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:30:03.837 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26531 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.849021+0000", 2024-09-17T21:30:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.984311833, 2024-09-17T21:30:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024653859, 2024-09-17T21:30:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26531 2024-09-17T21:30:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849021+0000", 2024-09-17T21:30:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849021+0000", 2024-09-17T21:30:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849021+0000", 2024-09-17T21:30:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6800000000000001e-07 2024-09-17T21:30:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849023+0000", 2024-09-17T21:30:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8789999999999999e-06 2024-09-17T21:30:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849029+0000", 2024-09-17T21:30:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1059999999999996e-06 2024-09-17T21:30:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849034+0000", 2024-09-17T21:30:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:30:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.873511+0000", 2024-09-17T21:30:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024476617999999999 2024-09-17T21:30:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.873592+0000", 2024-09-17T21:30:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.064e-05 2024-09-17T21:30:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.873675+0000", 2024-09-17T21:30:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3289999999999997e-05 2024-09-17T21:30:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26532 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.849040+0000", 2024-09-17T21:30:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.984292443999999, 2024-09-17T21:30:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024831517000000001, 2024-09-17T21:30:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26532 2024-09-17T21:30:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849040+0000", 2024-09-17T21:30:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849040+0000", 2024-09-17T21:30:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849041+0000", 2024-09-17T21:30:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6300000000000001e-07 2024-09-17T21:30:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849043+0000", 2024-09-17T21:30:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:30:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849049+0000", 2024-09-17T21:30:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9529999999999996e-06 2024-09-17T21:30:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849053+0000", 2024-09-17T21:30:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5569999999999997e-06 2024-09-17T21:30:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.873702+0000", 2024-09-17T21:30:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024648798999999999 2024-09-17T21:30:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.873790+0000", 2024-09-17T21:30:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7899999999999995e-05 2024-09-17T21:30:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.873872+0000", 2024-09-17T21:30:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2033000000000006e-05 2024-09-17T21:30:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26626 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 387072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853605+0000", 2024-09-17T21:30:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.979727306000001, 2024-09-17T21:30:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024510626000000001, 2024-09-17T21:30:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26626 2024-09-17T21:30:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853605+0000", 2024-09-17T21:30:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853605+0000", 2024-09-17T21:30:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853665+0000", 2024-09-17T21:30:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9477e-05 2024-09-17T21:30:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853668+0000", 2024-09-17T21:30:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3139999999999999e-06 2024-09-17T21:30:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853676+0000", 2024-09-17T21:30:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7389999999999999e-06 2024-09-17T21:30:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853682+0000", 2024-09-17T21:30:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8710000000000002e-06 2024-09-17T21:30:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.877964+0000", 2024-09-17T21:30:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024282142999999999 2024-09-17T21:30:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878038+0000", 2024-09-17T21:30:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3882999999999997e-05 2024-09-17T21:30:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878116+0000", 2024-09-17T21:30:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8199000000000001e-05 2024-09-17T21:30:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26627 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 399360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853691+0000", 2024-09-17T21:30:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.979641733999999, 2024-09-17T21:30:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024605452999999999, 2024-09-17T21:30:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26627 2024-09-17T21:30:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853691+0000", 2024-09-17T21:30:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853691+0000", 2024-09-17T21:30:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853692+0000", 2024-09-17T21:30:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7300000000000001e-07 2024-09-17T21:30:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853694+0000", 2024-09-17T21:30:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3120000000000001e-06 2024-09-17T21:30:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853700+0000", 2024-09-17T21:30:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1940000000000003e-06 2024-09-17T21:30:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853705+0000", 2024-09-17T21:30:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8980000000000004e-06 2024-09-17T21:30:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878143+0000", 2024-09-17T21:30:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024437708999999998 2024-09-17T21:30:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878218+0000", 2024-09-17T21:30:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5377999999999996e-05 2024-09-17T21:30:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878297+0000", 2024-09-17T21:30:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8489e-05 2024-09-17T21:30:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26628 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 413696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853711+0000", 2024-09-17T21:30:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.97962164, 2024-09-17T21:30:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024762052999999999, 2024-09-17T21:30:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26628 2024-09-17T21:30:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853711+0000", 2024-09-17T21:30:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853711+0000", 2024-09-17T21:30:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853712+0000", 2024-09-17T21:30:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.15e-07 2024-09-17T21:30:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853713+0000", 2024-09-17T21:30:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.79e-06 2024-09-17T21:30:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853719+0000", 2024-09-17T21:30:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0469999999999999e-06 2024-09-17T21:30:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853724+0000", 2024-09-17T21:30:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.69e-06 2024-09-17T21:30:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878324+0000", 2024-09-17T21:30:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024599875 2024-09-17T21:30:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878396+0000", 2024-09-17T21:30:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2486999999999999e-05 2024-09-17T21:30:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878473+0000", 2024-09-17T21:30:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6749000000000007e-05 2024-09-17T21:30:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26629 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853730+0000", 2024-09-17T21:30:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.979602511, 2024-09-17T21:30:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024923506000000002, 2024-09-17T21:30:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26629 2024-09-17T21:30:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853730+0000", 2024-09-17T21:30:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853730+0000", 2024-09-17T21:30:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853731+0000", 2024-09-17T21:30:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:30:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853733+0000", 2024-09-17T21:30:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:30:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853739+0000", 2024-09-17T21:30:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0159999999999997e-06 2024-09-17T21:30:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853744+0000", 2024-09-17T21:30:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0019999999999997e-06 2024-09-17T21:30:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878501+0000", 2024-09-17T21:30:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024757112000000001 2024-09-17T21:30:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878573+0000", 2024-09-17T21:30:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2006000000000001e-05 2024-09-17T21:30:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878654+0000", 2024-09-17T21:30:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0734000000000003e-05 2024-09-17T21:30:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26630 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853750+0000", 2024-09-17T21:30:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.979583022, 2024-09-17T21:30:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025078738999999999, 2024-09-17T21:30:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26630 2024-09-17T21:30:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853750+0000", 2024-09-17T21:30:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853750+0000", 2024-09-17T21:30:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853750+0000", 2024-09-17T21:30:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:30:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853752+0000", 2024-09-17T21:30:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8700000000000001e-06 2024-09-17T21:30:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853758+0000", 2024-09-17T21:30:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1340000000000001e-06 2024-09-17T21:30:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853763+0000", 2024-09-17T21:30:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9819999999999999e-06 2024-09-17T21:30:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878681+0000", 2024-09-17T21:30:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024917661000000001 2024-09-17T21:30:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878752+0000", 2024-09-17T21:30:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1132999999999998e-05 2024-09-17T21:30:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878829+0000", 2024-09-17T21:30:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6638999999999996e-05 2024-09-17T21:30:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.875 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26645 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 266240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.875 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853769+0000", 2024-09-17T21:30:03.875 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.979563848, 2024-09-17T21:30:03.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025237900000000001, 2024-09-17T21:30:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26645 2024-09-17T21:30:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853769+0000", 2024-09-17T21:30:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853769+0000", 2024-09-17T21:30:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853769+0000", 2024-09-17T21:30:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2599999999999998e-07 2024-09-17T21:30:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853771+0000", 2024-09-17T21:30:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9609999999999999e-06 2024-09-17T21:30:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853777+0000", 2024-09-17T21:30:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3630000000000002e-06 2024-09-17T21:30:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853781+0000", 2024-09-17T21:30:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7779999999999998e-06 2024-09-17T21:30:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878855+0000", 2024-09-17T21:30:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025073649999999999 2024-09-17T21:30:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878927+0000", 2024-09-17T21:30:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1500000000000003e-05 2024-09-17T21:30:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879007+0000", 2024-09-17T21:30:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0321999999999994e-05 2024-09-17T21:30:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26646 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 280576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853787+0000", 2024-09-17T21:30:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.979545615999999, 2024-09-17T21:30:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025570283999999999, 2024-09-17T21:30:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26646 2024-09-17T21:30:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853787+0000", 2024-09-17T21:30:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853787+0000", 2024-09-17T21:30:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853788+0000", 2024-09-17T21:30:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:30:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853793+0000", 2024-09-17T21:30:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0900000000000004e-06 2024-09-17T21:30:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853798+0000", 2024-09-17T21:30:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8479999999999997e-06 2024-09-17T21:30:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853803+0000", 2024-09-17T21:30:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7620000000000003e-06 2024-09-17T21:30:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879035+0000", 2024-09-17T21:30:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025231535999999999 2024-09-17T21:30:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879107+0000", 2024-09-17T21:30:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2154000000000001e-05 2024-09-17T21:30:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879357+0000", 2024-09-17T21:30:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025055800000000001 2024-09-17T21:30:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26647 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 292864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853809+0000", 2024-09-17T21:30:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.979523552, 2024-09-17T21:30:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025774357000000001, 2024-09-17T21:30:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26647 2024-09-17T21:30:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853809+0000", 2024-09-17T21:30:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853809+0000", 2024-09-17T21:30:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853810+0000", 2024-09-17T21:30:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7500000000000001e-07 2024-09-17T21:30:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853811+0000", 2024-09-17T21:30:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7749999999999999e-06 2024-09-17T21:30:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853818+0000", 2024-09-17T21:30:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7519999999999999e-06 2024-09-17T21:30:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853823+0000", 2024-09-17T21:30:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2399999999999998e-06 2024-09-17T21:30:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879387+0000", 2024-09-17T21:30:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02556394 2024-09-17T21:30:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879462+0000", 2024-09-17T21:30:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4697000000000007e-05 2024-09-17T21:30:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879584+0000", 2024-09-17T21:30:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012157799999999999 2024-09-17T21:30:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26648 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 307200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853862+0000", 2024-09-17T21:30:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.979471092000001, 2024-09-17T21:30:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025888738000000001, 2024-09-17T21:30:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26648 2024-09-17T21:30:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853862+0000", 2024-09-17T21:30:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853862+0000", 2024-09-17T21:30:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853862+0000", 2024-09-17T21:30:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1799999999999995e-07 2024-09-17T21:30:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853865+0000", 2024-09-17T21:30:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5950000000000001e-06 2024-09-17T21:30:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853872+0000", 2024-09-17T21:30:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1749999999999999e-06 2024-09-17T21:30:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853877+0000", 2024-09-17T21:30:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:30:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879609+0000", 2024-09-17T21:30:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025731970999999999 2024-09-17T21:30:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879674+0000", 2024-09-17T21:30:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4958999999999999e-05 2024-09-17T21:30:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879750+0000", 2024-09-17T21:30:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6327000000000003e-05 2024-09-17T21:30:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26649 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 321536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853886+0000", 2024-09-17T21:30:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.979446999, 2024-09-17T21:30:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026043475999999999, 2024-09-17T21:30:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26649 2024-09-17T21:30:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853886+0000", 2024-09-17T21:30:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853886+0000", 2024-09-17T21:30:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853886+0000", 2024-09-17T21:30:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:30:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853888+0000", 2024-09-17T21:30:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0789999999999999e-06 2024-09-17T21:30:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853894+0000", 2024-09-17T21:30:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0209999999999996e-06 2024-09-17T21:30:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853900+0000", 2024-09-17T21:30:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2220000000000001e-06 2024-09-17T21:30:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879779+0000", 2024-09-17T21:30:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025879049000000001 2024-09-17T21:30:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879852+0000", 2024-09-17T21:30:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3244999999999994e-05 2024-09-17T21:30:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879929+0000", 2024-09-17T21:30:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7472999999999995e-05 2024-09-17T21:30:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26650 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 333824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853906+0000", 2024-09-17T21:30:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.979426882, 2024-09-17T21:30:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026202728000000002, 2024-09-17T21:30:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26650 2024-09-17T21:30:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853906+0000", 2024-09-17T21:30:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853906+0000", 2024-09-17T21:30:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853906+0000", 2024-09-17T21:30:03.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:03.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853908+0000", 2024-09-17T21:30:03.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0030000000000001e-06 2024-09-17T21:30:03.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853914+0000", 2024-09-17T21:30:03.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3759999999999999e-06 2024-09-17T21:30:03.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853919+0000", 2024-09-17T21:30:03.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9139999999999999e-06 2024-09-17T21:30:03.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879956+0000", 2024-09-17T21:30:03.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026037523999999999 2024-09-17T21:30:03.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880027+0000", 2024-09-17T21:30:03.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0939000000000006e-05 2024-09-17T21:30:03.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880109+0000", 2024-09-17T21:30:03.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1594000000000005e-05 2024-09-17T21:30:03.907 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.907 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.907 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.907 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26651 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.907 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853925+0000", 2024-09-17T21:30:03.907 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.979408073, 2024-09-17T21:30:03.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026365942, 2024-09-17T21:30:03.907 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.907 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.907 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.908 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26651 2024-09-17T21:30:03.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.908 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853925+0000", 2024-09-17T21:30:03.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853925+0000", 2024-09-17T21:30:03.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853925+0000", 2024-09-17T21:30:03.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4799999999999999e-07 2024-09-17T21:30:03.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853927+0000", 2024-09-17T21:30:03.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9250000000000002e-06 2024-09-17T21:30:03.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853932+0000", 2024-09-17T21:30:03.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3489999999999999e-06 2024-09-17T21:30:03.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853937+0000", 2024-09-17T21:30:03.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4769999999999997e-06 2024-09-17T21:30:03.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880136+0000", 2024-09-17T21:30:03.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026199618000000001 2024-09-17T21:30:03.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880213+0000", 2024-09-17T21:30:03.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6594999999999994e-05 2024-09-17T21:30:03.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880291+0000", 2024-09-17T21:30:03.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.763e-05 2024-09-17T21:30:03.912 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.912 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.912 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.912 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26652 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.913 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853943+0000", 2024-09-17T21:30:03.913 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.979389612, 2024-09-17T21:30:03.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026527075000000001, 2024-09-17T21:30:03.913 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.913 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.913 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.913 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.913 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.913 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.913 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26652 2024-09-17T21:30:03.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.913 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853943+0000", 2024-09-17T21:30:03.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853943+0000", 2024-09-17T21:30:03.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853944+0000", 2024-09-17T21:30:03.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2599999999999998e-07 2024-09-17T21:30:03.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853946+0000", 2024-09-17T21:30:03.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9970000000000001e-06 2024-09-17T21:30:03.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853953+0000", 2024-09-17T21:30:03.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4080000000000001e-06 2024-09-17T21:30:03.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853958+0000", 2024-09-17T21:30:03.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7280000000000003e-06 2024-09-17T21:30:03.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880319+0000", 2024-09-17T21:30:03.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026360999 2024-09-17T21:30:03.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880391+0000", 2024-09-17T21:30:03.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2418999999999999e-05 2024-09-17T21:30:03.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880470+0000", 2024-09-17T21:30:03.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9197999999999996e-05 2024-09-17T21:30:03.917 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.917 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.917 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.917 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26653 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 374784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.918 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853964+0000", 2024-09-17T21:30:03.918 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.979368915, 2024-09-17T21:30:03.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026678573000000001, 2024-09-17T21:30:03.918 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.918 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.918 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.918 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26653 2024-09-17T21:30:03.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.918 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853964+0000", 2024-09-17T21:30:03.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853964+0000", 2024-09-17T21:30:03.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853964+0000", 2024-09-17T21:30:03.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:30:03.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853966+0000", 2024-09-17T21:30:03.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.888e-06 2024-09-17T21:30:03.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853972+0000", 2024-09-17T21:30:03.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T21:30:03.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853976+0000", 2024-09-17T21:30:03.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7060000000000003e-06 2024-09-17T21:30:03.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880499+0000", 2024-09-17T21:30:03.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026523105000000002 2024-09-17T21:30:03.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880570+0000", 2024-09-17T21:30:03.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0207000000000001e-05 2024-09-17T21:30:03.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880642+0000", 2024-09-17T21:30:03.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2886000000000007e-05 2024-09-17T21:30:03.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.922 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.922 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26654 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 389120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.922 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853982+0000", 2024-09-17T21:30:03.922 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.979350851, 2024-09-17T21:30:03.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026832110999999999, 2024-09-17T21:30:03.922 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.923 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.923 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.923 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26654 2024-09-17T21:30:03.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.923 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853982+0000", 2024-09-17T21:30:03.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853982+0000", 2024-09-17T21:30:03.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853982+0000", 2024-09-17T21:30:03.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8099999999999998e-07 2024-09-17T21:30:03.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853984+0000", 2024-09-17T21:30:03.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7460000000000001e-06 2024-09-17T21:30:03.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853990+0000", 2024-09-17T21:30:03.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1179999999999997e-06 2024-09-17T21:30:03.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853995+0000", 2024-09-17T21:30:03.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4789999999999998e-06 2024-09-17T21:30:03.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880670+0000", 2024-09-17T21:30:03.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026675299 2024-09-17T21:30:03.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880736+0000", 2024-09-17T21:30:03.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6476000000000005e-05 2024-09-17T21:30:03.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880814+0000", 2024-09-17T21:30:03.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7612000000000002e-05 2024-09-17T21:30:03.926 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.927 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.927 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.927 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26655 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 403456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.927 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.854001+0000", 2024-09-17T21:30:03.927 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.979332171999999, 2024-09-17T21:30:03.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026991347999999998, 2024-09-17T21:30:03.927 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.927 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.927 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.928 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.928 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.928 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26655 2024-09-17T21:30:03.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.928 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854001+0000", 2024-09-17T21:30:03.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854001+0000", 2024-09-17T21:30:03.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854001+0000", 2024-09-17T21:30:03.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.89e-07 2024-09-17T21:30:03.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854003+0000", 2024-09-17T21:30:03.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.826e-06 2024-09-17T21:30:03.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854009+0000", 2024-09-17T21:30:03.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7150000000000003e-06 2024-09-17T21:30:03.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854014+0000", 2024-09-17T21:30:03.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1109999999999999e-06 2024-09-17T21:30:03.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880842+0000", 2024-09-17T21:30:03.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026828161999999999 2024-09-17T21:30:03.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880912+0000", 2024-09-17T21:30:03.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0067000000000005e-05 2024-09-17T21:30:03.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880992+0000", 2024-09-17T21:30:03.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0078000000000001e-05 2024-09-17T21:30:03.932 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.932 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.932 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.932 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26656 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 415744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.932 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.854020+0000", 2024-09-17T21:30:03.932 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.979312713000001, 2024-09-17T21:30:03.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027156040999999999, 2024-09-17T21:30:03.933 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.933 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.933 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.933 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.933 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.933 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.933 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26656 2024-09-17T21:30:03.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.933 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854020+0000", 2024-09-17T21:30:03.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854020+0000", 2024-09-17T21:30:03.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854020+0000", 2024-09-17T21:30:03.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:30:03.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854023+0000", 2024-09-17T21:30:03.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2139999999999999e-06 2024-09-17T21:30:03.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854028+0000", 2024-09-17T21:30:03.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7570000000000001e-06 2024-09-17T21:30:03.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854033+0000", 2024-09-17T21:30:03.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3409999999999996e-06 2024-09-17T21:30:03.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.881018+0000", 2024-09-17T21:30:03.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026985529000000001 2024-09-17T21:30:03.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.881090+0000", 2024-09-17T21:30:03.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1946000000000005e-05 2024-09-17T21:30:03.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.881176+0000", 2024-09-17T21:30:03.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5886999999999999e-05 2024-09-17T21:30:03.938 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.938 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.938 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.938 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26657 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:03.938 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.854123+0000", 2024-09-17T21:30:03.938 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.979209426000001, 2024-09-17T21:30:03.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027224600000000002, 2024-09-17T21:30:03.938 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:03.938 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:03.939 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:03.939 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:03.939 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:03.939 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:03.939 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26657 2024-09-17T21:30:03.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.939 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:03.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:03.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854123+0000", 2024-09-17T21:30:03.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:03.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854123+0000", 2024-09-17T21:30:03.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:03.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:03.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854124+0000", 2024-09-17T21:30:03.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:30:03.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:03.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854127+0000", 2024-09-17T21:30:03.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0750000000000002e-06 2024-09-17T21:30:03.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:03.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854134+0000", 2024-09-17T21:30:03.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4630000000000004e-06 2024-09-17T21:30:03.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:03.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854141+0000", 2024-09-17T21:30:03.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:30:03.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:03.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.881204+0000", 2024-09-17T21:30:03.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027062685999999999 2024-09-17T21:30:03.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:03.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.881273+0000", 2024-09-17T21:30:03.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8841e-05 2024-09-17T21:30:03.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:03.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:03.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:03.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.881348+0000", 2024-09-17T21:30:03.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5265999999999994e-05 2024-09-17T21:30:03.943 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.943 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.944 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.944 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:03.944 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:03.944 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:03.946 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:30:03.946 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:03.946 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:03.947 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:03.947 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:03.947 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:03.948 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:03.948 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:03.948 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:03.949 INFO:teuthology.orchestra.run.smithi129.stdout:1534 write 0xb4884a2 thru 0xb48b8be (0x341d bytes) 2024-09-17T21:30:03.949 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:03.949 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:03.949 INFO:teuthology.orchestra.run.smithi129.stdout:1538 read 0xbb8d487 thru 0xbb9b24b (0xddc5 bytes) 2024-09-17T21:30:03.949 INFO:teuthology.orchestra.run.smithi129.stdout:1539 writesame 0xc7fada thru 0xc8abcc data_size 0x1(0xb0f3 bytes) 2024-09-17T21:30:03.950 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:03.950 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:03.951 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:03.943+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:03.951 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:03.951 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:03.952 INFO:teuthology.orchestra.run.smithi129.stdout:1541 write 0xe22bce thru 0xe2c8f1 (0x9d24 bytes) 2024-09-17T21:30:03.952 INFO:teuthology.orchestra.run.smithi129.stdout:1542 clone 11 order 18 su 8192 sc 14 2024-09-17T21:30:04.046 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:04.044+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:04.053 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:04.054 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:04.062 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:30:04.146 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:04.144+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:04.166 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:04.167 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:04.176 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:30:04.247 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:04.245+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:04.281 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:04.281 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:04.289 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:30:04.348 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:04.345+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:04.432 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:04.433 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:04.441 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:30:04.449 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:04.446+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:30:04.544 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:04.544 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:04.549 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:04.547+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:04.553 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:30:04.581 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is set 2024-09-17T21:30:04.650 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:04.648+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:04.651 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone10 from 0xddf436d (overlap 0xddf436d) to 0xb1902bd 2024-09-17T21:30:04.708 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:04.708 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:04.717 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:30:04.758 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:04.756+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:04.821 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-17T21:30:04.822 INFO:teuthology.orchestra.run.smithi129.stdout:1544 read 0x6292b2e thru 0x629665b (0x3b2e bytes) 2024-09-17T21:30:04.823 INFO:teuthology.orchestra.run.smithi129.stdout:1545 write 0x80765b9 thru 0x8085ee4 (0xf92c bytes) 2024-09-17T21:30:04.841 INFO:teuthology.orchestra.run.smithi129.stdout:1548 writesame 0x7aae0de thru 0x7abd155 data_size 0x4c(0xf078 bytes) 2024-09-17T21:30:04.853 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:04.850+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:30:04.858 INFO:teuthology.orchestra.run.smithi129.stdout:1549 write 0xbb32ad5 thru 0xbb38186 (0x56b2 bytes) 2024-09-17T21:30:04.870 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:04.870 INFO:teuthology.orchestra.run.smithi129.stdout:1551 read 0x32740f thru 0x32d4b5 (0x60a7 bytes) 2024-09-17T21:30:04.870 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:04.870 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:04.871 INFO:teuthology.orchestra.run.smithi129.stdout:truncating to largest ever: 0xede2bce 2024-09-17T21:30:04.871 INFO:teuthology.orchestra.run.smithi129.stdout:1552 trunc from 0xddf436d to 0xede2bce 2024-09-17T21:30:04.877 INFO:teuthology.orchestra.run.smithi129.stdout:1553 writesame 0x7adae1e thru 0x7ae94bc data_size 0x1(0xe69f bytes) 2024-09-17T21:30:04.880 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:30:04.904 INFO:teuthology.orchestra.run.smithi129.stdout:1555 trunc from 0xede2bce to 0x5b76241 2024-09-17T21:30:04.944 INFO:teuthology.orchestra.run.smithi129.stdout:1556 read 0x3f30b71 thru 0x3f38146 (0x75d6 bytes) 2024-09-17T21:30:04.944 INFO:teuthology.orchestra.run.smithi129.stdout:1559 read 0x401ecd0 thru 0x4021cf2 (0x3023 bytes) 2024-09-17T21:30:04.944 INFO:teuthology.orchestra.run.smithi129.stdout:1560 write 0x3633d8f thru 0x3642527 (0xe799 bytes) 2024-09-17T21:30:04.953 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:04.951+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:30:04.961 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:04.961 INFO:teuthology.orchestra.run.smithi129.stdout:1564 read 0x4cb7c4d thru 0x4cc2724 (0xaad8 bytes) 2024-09-17T21:30:04.968 INFO:teuthology.orchestra.run.smithi129.stdout:1566 trunc from 0x5b76241 to 0xb852a62 2024-09-17T21:30:04.974 INFO:teuthology.orchestra.run.smithi129.stdout:1567 read 0x9f3098 thru 0x9f72ca (0x4233 bytes) 2024-09-17T21:30:04.974 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:04.975 INFO:teuthology.orchestra.run.smithi129.stdout:1569 read 0x6bb22a0 thru 0x6bbfd62 (0xdac3 bytes) 2024-09-17T21:30:04.975 INFO:teuthology.orchestra.run.smithi129.stdout:1570 writesame 0x29431ea thru 0x294bca3 data_size 0x9(0x8aba bytes) 2024-09-17T21:30:04.993 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:04.993 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:04.996 INFO:teuthology.orchestra.run.smithi129.stdout:1571 trunc from 0xb852a62 to 0x4fc297 2024-09-17T21:30:05.004 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:30:05.055 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:05.053+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:05.061 INFO:teuthology.orchestra.run.smithi129.stdout:1572 trunc from 0x4fc297 to 0x52dbfc4 2024-09-17T21:30:05.069 INFO:teuthology.orchestra.run.smithi129.stdout:1573 trunc from 0x52dbfc4 to 0x626459e 2024-09-17T21:30:05.087 INFO:teuthology.orchestra.run.smithi129.stdout:1574 writesame 0xd344a3f thru 0xd34aff6 data_size 0x1e(0x65b8 bytes) 2024-09-17T21:30:05.088 INFO:teuthology.orchestra.run.smithi129.stdout:1575 punch from 0x5c98e89 to 0x5c99825, (0x99c bytes) 2024-09-17T21:30:05.088 INFO:teuthology.orchestra.run.smithi129.stdout:1577 punch from 0x32605f5 to 0x326f88e, (0xf299 bytes) 2024-09-17T21:30:05.088 INFO:teuthology.orchestra.run.smithi129.stdout:1578 write 0x1180352 thru 0x118b8c6 (0xb575 bytes) 2024-09-17T21:30:05.095 INFO:teuthology.orchestra.run.smithi129.stdout:1579 read 0xc39933d thru 0xc3a28a7 (0x956b bytes) 2024-09-17T21:30:05.097 INFO:teuthology.orchestra.run.smithi129.stdout:1581 trunc from 0xd34aff7 to 0xb1d6767 2024-09-17T21:30:05.104 INFO:teuthology.orchestra.run.smithi129.stdout:1582 read 0xa5178a thru 0xa575ca (0x5e41 bytes) 2024-09-17T21:30:05.104 INFO:teuthology.orchestra.run.smithi129.stdout:1583 write 0xa65529a thru 0xa658c0e (0x3975 bytes) 2024-09-17T21:30:05.115 INFO:teuthology.orchestra.run.smithi129.stdout:1584 trunc from 0xb1d6767 to 0x57a06df 2024-09-17T21:30:05.116 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:05.117 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:05.125 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:30:05.138 INFO:teuthology.orchestra.run.smithi129.stdout:1587 write 0xbcf2f77 thru 0xbcffd4b (0xcdd5 bytes) 2024-09-17T21:30:05.147 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:05.147 INFO:teuthology.orchestra.run.smithi129.stdout:1589 writesame 0xb887fc6 thru 0xb888dc8 data_size 0x1(0xe03 bytes) 2024-09-17T21:30:05.153 INFO:teuthology.orchestra.run.smithi129.stdout:1591 read 0x8147253 thru 0x814dd40 (0x6aee bytes) 2024-09-17T21:30:05.153 INFO:teuthology.orchestra.run.smithi129.stdout:1592 read 0x1fb9d36 thru 0x1fc3215 (0x94e0 bytes) 2024-09-17T21:30:05.153 INFO:teuthology.orchestra.run.smithi129.stdout:1593 write 0x153718 thru 0x15e680 (0xaf69 bytes) 2024-09-17T21:30:05.167 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:05.165+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:05.180 INFO:teuthology.orchestra.run.smithi129.stdout:1594 read 0x502a38d thru 0x503705d (0xccd1 bytes) 2024-09-17T21:30:05.180 INFO:teuthology.orchestra.run.smithi129.stdout:1596 punch from 0x272a8ac to 0x27341bd, (0x9911 bytes) 2024-09-17T21:30:05.180 INFO:teuthology.orchestra.run.smithi129.stdout:1598 read 0x93b9c6c thru 0x93c6085 (0xc41a bytes) 2024-09-17T21:30:05.180 INFO:teuthology.orchestra.run.smithi129.stdout:1602 writesame 0x9f70df thru 0xa066b8 data_size 0x1(0xf5da bytes) 2024-09-17T21:30:05.203 INFO:teuthology.orchestra.run.smithi129.stdout:1603 trunc from 0xbcffd4c to 0x634c889 2024-09-17T21:30:05.214 INFO:teuthology.orchestra.run.smithi129.stdout:1604 punch from 0x3f2eff5 to 0x3f33356, (0x4361 bytes) 2024-09-17T21:30:05.214 INFO:teuthology.orchestra.run.smithi129.stdout:1605 read 0x40cc203 thru 0x40d5237 (0x9035 bytes) 2024-09-17T21:30:05.214 INFO:teuthology.orchestra.run.smithi129.stdout:1606 read 0x114e011 thru 0x1150692 (0x2682 bytes) 2024-09-17T21:30:05.215 INFO:teuthology.orchestra.run.smithi129.stdout:1607 write 0xd6437 thru 0xd8949 (0x2513 bytes) 2024-09-17T21:30:05.222 INFO:teuthology.orchestra.run.smithi129.stdout:1611 trunc from 0x634c889 to 0x5f6d622 2024-09-17T21:30:05.241 INFO:teuthology.orchestra.run.smithi129.stdout:1613 writesame 0xbc3068e thru 0xbc38c85 data_size 0xc(0x85f8 bytes) 2024-09-17T21:30:05.241 INFO:teuthology.orchestra.run.smithi129.stdout:1614 read 0x6738a02 thru 0x6742853 (0x9e52 bytes) 2024-09-17T21:30:05.241 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:05.241 INFO:teuthology.orchestra.run.smithi129.stdout:1616 write 0x97eb005 thru 0x97f4222 (0x921e bytes) 2024-09-17T21:30:05.257 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:05.255+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:30:05.264 INFO:teuthology.orchestra.run.smithi129.stdout:1617 trunc from 0xbc38c86 to 0x3939096 2024-09-17T21:30:05.273 INFO:teuthology.orchestra.run.smithi129.stdout:1618 punch from 0x2b83f54 to 0x2b85b44, (0x1bf0 bytes) 2024-09-17T21:30:05.279 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:05.280 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:05.281 INFO:teuthology.orchestra.run.smithi129.stdout:1620 trunc from 0x3939096 to 0xa9275a9 2024-09-17T21:30:05.287 INFO:teuthology.orchestra.run.smithi129.stdout:1624 read 0x2b94f3d thru 0x2b967b9 (0x187d bytes) 2024-09-17T21:30:05.289 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:30:05.296 INFO:teuthology.orchestra.run.smithi129.stdout:1625 writesame 0xd8c25d4 thru 0xd8cee4d data_size 0x2(0xc87a bytes) 2024-09-17T21:30:05.296 INFO:teuthology.orchestra.run.smithi129.stdout:1627 read 0x2e1d8bd thru 0x2e2cd41 (0xf485 bytes) 2024-09-17T21:30:05.296 INFO:teuthology.orchestra.run.smithi129.stdout:1628 write 0x4123920 thru 0x412ceb4 (0x9595 bytes) 2024-09-17T21:30:05.307 INFO:teuthology.orchestra.run.smithi129.stdout:1630 punch from 0x7a4fdb6 to 0x7a54a36, (0x4c80 bytes) 2024-09-17T21:30:05.307 INFO:teuthology.orchestra.run.smithi129.stdout:1631 write 0xb9bd9c1 thru 0xb9ca243 (0xc883 bytes) 2024-09-17T21:30:05.323 INFO:teuthology.orchestra.run.smithi129.stdout:1632 writesame 0xe8ae5e0 thru 0xe8b86e5 data_size 0x1(0xa106 bytes) 2024-09-17T21:30:05.339 INFO:teuthology.orchestra.run.smithi129.stdout:1635 trunc from 0xe8b86e6 to 0xb68fdd 2024-09-17T21:30:05.358 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:05.356+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:05.365 INFO:teuthology.orchestra.run.smithi129.stdout:1637 writesame 0x265cd7e thru 0x26696a5 data_size 0x8(0xc928 bytes) 2024-09-17T21:30:05.376 INFO:teuthology.orchestra.run.smithi129.stdout:1638 write 0x657e5f1 thru 0x658403b (0x5a4b bytes) 2024-09-17T21:30:05.390 INFO:teuthology.orchestra.run.smithi129.stdout:1639 writesame 0x69632a9 thru 0x696dcf5 data_size 0x1(0xaa4d bytes) 2024-09-17T21:30:05.390 INFO:teuthology.orchestra.run.smithi129.stdout:1640 writesame 0x1cf753d thru 0x1d052f0 data_size 0xe(0xddb4 bytes) 2024-09-17T21:30:05.399 INFO:teuthology.orchestra.run.smithi129.stdout:1641 read 0x4c29305 thru 0x4c334af (0xa1ab bytes) 2024-09-17T21:30:05.400 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:05.400 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:05.409 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:30:05.414 INFO:teuthology.orchestra.run.smithi129.stdout:1642 write 0xeb3735d thru 0xeb42f06 (0xbbaa bytes) 2024-09-17T21:30:05.426 INFO:teuthology.orchestra.run.smithi129.stdout:1644 read 0x5f162d7 thru 0x5f21ba8 (0xb8d2 bytes) 2024-09-17T21:30:05.426 INFO:teuthology.orchestra.run.smithi129.stdout:1645 punch from 0x2ac0954 to 0x2ac61a1, (0x584d bytes) 2024-09-17T21:30:05.426 INFO:teuthology.orchestra.run.smithi129.stdout:1646 read 0xc289abf thru 0xc294e52 (0xb394 bytes) 2024-09-17T21:30:05.441 INFO:teuthology.orchestra.run.smithi129.stdout:1647 trunc from 0xeb42f07 to 0x2150271 2024-09-17T21:30:05.453 INFO:teuthology.orchestra.run.smithi129.stdout:1648 write 0x1040f09 thru 0x104fbc7 (0xecbf bytes) 2024-09-17T21:30:05.466 INFO:teuthology.orchestra.run.smithi129.stdout:1650 trunc from 0x2150271 to 0x4f0030b 2024-09-17T21:30:05.472 INFO:teuthology.orchestra.run.smithi129.stdout:1651 trunc from 0x4f0030b to 0x582edcf 2024-09-17T21:30:05.478 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:05.478 INFO:teuthology.orchestra.run.smithi129.stdout:1653 punch from 0x48d28a1 to 0x48db468, (0x8bc7 bytes) 2024-09-17T21:30:05.478 INFO:teuthology.orchestra.run.smithi129.stdout:1654 read 0x22f7a90 thru 0x22f948a (0x19fb bytes) 2024-09-17T21:30:05.487 INFO:teuthology.orchestra.run.smithi129.stdout:1656 write 0x6b5c1b2 thru 0x6b61442 (0x5291 bytes) 2024-09-17T21:30:05.493 INFO:teuthology.orchestra.run.smithi129.stdout:1657 punch from 0x4b9dd9a to 0x4ba724f, (0x94b5 bytes) 2024-09-17T21:30:05.493 INFO:teuthology.orchestra.run.smithi129.stdout:1659 read 0x2ee1229 thru 0x2ee997f (0x8757 bytes) 2024-09-17T21:30:05.493 INFO:teuthology.orchestra.run.smithi129.stdout:1660 write 0xde5bea thru 0xde6490 (0x8a7 bytes) 2024-09-17T21:30:05.494 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:05.492+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:05.504 INFO:teuthology.orchestra.run.smithi129.stdout:1661 writesame 0x960bc9b thru 0x9615511 data_size 0x1(0x9877 bytes) 2024-09-17T21:30:05.504 INFO:teuthology.orchestra.run.smithi129.stdout:1662 read 0x4613d79 thru 0x46182a7 (0x452f bytes) 2024-09-17T21:30:05.512 INFO:teuthology.orchestra.run.smithi129.stdout:1664 writesame 0xbc45597 thru 0xbc47224 data_size 0x1(0x1c8e bytes) 2024-09-17T21:30:05.512 INFO:teuthology.orchestra.run.smithi129.stdout:1666 clone 12 order 19 su 4096 sc 10 2024-09-17T21:30:05.558 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:05.558 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:05.560 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:05.558+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:05.567 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:30:05.656 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone11 from 0xbc47225 (overlap 0x4fc297) to 0x7f9db 2024-09-17T21:30:05.661 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:05.659+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:05.692 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:05.692 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:05.704 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:30:05.762 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:05.760+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:05.771 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-17T21:30:05.863 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:05.860+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:05.871 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:05.871 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:05.882 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:30:05.953 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:30:05.966 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:05.964+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:06.024 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:06.025 INFO:teuthology.orchestra.run.smithi129.stdout:1668 trunc from 0xbc47225 to 0xcac9e07 2024-09-17T21:30:06.032 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:06.032 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:06.035 INFO:teuthology.orchestra.run.smithi129.stdout:1669 punch from 0x67bd675 to 0x67cb20d, (0xdb98 bytes) 2024-09-17T21:30:06.035 INFO:teuthology.orchestra.run.smithi129.stdout:1670 write 0x513e9ed thru 0x513ff62 (0x1576 bytes) 2024-09-17T21:30:06.041 INFO:teuthology.orchestra.run.smithi129.stdout:1671 read 0x6d2f841 thru 0x6d3f2b3 (0xfa73 bytes) 2024-09-17T21:30:06.041 INFO:teuthology.orchestra.run.smithi129.stdout:1672 punch from 0xc80fb1b to 0xc8139bf, (0x3ea4 bytes) 2024-09-17T21:30:06.041 INFO:teuthology.orchestra.run.smithi129.stdout:1673 clone 13 order 25 su 1048576 sc 2 2024-09-17T21:30:06.042 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:30:06.065 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:06.063+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:06.092 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:06.092 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:06.109 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:30:06.109 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:30:06.125 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:06.126 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:06.143 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:06.143 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:06.150 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:06.150 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:06.150 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:06.150 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:06.159 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:30:06.160 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:06.160 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:06.165 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:06.163+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:06.177 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:06.177 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:06.194 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:06.194 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:06.211 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:06.212 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:06.264 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:06.265 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:06.265 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:06.265 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:06.265 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:06.266 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:06.264+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:06.273 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:30:06.366 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:06.364+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:06.377 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:06.378 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:06.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:06.378 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:06.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.378 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5493 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.378 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571177+0000", 2024-09-17T21:30:06.378 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.803749236000002, 2024-09-17T21:30:06.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043037758000000002, 2024-09-17T21:30:06.378 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.378 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.378 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.378 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5493 2024-09-17T21:30:06.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.379 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571177+0000", 2024-09-17T21:30:06.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571177+0000", 2024-09-17T21:30:06.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571178+0000", 2024-09-17T21:30:06.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8800000000000003e-07 2024-09-17T21:30:06.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571180+0000", 2024-09-17T21:30:06.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.17e-06 2024-09-17T21:30:06.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571187+0000", 2024-09-17T21:30:06.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.345e-06 2024-09-17T21:30:06.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571192+0000", 2024-09-17T21:30:06.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8300000000000003e-06 2024-09-17T21:30:06.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.613913+0000", 2024-09-17T21:30:06.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042721071999999999 2024-09-17T21:30:06.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614000+0000", 2024-09-17T21:30:06.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7027000000000006e-05 2024-09-17T21:30:06.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614215+0000", 2024-09-17T21:30:06.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482599999999999 2024-09-17T21:30:06.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.382 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.382 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5495 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.382 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571198+0000", 2024-09-17T21:30:06.383 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.803728027999998, 2024-09-17T21:30:06.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043378574000000003, 2024-09-17T21:30:06.383 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.383 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.383 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.383 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5495 2024-09-17T21:30:06.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.383 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571198+0000", 2024-09-17T21:30:06.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571198+0000", 2024-09-17T21:30:06.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571199+0000", 2024-09-17T21:30:06.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:30:06.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571200+0000", 2024-09-17T21:30:06.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7120000000000001e-06 2024-09-17T21:30:06.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571207+0000", 2024-09-17T21:30:06.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7159999999999997e-06 2024-09-17T21:30:06.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571212+0000", 2024-09-17T21:30:06.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5410000000000002e-06 2024-09-17T21:30:06.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614255+0000", 2024-09-17T21:30:06.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043043658999999998 2024-09-17T21:30:06.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614330+0000", 2024-09-17T21:30:06.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4640000000000004e-05 2024-09-17T21:30:06.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614577+0000", 2024-09-17T21:30:06.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024695400000000002 2024-09-17T21:30:06.386 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.387 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.387 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.387 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5497 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 329728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.387 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571218+0000", 2024-09-17T21:30:06.387 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.803708666999999, 2024-09-17T21:30:06.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043663621999999999, 2024-09-17T21:30:06.387 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.387 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.387 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.387 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.387 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.388 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5497 2024-09-17T21:30:06.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.388 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571218+0000", 2024-09-17T21:30:06.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571218+0000", 2024-09-17T21:30:06.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571218+0000", 2024-09-17T21:30:06.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6199999999999999e-07 2024-09-17T21:30:06.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571220+0000", 2024-09-17T21:30:06.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.655e-06 2024-09-17T21:30:06.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571227+0000", 2024-09-17T21:30:06.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9829999999999999e-06 2024-09-17T21:30:06.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571231+0000", 2024-09-17T21:30:06.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5310000000000003e-06 2024-09-17T21:30:06.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614621+0000", 2024-09-17T21:30:06.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043389469 2024-09-17T21:30:06.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614699+0000", 2024-09-17T21:30:06.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8490000000000002e-05 2024-09-17T21:30:06.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614881+0000", 2024-09-17T21:30:06.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000182132 2024-09-17T21:30:06.391 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.391 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.391 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.391 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5499 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 342016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.391 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571237+0000", 2024-09-17T21:30:06.391 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.803689067000001, 2024-09-17T21:30:06.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043955813000000003, 2024-09-17T21:30:06.391 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.391 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.392 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.392 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.392 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.392 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.392 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5499 2024-09-17T21:30:06.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.392 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571237+0000", 2024-09-17T21:30:06.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571237+0000", 2024-09-17T21:30:06.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571238+0000", 2024-09-17T21:30:06.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:30:06.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571239+0000", 2024-09-17T21:30:06.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:30:06.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571246+0000", 2024-09-17T21:30:06.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8390000000000001e-06 2024-09-17T21:30:06.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571251+0000", 2024-09-17T21:30:06.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4349999999999999e-06 2024-09-17T21:30:06.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614924+0000", 2024-09-17T21:30:06.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043673099999999999 2024-09-17T21:30:06.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.614983+0000", 2024-09-17T21:30:06.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8794e-05 2024-09-17T21:30:06.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615193+0000", 2024-09-17T21:30:06.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000210545 2024-09-17T21:30:06.395 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.395 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.395 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.396 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5501 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 356352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.396 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571257+0000", 2024-09-17T21:30:06.396 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.803669834000001, 2024-09-17T21:30:06.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044214865999999999, 2024-09-17T21:30:06.396 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.396 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.396 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.396 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.396 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.396 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.396 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5501 2024-09-17T21:30:06.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.396 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571257+0000", 2024-09-17T21:30:06.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571257+0000", 2024-09-17T21:30:06.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571257+0000", 2024-09-17T21:30:06.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:30:06.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571259+0000", 2024-09-17T21:30:06.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9460000000000001e-06 2024-09-17T21:30:06.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571266+0000", 2024-09-17T21:30:06.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0439999999999998e-06 2024-09-17T21:30:06.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571271+0000", 2024-09-17T21:30:06.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6500000000000004e-06 2024-09-17T21:30:06.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615238+0000", 2024-09-17T21:30:06.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043967570999999997 2024-09-17T21:30:06.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615293+0000", 2024-09-17T21:30:06.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4857e-05 2024-09-17T21:30:06.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615471+0000", 2024-09-17T21:30:06.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00017844900000000001 2024-09-17T21:30:06.400 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.400 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.400 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.400 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5503 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 370688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.400 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571276+0000", 2024-09-17T21:30:06.400 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.803649955000001, 2024-09-17T21:30:06.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044418055999999997, 2024-09-17T21:30:06.400 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.400 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.400 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.401 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5503 2024-09-17T21:30:06.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.401 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571276+0000", 2024-09-17T21:30:06.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571276+0000", 2024-09-17T21:30:06.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571277+0000", 2024-09-17T21:30:06.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.46e-07 2024-09-17T21:30:06.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571286+0000", 2024-09-17T21:30:06.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8810000000000002e-06 2024-09-17T21:30:06.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571293+0000", 2024-09-17T21:30:06.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5499999999999997e-06 2024-09-17T21:30:06.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571303+0000", 2024-09-17T21:30:06.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5279999999999992e-06 2024-09-17T21:30:06.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615514+0000", 2024-09-17T21:30:06.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044211617000000002 2024-09-17T21:30:06.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615593+0000", 2024-09-17T21:30:06.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8768000000000002e-05 2024-09-17T21:30:06.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615694+0000", 2024-09-17T21:30:06.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101366 2024-09-17T21:30:06.404 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.404 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.404 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.404 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5548 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 194560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.405 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571310+0000", 2024-09-17T21:30:06.405 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.803616866999999, 2024-09-17T21:30:06.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044804920999999998, 2024-09-17T21:30:06.405 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.405 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.405 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.405 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.405 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.405 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.405 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5548 2024-09-17T21:30:06.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.405 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571310+0000", 2024-09-17T21:30:06.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571310+0000", 2024-09-17T21:30:06.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571310+0000", 2024-09-17T21:30:06.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8799999999999998e-07 2024-09-17T21:30:06.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571312+0000", 2024-09-17T21:30:06.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8920000000000001e-06 2024-09-17T21:30:06.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571319+0000", 2024-09-17T21:30:06.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9360000000000002e-06 2024-09-17T21:30:06.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571323+0000", 2024-09-17T21:30:06.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6739999999999996e-06 2024-09-17T21:30:06.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615725+0000", 2024-09-17T21:30:06.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044401949000000003 2024-09-17T21:30:06.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.615801+0000", 2024-09-17T21:30:06.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5150999999999999e-05 2024-09-17T21:30:06.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616114+0000", 2024-09-17T21:30:06.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00031393099999999999 2024-09-17T21:30:06.409 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.409 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.409 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.409 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5551 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 208896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.409 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571329+0000", 2024-09-17T21:30:06.409 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.803596945999999, 2024-09-17T21:30:06.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045083356999999998, 2024-09-17T21:30:06.409 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.409 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.409 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.409 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.409 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.410 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.410 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5551 2024-09-17T21:30:06.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.410 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571329+0000", 2024-09-17T21:30:06.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571329+0000", 2024-09-17T21:30:06.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571330+0000", 2024-09-17T21:30:06.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5499999999999999e-07 2024-09-17T21:30:06.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571331+0000", 2024-09-17T21:30:06.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.691e-06 2024-09-17T21:30:06.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571338+0000", 2024-09-17T21:30:06.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-17T21:30:06.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571343+0000", 2024-09-17T21:30:06.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6260000000000003e-06 2024-09-17T21:30:06.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616174+0000", 2024-09-17T21:30:06.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044831322 2024-09-17T21:30:06.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616224+0000", 2024-09-17T21:30:06.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9950000000000001e-05 2024-09-17T21:30:06.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616413+0000", 2024-09-17T21:30:06.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018850400000000001 2024-09-17T21:30:06.413 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.413 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.413 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.413 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5553 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.413 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571349+0000", 2024-09-17T21:30:06.414 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.803577416, 2024-09-17T21:30:06.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045359177000000001, 2024-09-17T21:30:06.414 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.414 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.414 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.414 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5553 2024-09-17T21:30:06.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.414 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571349+0000", 2024-09-17T21:30:06.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571349+0000", 2024-09-17T21:30:06.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571349+0000", 2024-09-17T21:30:06.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.15e-07 2024-09-17T21:30:06.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571351+0000", 2024-09-17T21:30:06.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6750000000000001e-06 2024-09-17T21:30:06.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571360+0000", 2024-09-17T21:30:06.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6270000000000006e-06 2024-09-17T21:30:06.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571364+0000", 2024-09-17T21:30:06.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7029999999999997e-06 2024-09-17T21:30:06.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616459+0000", 2024-09-17T21:30:06.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045094672000000002 2024-09-17T21:30:06.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616503+0000", 2024-09-17T21:30:06.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4032000000000001e-05 2024-09-17T21:30:06.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616708+0000", 2024-09-17T21:30:06.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020515299999999999 2024-09-17T21:30:06.420 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.420 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.420 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.420 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5555 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 235520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.421 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571371+0000", 2024-09-17T21:30:06.421 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.803555836000001, 2024-09-17T21:30:06.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045673013999999998, 2024-09-17T21:30:06.421 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.421 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.421 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.421 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.421 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.421 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.421 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5555 2024-09-17T21:30:06.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.421 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571371+0000", 2024-09-17T21:30:06.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571371+0000", 2024-09-17T21:30:06.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571371+0000", 2024-09-17T21:30:06.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9400000000000001e-07 2024-09-17T21:30:06.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571373+0000", 2024-09-17T21:30:06.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.714e-06 2024-09-17T21:30:06.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571379+0000", 2024-09-17T21:30:06.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0449999999999997e-06 2024-09-17T21:30:06.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571383+0000", 2024-09-17T21:30:06.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.963e-06 2024-09-17T21:30:06.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616752+0000", 2024-09-17T21:30:06.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045369481000000003 2024-09-17T21:30:06.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.616828+0000", 2024-09-17T21:30:06.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5961e-05 2024-09-17T21:30:06.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617044+0000", 2024-09-17T21:30:06.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021545599999999999 2024-09-17T21:30:06.425 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.425 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.425 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.425 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5556 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.425 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571392+0000", 2024-09-17T21:30:06.425 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.803533977000001, 2024-09-17T21:30:06.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045971310000000001, 2024-09-17T21:30:06.425 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.425 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.425 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.425 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.426 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.426 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.426 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5556 2024-09-17T21:30:06.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.426 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571392+0000", 2024-09-17T21:30:06.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571392+0000", 2024-09-17T21:30:06.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571393+0000", 2024-09-17T21:30:06.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1399999999999997e-07 2024-09-17T21:30:06.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571395+0000", 2024-09-17T21:30:06.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.956e-06 2024-09-17T21:30:06.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571401+0000", 2024-09-17T21:30:06.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9089999999999997e-06 2024-09-17T21:30:06.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571405+0000", 2024-09-17T21:30:06.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6410000000000001e-06 2024-09-17T21:30:06.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617070+0000", 2024-09-17T21:30:06.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045664874000000001 2024-09-17T21:30:06.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617130+0000", 2024-09-17T21:30:06.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9756999999999997e-05 2024-09-17T21:30:06.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617364+0000", 2024-09-17T21:30:06.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000233759 2024-09-17T21:30:06.429 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.429 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.429 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.429 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5559 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 264192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.429 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571411+0000", 2024-09-17T21:30:06.429 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.803514968000002, 2024-09-17T21:30:06.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046332908999999999, 2024-09-17T21:30:06.429 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.430 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.430 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.430 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.430 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.430 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.430 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5559 2024-09-17T21:30:06.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.430 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571411+0000", 2024-09-17T21:30:06.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571411+0000", 2024-09-17T21:30:06.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571412+0000", 2024-09-17T21:30:06.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.65e-07 2024-09-17T21:30:06.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571419+0000", 2024-09-17T21:30:06.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2899999999999997e-06 2024-09-17T21:30:06.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571425+0000", 2024-09-17T21:30:06.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3400000000000003e-06 2024-09-17T21:30:06.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571430+0000", 2024-09-17T21:30:06.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4499999999999997e-06 2024-09-17T21:30:06.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617455+0000", 2024-09-17T21:30:06.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046024661000000001 2024-09-17T21:30:06.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617532+0000", 2024-09-17T21:30:06.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7712000000000004e-05 2024-09-17T21:30:06.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617744+0000", 2024-09-17T21:30:06.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000212091 2024-09-17T21:30:06.433 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.433 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.433 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.434 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5561 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 276480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.434 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571436+0000", 2024-09-17T21:30:06.434 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.803490179000001, 2024-09-17T21:30:06.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046615325999999999, 2024-09-17T21:30:06.434 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.434 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.434 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.434 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.434 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.434 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.434 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5561 2024-09-17T21:30:06.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.434 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571436+0000", 2024-09-17T21:30:06.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571436+0000", 2024-09-17T21:30:06.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571437+0000", 2024-09-17T21:30:06.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:30:06.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571439+0000", 2024-09-17T21:30:06.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9130000000000001e-06 2024-09-17T21:30:06.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571444+0000", 2024-09-17T21:30:06.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5110000000000003e-06 2024-09-17T21:30:06.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571448+0000", 2024-09-17T21:30:06.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3749999999999996e-06 2024-09-17T21:30:06.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617775+0000", 2024-09-17T21:30:06.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046326668000000001 2024-09-17T21:30:06.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.617851+0000", 2024-09-17T21:30:06.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6192000000000005e-05 2024-09-17T21:30:06.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618052+0000", 2024-09-17T21:30:06.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020022900000000001 2024-09-17T21:30:06.438 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.438 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.438 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.438 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5563 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 290816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.438 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571454+0000", 2024-09-17T21:30:06.438 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.803471901999998, 2024-09-17T21:30:06.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046975959999999997, 2024-09-17T21:30:06.438 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.438 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.438 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.438 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.439 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.439 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.439 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5563 2024-09-17T21:30:06.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.439 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571454+0000", 2024-09-17T21:30:06.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571454+0000", 2024-09-17T21:30:06.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571455+0000", 2024-09-17T21:30:06.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:30:06.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571456+0000", 2024-09-17T21:30:06.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.64e-06 2024-09-17T21:30:06.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571462+0000", 2024-09-17T21:30:06.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6230000000000002e-06 2024-09-17T21:30:06.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571467+0000", 2024-09-17T21:30:06.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5199999999999999e-06 2024-09-17T21:30:06.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618091+0000", 2024-09-17T21:30:06.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046624489999999998 2024-09-17T21:30:06.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618172+0000", 2024-09-17T21:30:06.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1271000000000002e-05 2024-09-17T21:30:06.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618430+0000", 2024-09-17T21:30:06.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025807199999999998 2024-09-17T21:30:06.442 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.442 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.442 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.442 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5566 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.443 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571472+0000", 2024-09-17T21:30:06.443 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.803454008999999, 2024-09-17T21:30:06.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047328261000000003, 2024-09-17T21:30:06.443 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.443 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.443 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.443 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5566 2024-09-17T21:30:06.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.443 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571472+0000", 2024-09-17T21:30:06.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571472+0000", 2024-09-17T21:30:06.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571473+0000", 2024-09-17T21:30:06.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:30:06.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571474+0000", 2024-09-17T21:30:06.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.668e-06 2024-09-17T21:30:06.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571480+0000", 2024-09-17T21:30:06.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6899999999999997e-06 2024-09-17T21:30:06.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571489+0000", 2024-09-17T21:30:06.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8589999999999994e-06 2024-09-17T21:30:06.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618481+0000", 2024-09-17T21:30:06.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046992468000000003 2024-09-17T21:30:06.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618563+0000", 2024-09-17T21:30:06.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1464000000000005e-05 2024-09-17T21:30:06.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618801+0000", 2024-09-17T21:30:06.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000237762 2024-09-17T21:30:06.447 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.447 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.447 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.447 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5568 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 317440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.447 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571495+0000", 2024-09-17T21:30:06.447 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.803431522, 2024-09-17T21:30:06.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047753012999999997, 2024-09-17T21:30:06.447 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.448 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.448 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.448 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.448 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.448 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.448 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5568 2024-09-17T21:30:06.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.448 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571495+0000", 2024-09-17T21:30:06.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571495+0000", 2024-09-17T21:30:06.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571495+0000", 2024-09-17T21:30:06.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.89e-07 2024-09-17T21:30:06.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571497+0000", 2024-09-17T21:30:06.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.739e-06 2024-09-17T21:30:06.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571503+0000", 2024-09-17T21:30:06.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:30:06.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571507+0000", 2024-09-17T21:30:06.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3560000000000003e-06 2024-09-17T21:30:06.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618877+0000", 2024-09-17T21:30:06.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047370019999999999 2024-09-17T21:30:06.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.618964+0000", 2024-09-17T21:30:06.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6615999999999999e-05 2024-09-17T21:30:06.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619248+0000", 2024-09-17T21:30:06.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000284154 2024-09-17T21:30:06.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.451 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.451 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5570 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 331776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.451 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571513+0000", 2024-09-17T21:30:06.452 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.80341366, 2024-09-17T21:30:06.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048061883999999999, 2024-09-17T21:30:06.452 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.452 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.452 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.452 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5570 2024-09-17T21:30:06.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.452 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571513+0000", 2024-09-17T21:30:06.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571513+0000", 2024-09-17T21:30:06.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571513+0000", 2024-09-17T21:30:06.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4700000000000002e-07 2024-09-17T21:30:06.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571518+0000", 2024-09-17T21:30:06.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.242e-06 2024-09-17T21:30:06.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571874+0000", 2024-09-17T21:30:06.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00035530799999999998 2024-09-17T21:30:06.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571884+0000", 2024-09-17T21:30:06.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0733e-05 2024-09-17T21:30:06.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619283+0000", 2024-09-17T21:30:06.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047398218999999998 2024-09-17T21:30:06.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619361+0000", 2024-09-17T21:30:06.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8660999999999998e-05 2024-09-17T21:30:06.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619575+0000", 2024-09-17T21:30:06.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021337400000000001 2024-09-17T21:30:06.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.456 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.456 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5572 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 346112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.456 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571897+0000", 2024-09-17T21:30:06.456 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.803029709, 2024-09-17T21:30:06.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048019328999999999, 2024-09-17T21:30:06.456 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.456 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.456 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.456 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.457 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5572 2024-09-17T21:30:06.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.457 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571897+0000", 2024-09-17T21:30:06.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571897+0000", 2024-09-17T21:30:06.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571897+0000", 2024-09-17T21:30:06.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8100000000000002e-07 2024-09-17T21:30:06.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571901+0000", 2024-09-17T21:30:06.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4369999999999999e-06 2024-09-17T21:30:06.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571909+0000", 2024-09-17T21:30:06.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.841e-06 2024-09-17T21:30:06.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571915+0000", 2024-09-17T21:30:06.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5130000000000002e-06 2024-09-17T21:30:06.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619619+0000", 2024-09-17T21:30:06.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047704203000000001 2024-09-17T21:30:06.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619698+0000", 2024-09-17T21:30:06.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8656e-05 2024-09-17T21:30:06.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619916+0000", 2024-09-17T21:30:06.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021789800000000001 2024-09-17T21:30:06.460 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.460 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.460 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.460 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5574 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 358400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.460 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571923+0000", 2024-09-17T21:30:06.460 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.803003679, 2024-09-17T21:30:06.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048336208999999998, 2024-09-17T21:30:06.460 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.460 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.461 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.461 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5574 2024-09-17T21:30:06.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.461 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571923+0000", 2024-09-17T21:30:06.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571923+0000", 2024-09-17T21:30:06.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571923+0000", 2024-09-17T21:30:06.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9900000000000001e-07 2024-09-17T21:30:06.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571925+0000", 2024-09-17T21:30:06.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.978e-06 2024-09-17T21:30:06.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571931+0000", 2024-09-17T21:30:06.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2530000000000001e-06 2024-09-17T21:30:06.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571936+0000", 2024-09-17T21:30:06.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6700000000000002e-06 2024-09-17T21:30:06.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.619957+0000", 2024-09-17T21:30:06.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048020500000000001 2024-09-17T21:30:06.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.620037+0000", 2024-09-17T21:30:06.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0016000000000001e-05 2024-09-17T21:30:06.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.620259+0000", 2024-09-17T21:30:06.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022239300000000001 2024-09-17T21:30:06.464 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.464 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.464 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.465 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:5576 3.19 3:9e4fea6b:::rbd_data.11202c1b6cb4.0000000000000003:4 [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:06.465 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:48.571944+0000", 2024-09-17T21:30:06.465 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.802982553, 2024-09-17T21:30:06.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048513909000000001, 2024-09-17T21:30:06.465 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:06.465 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:06.465 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:06.465 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:06.465 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:06.465 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:06.465 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5576 2024-09-17T21:30:06.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.465 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:06.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:06.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571944+0000", 2024-09-17T21:30:06.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:06.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571944+0000", 2024-09-17T21:30:06.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:06.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:06.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571952+0000", 2024-09-17T21:30:06.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7729999999999999e-06 2024-09-17T21:30:06.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:06.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571954+0000", 2024-09-17T21:30:06.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8310000000000001e-06 2024-09-17T21:30:06.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:06.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571964+0000", 2024-09-17T21:30:06.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4450000000000002e-06 2024-09-17T21:30:06.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:06.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.571976+0000", 2024-09-17T21:30:06.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1793999999999999e-05 2024-09-17T21:30:06.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:06.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.620299+0000", 2024-09-17T21:30:06.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048323558000000003 2024-09-17T21:30:06.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:06.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.620373+0000", 2024-09-17T21:30:06.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4262000000000002e-05 2024-09-17T21:30:06.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:06.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:06.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:06.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:48.620458+0000", 2024-09-17T21:30:06.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4246000000000006e-05 2024-09-17T21:30:06.468 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.468 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.469 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.469 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:06.469 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:06.469 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:06.470 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:30:06.471 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:06.468+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:06.570 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:06.568+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:06.600 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:30:06.623 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:06.623 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:06.623 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:06.623 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:06.632 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:30:06.673 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:06.671+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:06.689 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone12 from 0xcac9e07 (overlap 0xbc47225) to 0x0 2024-09-17T21:30:06.772 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:06.769+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:30:06.836 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:06.836 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:06.837 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:06.837 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:06.837 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:06.852 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:30:06.892 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:06.890+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:06.973 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:06.971+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:07.063 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:07.063 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:07.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:07.063 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:07.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.063 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:34547 3.4 e34/25 3:24c07f74:::rbd_object_map.112071f51ee8:head v 34'29, mlcod=34'27)", 2024-09-17T21:30:07.063 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:52.585792+0000", 2024-09-17T21:30:07.063 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.474113901999999, 2024-09-17T21:30:07.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0032798810000000001, 2024-09-17T21:30:07.063 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.064 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.064 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:07.064 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.585792+0000", 2024-09-17T21:30:07.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.585792+0000", 2024-09-17T21:30:07.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.585793+0000", 2024-09-17T21:30:07.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8899999999999998e-07 2024-09-17T21:30:07.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.585799+0000", 2024-09-17T21:30:07.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2929999999999997e-06 2024-09-17T21:30:07.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.585812+0000", 2024-09-17T21:30:07.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2276000000000001e-05 2024-09-17T21:30:07.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.585819+0000", 2024-09-17T21:30:07.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9979999999999997e-06 2024-09-17T21:30:07.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.585868+0000", 2024-09-17T21:30:07.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9750999999999999e-05 2024-09-17T21:30:07.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.585891+0000", 2024-09-17T21:30:07.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2149000000000001e-05 2024-09-17T21:30:07.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:07.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.589040+0000", 2024-09-17T21:30:07.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0031496929999999999 2024-09-17T21:30:07.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:52.589072+0000", 2024-09-17T21:30:07.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1931999999999999e-05 2024-09-17T21:30:07.067 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.067 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.067 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.068 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:37887 3.1f e41/37 3:fc790e8e:::rbd_data.1120ab27bfd0.0000000000000054:head v 41'27, mlcod=39'25)", 2024-09-17T21:30:07.068 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:02.873463+0000", 2024-09-17T21:30:07.068 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.1864433420000005, 2024-09-17T21:30:07.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0029774240000000002, 2024-09-17T21:30:07.068 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.068 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.068 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:07.068 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.873463+0000", 2024-09-17T21:30:07.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.873463+0000", 2024-09-17T21:30:07.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.873464+0000", 2024-09-17T21:30:07.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1399999999999995e-07 2024-09-17T21:30:07.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.873471+0000", 2024-09-17T21:30:07.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0310000000000001e-06 2024-09-17T21:30:07.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.873485+0000", 2024-09-17T21:30:07.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4552999999999999e-05 2024-09-17T21:30:07.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.873493+0000", 2024-09-17T21:30:07.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3039999999999999e-06 2024-09-17T21:30:07.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.873560+0000", 2024-09-17T21:30:07.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7136000000000002e-05 2024-09-17T21:30:07.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.873574+0000", 2024-09-17T21:30:07.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4459e-05 2024-09-17T21:30:07.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:07.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.876404+0000", 2024-09-17T21:30:07.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0028292870000000002 2024-09-17T21:30:07.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.876440+0000", 2024-09-17T21:30:07.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6739999999999997e-05 2024-09-17T21:30:07.072 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.072 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.072 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.073 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:38400 3.0 e41/25 3:050aa3f8:::rbd_data.1120ab27bfd0.0000000000000001:head v 41'16, mlcod=41'14)", 2024-09-17T21:30:07.073 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:02.918196+0000", 2024-09-17T21:30:07.073 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.1417099339999996, 2024-09-17T21:30:07.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0028826490000000001, 2024-09-17T21:30:07.073 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.073 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.073 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:07.073 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.918196+0000", 2024-09-17T21:30:07.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.918196+0000", 2024-09-17T21:30:07.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.918198+0000", 2024-09-17T21:30:07.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.928e-06 2024-09-17T21:30:07.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.919127+0000", 2024-09-17T21:30:07.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00092894600000000005 2024-09-17T21:30:07.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.919152+0000", 2024-09-17T21:30:07.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4674000000000001e-05 2024-09-17T21:30:07.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.919163+0000", 2024-09-17T21:30:07.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1613999999999999e-05 2024-09-17T21:30:07.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.919223+0000", 2024-09-17T21:30:07.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9682999999999998e-05 2024-09-17T21:30:07.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.919247+0000", 2024-09-17T21:30:07.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3583000000000001e-05 2024-09-17T21:30:07.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:07.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.921040+0000", 2024-09-17T21:30:07.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0017930559999999999 2024-09-17T21:30:07.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.921079+0000", 2024-09-17T21:30:07.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9165000000000002e-05 2024-09-17T21:30:07.077 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.077 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.077 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.078 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:39715 3.0 e42/25 3:050aa3f8:::rbd_data.1120ab27bfd0.0000000000000001:head v 42'21, mlcod=41'18)", 2024-09-17T21:30:07.078 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:03.634569+0000", 2024-09-17T21:30:07.078 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.4253371260000001, 2024-09-17T21:30:07.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0044088290000000004, 2024-09-17T21:30:07.078 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.078 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.078 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:07.078 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.634569+0000", 2024-09-17T21:30:07.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.634569+0000", 2024-09-17T21:30:07.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.634571+0000", 2024-09-17T21:30:07.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5319999999999999e-06 2024-09-17T21:30:07.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.634583+0000", 2024-09-17T21:30:07.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.261e-05 2024-09-17T21:30:07.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.634603+0000", 2024-09-17T21:30:07.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9828999999999999e-05 2024-09-17T21:30:07.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.634613+0000", 2024-09-17T21:30:07.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5389999999999996e-06 2024-09-17T21:30:07.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.636599+0000", 2024-09-17T21:30:07.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0019867890000000001 2024-09-17T21:30:07.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.636621+0000", 2024-09-17T21:30:07.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2110999999999999e-05 2024-09-17T21:30:07.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:07.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.638927+0000", 2024-09-17T21:30:07.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00230532 2024-09-17T21:30:07.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.638978+0000", 2024-09-17T21:30:07.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1099e-05 2024-09-17T21:30:07.082 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.082 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.082 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.082 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:40209 3.1f e42/37 3:f8f84c17:::rbd_data.1120ab27bfd0.000000000000000f:head v 42'32, mlcod=41'29)", 2024-09-17T21:30:07.082 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:03.653656+0000", 2024-09-17T21:30:07.083 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.406249957, 2024-09-17T21:30:07.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0030197269999999998, 2024-09-17T21:30:07.083 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.083 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.083 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:07.083 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.653656+0000", 2024-09-17T21:30:07.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.653656+0000", 2024-09-17T21:30:07.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.653658+0000", 2024-09-17T21:30:07.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8980000000000001e-06 2024-09-17T21:30:07.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.653671+0000", 2024-09-17T21:30:07.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2642e-05 2024-09-17T21:30:07.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.653693+0000", 2024-09-17T21:30:07.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1987000000000001e-05 2024-09-17T21:30:07.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.653704+0000", 2024-09-17T21:30:07.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0890000000000001e-05 2024-09-17T21:30:07.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.653792+0000", 2024-09-17T21:30:07.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7892000000000005e-05 2024-09-17T21:30:07.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.653814+0000", 2024-09-17T21:30:07.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2218999999999999e-05 2024-09-17T21:30:07.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:07.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.656635+0000", 2024-09-17T21:30:07.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0028211400000000002 2024-09-17T21:30:07.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.656676+0000", 2024-09-17T21:30:07.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1059000000000001e-05 2024-09-17T21:30:07.087 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.087 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.087 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.087 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:45801 3.4 e45/25 3:2431819f:::rbd_data.1120b21c3309.0000000000000000:head v 45'118, mlcod=44'116)", 2024-09-17T21:30:07.087 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.735946+0000", 2024-09-17T21:30:07.087 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.32396046899999997, 2024-09-17T21:30:07.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0054180599999999997, 2024-09-17T21:30:07.087 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.087 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.087 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:07.087 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.735946+0000", 2024-09-17T21:30:07.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.735946+0000", 2024-09-17T21:30:07.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.735947+0000", 2024-09-17T21:30:07.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.477e-06 2024-09-17T21:30:07.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.736212+0000", 2024-09-17T21:30:07.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026511299999999998 2024-09-17T21:30:07.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.736235+0000", 2024-09-17T21:30:07.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2742e-05 2024-09-17T21:30:07.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.736245+0000", 2024-09-17T21:30:07.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0172e-05 2024-09-17T21:30:07.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.736328+0000", 2024-09-17T21:30:07.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2898000000000005e-05 2024-09-17T21:30:07.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.736349+0000", 2024-09-17T21:30:07.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1046000000000001e-05 2024-09-17T21:30:07.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:07.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.741303+0000", 2024-09-17T21:30:07.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0049536980000000003 2024-09-17T21:30:07.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.741364+0000", 2024-09-17T21:30:07.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0914e-05 2024-09-17T21:30:07.091 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.091 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.091 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.091 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:45802 3.4 e45/25 3:2431819f:::rbd_data.1120b21c3309.0000000000000000:head v 45'120, mlcod=44'116)", 2024-09-17T21:30:07.091 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.738702+0000", 2024-09-17T21:30:07.092 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.32120394499999999, 2024-09-17T21:30:07.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0032917089999999999, 2024-09-17T21:30:07.092 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.092 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.092 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:07.092 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.738702+0000", 2024-09-17T21:30:07.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.738702+0000", 2024-09-17T21:30:07.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.738703+0000", 2024-09-17T21:30:07.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.037e-06 2024-09-17T21:30:07.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.738712+0000", 2024-09-17T21:30:07.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7690000000000002e-06 2024-09-17T21:30:07.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.738724+0000", 2024-09-17T21:30:07.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1552999999999999e-05 2024-09-17T21:30:07.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.738732+0000", 2024-09-17T21:30:07.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6209999999999993e-06 2024-09-17T21:30:07.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.738807+0000", 2024-09-17T21:30:07.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4474999999999994e-05 2024-09-17T21:30:07.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.738828+0000", 2024-09-17T21:30:07.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1047e-05 2024-09-17T21:30:07.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:07.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.741950+0000", 2024-09-17T21:30:07.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0031222200000000002 2024-09-17T21:30:07.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.741994+0000", 2024-09-17T21:30:07.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3986999999999997e-05 2024-09-17T21:30:07.096 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.096 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.096 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.096 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:45807 3.9 e45/25 3:9731cab8:::rbd_data.1120b21c3309.0000000000000005:head v 45'23, mlcod=44'21)", 2024-09-17T21:30:07.096 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.742709+0000", 2024-09-17T21:30:07.096 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.31719681999999999, 2024-09-17T21:30:07.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039918530000000001, 2024-09-17T21:30:07.096 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.096 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.096 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:07.097 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.742709+0000", 2024-09-17T21:30:07.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.742709+0000", 2024-09-17T21:30:07.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.742710+0000", 2024-09-17T21:30:07.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0809999999999999e-06 2024-09-17T21:30:07.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.743492+0000", 2024-09-17T21:30:07.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00078200199999999996 2024-09-17T21:30:07.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.743512+0000", 2024-09-17T21:30:07.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9602999999999998e-05 2024-09-17T21:30:07.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.743521+0000", 2024-09-17T21:30:07.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2250000000000006e-06 2024-09-17T21:30:07.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.743596+0000", 2024-09-17T21:30:07.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4238000000000003e-05 2024-09-17T21:30:07.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.743616+0000", 2024-09-17T21:30:07.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0446e-05 2024-09-17T21:30:07.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:07.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.746660+0000", 2024-09-17T21:30:07.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0030443219999999999 2024-09-17T21:30:07.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.746701+0000", 2024-09-17T21:30:07.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0936000000000002e-05 2024-09-17T21:30:07.100 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.100 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.100 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.101 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:46866 3.1f e45/37 3:fdf24e33:::rbd_data.1120b21c3309.000000000000000d:head v 45'274, mlcod=44'272)", 2024-09-17T21:30:07.101 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.795667+0000", 2024-09-17T21:30:07.101 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.26423908499999998, 2024-09-17T21:30:07.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036764810000000001, 2024-09-17T21:30:07.101 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.101 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.101 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:07.101 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.795667+0000", 2024-09-17T21:30:07.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.795667+0000", 2024-09-17T21:30:07.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.795669+0000", 2024-09-17T21:30:07.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6220000000000001e-06 2024-09-17T21:30:07.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.795728+0000", 2024-09-17T21:30:07.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9378e-05 2024-09-17T21:30:07.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.795746+0000", 2024-09-17T21:30:07.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8332e-05 2024-09-17T21:30:07.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.795756+0000", 2024-09-17T21:30:07.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1330000000000005e-06 2024-09-17T21:30:07.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.795839+0000", 2024-09-17T21:30:07.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3092999999999999e-05 2024-09-17T21:30:07.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.795860+0000", 2024-09-17T21:30:07.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1183e-05 2024-09-17T21:30:07.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:07.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.799295+0000", 2024-09-17T21:30:07.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003434721 2024-09-17T21:30:07.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.799344+0000", 2024-09-17T21:30:07.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9018999999999999e-05 2024-09-17T21:30:07.105 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.105 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.105 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.105 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:47375 3.9 e45/25 3:9491c5fc:::rbd_data.1120b21c3309.0000000000000019:head v 45'27, mlcod=45'25)", 2024-09-17T21:30:07.105 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.830179+0000", 2024-09-17T21:30:07.105 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.22972674700000001, 2024-09-17T21:30:07.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0037068410000000002, 2024-09-17T21:30:07.105 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.105 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.105 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:07.106 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.830179+0000", 2024-09-17T21:30:07.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.830179+0000", 2024-09-17T21:30:07.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.830181+0000", 2024-09-17T21:30:07.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.463e-06 2024-09-17T21:30:07.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.830231+0000", 2024-09-17T21:30:07.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0447e-05 2024-09-17T21:30:07.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.830251+0000", 2024-09-17T21:30:07.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9681999999999999e-05 2024-09-17T21:30:07.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.830260+0000", 2024-09-17T21:30:07.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0569999999999999e-06 2024-09-17T21:30:07.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.830319+0000", 2024-09-17T21:30:07.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8615e-05 2024-09-17T21:30:07.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.830340+0000", 2024-09-17T21:30:07.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1243e-05 2024-09-17T21:30:07.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:07.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.833845+0000", 2024-09-17T21:30:07.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0035048950000000001 2024-09-17T21:30:07.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.833886+0000", 2024-09-17T21:30:07.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1439000000000001e-05 2024-09-17T21:30:07.109 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.109 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.109 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.110 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:47424 3.4 e45/25 3:23621007:::rbd_data.1120b21c3309.000000000000001f:head v 45'126, mlcod=45'123)", 2024-09-17T21:30:07.110 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.835534+0000", 2024-09-17T21:30:07.110 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.224371979, 2024-09-17T21:30:07.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038193630000000001, 2024-09-17T21:30:07.110 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.110 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.110 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:07.110 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.835534+0000", 2024-09-17T21:30:07.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.835534+0000", 2024-09-17T21:30:07.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.835536+0000", 2024-09-17T21:30:07.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3009999999999999e-06 2024-09-17T21:30:07.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.835567+0000", 2024-09-17T21:30:07.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1130999999999997e-05 2024-09-17T21:30:07.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.835582+0000", 2024-09-17T21:30:07.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5566e-05 2024-09-17T21:30:07.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.835590+0000", 2024-09-17T21:30:07.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0150000000000003e-06 2024-09-17T21:30:07.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.835675+0000", 2024-09-17T21:30:07.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4868000000000001e-05 2024-09-17T21:30:07.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.835694+0000", 2024-09-17T21:30:07.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9154999999999999e-05 2024-09-17T21:30:07.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:07.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.839306+0000", 2024-09-17T21:30:07.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003611923 2024-09-17T21:30:07.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.839354+0000", 2024-09-17T21:30:07.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7404000000000002e-05 2024-09-17T21:30:07.114 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.114 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.114 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.114 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48021 3.5 e45/37 3:a6441874:::rbd_data.1120b21c3309.0000000000000039:head v 45'62, mlcod=44'60)", 2024-09-17T21:30:07.114 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.876760+0000", 2024-09-17T21:30:07.115 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.18314603099999999, 2024-09-17T21:30:07.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045026060000000001, 2024-09-17T21:30:07.115 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.115 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.115 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:07.115 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.876760+0000", 2024-09-17T21:30:07.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.876760+0000", 2024-09-17T21:30:07.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.876761+0000", 2024-09-17T21:30:07.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.85e-07 2024-09-17T21:30:07.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.876798+0000", 2024-09-17T21:30:07.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7366000000000001e-05 2024-09-17T21:30:07.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.876812+0000", 2024-09-17T21:30:07.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.366e-05 2024-09-17T21:30:07.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.876819+0000", 2024-09-17T21:30:07.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3669999999999999e-06 2024-09-17T21:30:07.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.876873+0000", 2024-09-17T21:30:07.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3650000000000003e-05 2024-09-17T21:30:07.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.876893+0000", 2024-09-17T21:30:07.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0349000000000001e-05 2024-09-17T21:30:07.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:07.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.881218+0000", 2024-09-17T21:30:07.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043242339999999997 2024-09-17T21:30:07.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.881263+0000", 2024-09-17T21:30:07.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5095000000000001e-05 2024-09-17T21:30:07.119 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.119 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.119 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.119 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48022 3.5 e45/37 3:a6441874:::rbd_data.1120b21c3309.0000000000000039:head v 45'64, mlcod=44'60)", 2024-09-17T21:30:07.119 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.877803+0000", 2024-09-17T21:30:07.119 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.18210291100000001, 2024-09-17T21:30:07.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063935559999999999, 2024-09-17T21:30:07.119 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.119 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.119 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:07.119 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.877803+0000", 2024-09-17T21:30:07.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.877803+0000", 2024-09-17T21:30:07.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.877804+0000", 2024-09-17T21:30:07.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3399999999999998e-07 2024-09-17T21:30:07.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.877810+0000", 2024-09-17T21:30:07.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6790000000000002e-06 2024-09-17T21:30:07.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.877821+0000", 2024-09-17T21:30:07.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.113e-05 2024-09-17T21:30:07.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.877827+0000", 2024-09-17T21:30:07.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3589999999999996e-06 2024-09-17T21:30:07.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.877877+0000", 2024-09-17T21:30:07.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9985000000000003e-05 2024-09-17T21:30:07.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.877898+0000", 2024-09-17T21:30:07.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0523e-05 2024-09-17T21:30:07.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:07.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.884142+0000", 2024-09-17T21:30:07.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0062445039999999997 2024-09-17T21:30:07.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.884197+0000", 2024-09-17T21:30:07.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4542000000000001e-05 2024-09-17T21:30:07.123 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.123 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.123 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.123 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "pg_update_log_missing(3.4 epoch 45/25 rep_tid 240 entries 45'130 (0'0) error 3:203aa520:::rbd_data.1120b21c3309.0000000000000040:head by client.4384.0:48055 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 45'128)", 2024-09-17T21:30:07.124 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.880843+0000", 2024-09-17T21:30:07.124 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.17906288200000001, 2024-09-17T21:30:07.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0032959679999999998, 2024-09-17T21:30:07.124 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.124 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.124 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T21:30:07.124 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.880843+0000", 2024-09-17T21:30:07.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.880843+0000", 2024-09-17T21:30:07.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.880844+0000", 2024-09-17T21:30:07.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9999999999999996e-07 2024-09-17T21:30:07.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.880853+0000", 2024-09-17T21:30:07.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.568e-06 2024-09-17T21:30:07.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.880873+0000", 2024-09-17T21:30:07.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9941000000000001e-05 2024-09-17T21:30:07.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.880888+0000", 2024-09-17T21:30:07.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4831000000000001e-05 2024-09-17T21:30:07.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.880982+0000", 2024-09-17T21:30:07.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4376000000000006e-05 2024-09-17T21:30:07.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.884139+0000", 2024-09-17T21:30:07.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003157352 2024-09-17T21:30:07.127 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.127 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.127 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.127 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48295 3.9 e45/25 3:97228dae:::rbd_data.1120b21c3309.000000000000004f:head v 45'31, mlcod=45'29)", 2024-09-17T21:30:07.127 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.901458+0000", 2024-09-17T21:30:07.127 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.15844818699999999, 2024-09-17T21:30:07.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038023079999999999, 2024-09-17T21:30:07.128 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.128 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.128 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:07.128 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.901458+0000", 2024-09-17T21:30:07.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.901458+0000", 2024-09-17T21:30:07.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.901459+0000", 2024-09-17T21:30:07.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0330000000000001e-06 2024-09-17T21:30:07.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.901466+0000", 2024-09-17T21:30:07.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2370000000000003e-06 2024-09-17T21:30:07.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.901482+0000", 2024-09-17T21:30:07.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5264000000000002e-05 2024-09-17T21:30:07.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.901489+0000", 2024-09-17T21:30:07.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0450000000000003e-06 2024-09-17T21:30:07.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.901555+0000", 2024-09-17T21:30:07.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6093000000000006e-05 2024-09-17T21:30:07.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.901574+0000", 2024-09-17T21:30:07.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9281999999999999e-05 2024-09-17T21:30:07.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:07.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.905205+0000", 2024-09-17T21:30:07.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036306300000000001 2024-09-17T21:30:07.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.905260+0000", 2024-09-17T21:30:07.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5723999999999997e-05 2024-09-17T21:30:07.131 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.131 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.131 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.132 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48297 3.1f e45/37 3:fb756001:::rbd_data.1120b21c3309.0000000000000054:head v 45'282, mlcod=45'280)", 2024-09-17T21:30:07.132 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.902030+0000", 2024-09-17T21:30:07.132 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.157876403, 2024-09-17T21:30:07.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0032221699999999999, 2024-09-17T21:30:07.132 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.132 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.132 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:07.132 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.902030+0000", 2024-09-17T21:30:07.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.902030+0000", 2024-09-17T21:30:07.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.902031+0000", 2024-09-17T21:30:07.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.16e-07 2024-09-17T21:30:07.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.902037+0000", 2024-09-17T21:30:07.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:30:07.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.902051+0000", 2024-09-17T21:30:07.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4165000000000001e-05 2024-09-17T21:30:07.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.902058+0000", 2024-09-17T21:30:07.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0849999999999999e-06 2024-09-17T21:30:07.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.902109+0000", 2024-09-17T21:30:07.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1258999999999998e-05 2024-09-17T21:30:07.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.902130+0000", 2024-09-17T21:30:07.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0854000000000001e-05 2024-09-17T21:30:07.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:07.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.905218+0000", 2024-09-17T21:30:07.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0030881799999999998 2024-09-17T21:30:07.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.905252+0000", 2024-09-17T21:30:07.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3717000000000003e-05 2024-09-17T21:30:07.136 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.136 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.136 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.136 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48304 3.0 e45/25 3:07d2aa03:::rbd_data.1120b21c3309.0000000000000056:head v 45'33, mlcod=43'31)", 2024-09-17T21:30:07.136 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.904321+0000", 2024-09-17T21:30:07.136 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.15558566300000001, 2024-09-17T21:30:07.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0041472260000000004, 2024-09-17T21:30:07.136 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.137 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.137 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:07.137 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904321+0000", 2024-09-17T21:30:07.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904321+0000", 2024-09-17T21:30:07.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904321+0000", 2024-09-17T21:30:07.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8899999999999998e-07 2024-09-17T21:30:07.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904327+0000", 2024-09-17T21:30:07.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4380000000000003e-06 2024-09-17T21:30:07.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904338+0000", 2024-09-17T21:30:07.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1131999999999999e-05 2024-09-17T21:30:07.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904345+0000", 2024-09-17T21:30:07.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7089999999999996e-06 2024-09-17T21:30:07.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904399+0000", 2024-09-17T21:30:07.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4573000000000001e-05 2024-09-17T21:30:07.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904417+0000", 2024-09-17T21:30:07.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8159999999999999e-05 2024-09-17T21:30:07.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:07.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.908414+0000", 2024-09-17T21:30:07.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039968240000000004 2024-09-17T21:30:07.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.908468+0000", 2024-09-17T21:30:07.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3600999999999997e-05 2024-09-17T21:30:07.140 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.140 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.140 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.141 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "pg_update_log_missing(3.4 epoch 45/25 rep_tid 251 entries 45'135 (0'0) error 3:22ae8e31:::rbd_data.1120b21c3309.000000000000005a:head by client.4384.0:48303 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 45'133)", 2024-09-17T21:30:07.141 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.904364+0000", 2024-09-17T21:30:07.141 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.15554226099999999, 2024-09-17T21:30:07.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004087696, 2024-09-17T21:30:07.141 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.141 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.141 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T21:30:07.141 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904364+0000", 2024-09-17T21:30:07.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904364+0000", 2024-09-17T21:30:07.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904364+0000", 2024-09-17T21:30:07.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0100000000000001e-07 2024-09-17T21:30:07.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904370+0000", 2024-09-17T21:30:07.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.19e-06 2024-09-17T21:30:07.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904385+0000", 2024-09-17T21:30:07.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4827999999999999e-05 2024-09-17T21:30:07.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904399+0000", 2024-09-17T21:30:07.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3549e-05 2024-09-17T21:30:07.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904499+0000", 2024-09-17T21:30:07.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9815999999999994e-05 2024-09-17T21:30:07.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.908452+0000", 2024-09-17T21:30:07.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039530119999999997 2024-09-17T21:30:07.144 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.144 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.144 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.144 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48461 3.5 e45/37 3:a5a8d98d:::rbd_data.1120b21c3309.00000000000000a1:head v 45'66, mlcod=45'64)", 2024-09-17T21:30:07.144 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.963297+0000", 2024-09-17T21:30:07.145 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.096608796999999996, 2024-09-17T21:30:07.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0028072430000000001, 2024-09-17T21:30:07.145 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.145 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.145 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:07.145 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.963297+0000", 2024-09-17T21:30:07.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.963297+0000", 2024-09-17T21:30:07.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.963298+0000", 2024-09-17T21:30:07.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3900000000000004e-07 2024-09-17T21:30:07.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.963304+0000", 2024-09-17T21:30:07.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3389999999999998e-06 2024-09-17T21:30:07.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.963320+0000", 2024-09-17T21:30:07.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5203e-05 2024-09-17T21:30:07.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.963327+0000", 2024-09-17T21:30:07.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:30:07.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.963368+0000", 2024-09-17T21:30:07.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1187999999999999e-05 2024-09-17T21:30:07.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.963385+0000", 2024-09-17T21:30:07.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6388e-05 2024-09-17T21:30:07.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:07.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.966059+0000", 2024-09-17T21:30:07.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0026738830000000002 2024-09-17T21:30:07.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.966105+0000", 2024-09-17T21:30:07.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5912999999999999e-05 2024-09-17T21:30:07.149 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.149 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.149 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.149 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48462 3.4 e45/25 3:24fc1191:::rbd_data.1120b21c3309.00000000000000a9:head v 45'142, mlcod=45'139)", 2024-09-17T21:30:07.149 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.964224+0000", 2024-09-17T21:30:07.149 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.095682457999999998, 2024-09-17T21:30:07.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043470440000000004, 2024-09-17T21:30:07.149 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.149 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.150 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:07.150 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.964224+0000", 2024-09-17T21:30:07.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.964224+0000", 2024-09-17T21:30:07.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.964224+0000", 2024-09-17T21:30:07.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4300000000000002e-07 2024-09-17T21:30:07.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.964232+0000", 2024-09-17T21:30:07.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5669999999999997e-06 2024-09-17T21:30:07.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.964243+0000", 2024-09-17T21:30:07.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0981e-05 2024-09-17T21:30:07.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.964249+0000", 2024-09-17T21:30:07.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2079999999999997e-06 2024-09-17T21:30:07.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.964319+0000", 2024-09-17T21:30:07.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0152000000000002e-05 2024-09-17T21:30:07.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.964338+0000", 2024-09-17T21:30:07.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8448999999999999e-05 2024-09-17T21:30:07.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:07.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.968525+0000", 2024-09-17T21:30:07.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0041876220000000002 2024-09-17T21:30:07.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.968571+0000", 2024-09-17T21:30:07.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5321999999999997e-05 2024-09-17T21:30:07.153 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.153 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.153 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.154 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.154 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.154 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:07.155 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:30:07.156 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:07.153+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:07.191 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-17T21:30:07.194 INFO:teuthology.orchestra.run.smithi129.stdout:1674 writesame 0xa7c201a thru 0xa7c7959 data_size 0x54(0x5940 bytes) 2024-09-17T21:30:07.256 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:07.253+0000 7fbbc1761640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33587) UID: 0 2024-09-17T21:30:07.296 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:07.296 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:07.296 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:07.296 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:07.305 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:30:07.388 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:07.386+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:07.436 INFO:teuthology.orchestra.run.smithi129.stdout:1675 read 0x47609f4 thru 0x47696b6 (0x8cc3 bytes) 2024-09-17T21:30:07.436 INFO:teuthology.orchestra.run.smithi129.stdout:1676 read 0x9c0a3ee thru 0x9c0e9bd (0x45d0 bytes) 2024-09-17T21:30:07.436 INFO:teuthology.orchestra.run.smithi129.stdout:1677 punch from 0x11d7015 to 0x11d7b3c, (0xb27 bytes) 2024-09-17T21:30:07.436 INFO:teuthology.orchestra.run.smithi129.stdout:1678 write 0x6341ccb thru 0x634b8d5 (0x9c0b bytes) 2024-09-17T21:30:07.454 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:07.455 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:07.455 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:07.455 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:07.455 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:07.457 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:07.454+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:07.463 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:30:07.558 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:07.556+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:07.594 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is set 2024-09-17T21:30:07.619 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26531 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.849021+0000", 2024-09-17T21:30:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.767051233, 2024-09-17T21:30:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024653859, 2024-09-17T21:30:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26531 2024-09-17T21:30:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849021+0000", 2024-09-17T21:30:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849021+0000", 2024-09-17T21:30:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849021+0000", 2024-09-17T21:30:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6800000000000001e-07 2024-09-17T21:30:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849023+0000", 2024-09-17T21:30:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8789999999999999e-06 2024-09-17T21:30:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849029+0000", 2024-09-17T21:30:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1059999999999996e-06 2024-09-17T21:30:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849034+0000", 2024-09-17T21:30:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:30:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.873511+0000", 2024-09-17T21:30:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024476617999999999 2024-09-17T21:30:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.873592+0000", 2024-09-17T21:30:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.064e-05 2024-09-17T21:30:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.873675+0000", 2024-09-17T21:30:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3289999999999997e-05 2024-09-17T21:30:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26532 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.849040+0000", 2024-09-17T21:30:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.767031844000002, 2024-09-17T21:30:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024831517000000001, 2024-09-17T21:30:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26532 2024-09-17T21:30:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849040+0000", 2024-09-17T21:30:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849040+0000", 2024-09-17T21:30:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849041+0000", 2024-09-17T21:30:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6300000000000001e-07 2024-09-17T21:30:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849043+0000", 2024-09-17T21:30:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:30:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849049+0000", 2024-09-17T21:30:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9529999999999996e-06 2024-09-17T21:30:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.849053+0000", 2024-09-17T21:30:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5569999999999997e-06 2024-09-17T21:30:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.873702+0000", 2024-09-17T21:30:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024648798999999999 2024-09-17T21:30:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.873790+0000", 2024-09-17T21:30:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7899999999999995e-05 2024-09-17T21:30:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.873872+0000", 2024-09-17T21:30:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2033000000000006e-05 2024-09-17T21:30:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26626 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 387072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853605+0000", 2024-09-17T21:30:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.762466706000001, 2024-09-17T21:30:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024510626000000001, 2024-09-17T21:30:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26626 2024-09-17T21:30:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853605+0000", 2024-09-17T21:30:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853605+0000", 2024-09-17T21:30:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853665+0000", 2024-09-17T21:30:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9477e-05 2024-09-17T21:30:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853668+0000", 2024-09-17T21:30:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3139999999999999e-06 2024-09-17T21:30:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853676+0000", 2024-09-17T21:30:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7389999999999999e-06 2024-09-17T21:30:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853682+0000", 2024-09-17T21:30:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8710000000000002e-06 2024-09-17T21:30:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.877964+0000", 2024-09-17T21:30:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024282142999999999 2024-09-17T21:30:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878038+0000", 2024-09-17T21:30:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3882999999999997e-05 2024-09-17T21:30:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878116+0000", 2024-09-17T21:30:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8199000000000001e-05 2024-09-17T21:30:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26627 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 399360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853691+0000", 2024-09-17T21:30:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.762381134000002, 2024-09-17T21:30:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024605452999999999, 2024-09-17T21:30:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26627 2024-09-17T21:30:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853691+0000", 2024-09-17T21:30:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853691+0000", 2024-09-17T21:30:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853692+0000", 2024-09-17T21:30:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7300000000000001e-07 2024-09-17T21:30:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853694+0000", 2024-09-17T21:30:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3120000000000001e-06 2024-09-17T21:30:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853700+0000", 2024-09-17T21:30:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1940000000000003e-06 2024-09-17T21:30:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853705+0000", 2024-09-17T21:30:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8980000000000004e-06 2024-09-17T21:30:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878143+0000", 2024-09-17T21:30:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024437708999999998 2024-09-17T21:30:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878218+0000", 2024-09-17T21:30:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5377999999999996e-05 2024-09-17T21:30:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878297+0000", 2024-09-17T21:30:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8489e-05 2024-09-17T21:30:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26628 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 413696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853711+0000", 2024-09-17T21:30:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.762361040000002, 2024-09-17T21:30:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024762052999999999, 2024-09-17T21:30:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26628 2024-09-17T21:30:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853711+0000", 2024-09-17T21:30:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853711+0000", 2024-09-17T21:30:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853712+0000", 2024-09-17T21:30:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.15e-07 2024-09-17T21:30:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853713+0000", 2024-09-17T21:30:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.79e-06 2024-09-17T21:30:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853719+0000", 2024-09-17T21:30:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0469999999999999e-06 2024-09-17T21:30:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853724+0000", 2024-09-17T21:30:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.69e-06 2024-09-17T21:30:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878324+0000", 2024-09-17T21:30:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024599875 2024-09-17T21:30:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878396+0000", 2024-09-17T21:30:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2486999999999999e-05 2024-09-17T21:30:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878473+0000", 2024-09-17T21:30:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6749000000000007e-05 2024-09-17T21:30:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26629 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853730+0000", 2024-09-17T21:30:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.762341911, 2024-09-17T21:30:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024923506000000002, 2024-09-17T21:30:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26629 2024-09-17T21:30:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853730+0000", 2024-09-17T21:30:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853730+0000", 2024-09-17T21:30:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853731+0000", 2024-09-17T21:30:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:30:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853733+0000", 2024-09-17T21:30:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:30:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853739+0000", 2024-09-17T21:30:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0159999999999997e-06 2024-09-17T21:30:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853744+0000", 2024-09-17T21:30:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0019999999999997e-06 2024-09-17T21:30:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878501+0000", 2024-09-17T21:30:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024757112000000001 2024-09-17T21:30:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878573+0000", 2024-09-17T21:30:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2006000000000001e-05 2024-09-17T21:30:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878654+0000", 2024-09-17T21:30:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0734000000000003e-05 2024-09-17T21:30:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26630 3.b 3:d4297b01:::rbd_data.11202c1b6cb4.000000000000003e:4 [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853750+0000", 2024-09-17T21:30:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.762322422, 2024-09-17T21:30:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025078738999999999, 2024-09-17T21:30:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26630 2024-09-17T21:30:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853750+0000", 2024-09-17T21:30:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853750+0000", 2024-09-17T21:30:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853750+0000", 2024-09-17T21:30:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:30:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853752+0000", 2024-09-17T21:30:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8700000000000001e-06 2024-09-17T21:30:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853758+0000", 2024-09-17T21:30:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1340000000000001e-06 2024-09-17T21:30:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853763+0000", 2024-09-17T21:30:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9819999999999999e-06 2024-09-17T21:30:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878681+0000", 2024-09-17T21:30:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024917661000000001 2024-09-17T21:30:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878752+0000", 2024-09-17T21:30:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1132999999999998e-05 2024-09-17T21:30:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878829+0000", 2024-09-17T21:30:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6638999999999996e-05 2024-09-17T21:30:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26645 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 266240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853769+0000", 2024-09-17T21:30:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.762303247999998, 2024-09-17T21:30:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025237900000000001, 2024-09-17T21:30:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26645 2024-09-17T21:30:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853769+0000", 2024-09-17T21:30:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853769+0000", 2024-09-17T21:30:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853769+0000", 2024-09-17T21:30:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2599999999999998e-07 2024-09-17T21:30:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853771+0000", 2024-09-17T21:30:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9609999999999999e-06 2024-09-17T21:30:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853777+0000", 2024-09-17T21:30:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3630000000000002e-06 2024-09-17T21:30:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853781+0000", 2024-09-17T21:30:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7779999999999998e-06 2024-09-17T21:30:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878855+0000", 2024-09-17T21:30:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025073649999999999 2024-09-17T21:30:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.878927+0000", 2024-09-17T21:30:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1500000000000003e-05 2024-09-17T21:30:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879007+0000", 2024-09-17T21:30:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0321999999999994e-05 2024-09-17T21:30:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26646 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 280576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853787+0000", 2024-09-17T21:30:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.762285016, 2024-09-17T21:30:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025570283999999999, 2024-09-17T21:30:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26646 2024-09-17T21:30:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853787+0000", 2024-09-17T21:30:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853787+0000", 2024-09-17T21:30:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853788+0000", 2024-09-17T21:30:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:30:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853793+0000", 2024-09-17T21:30:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0900000000000004e-06 2024-09-17T21:30:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853798+0000", 2024-09-17T21:30:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8479999999999997e-06 2024-09-17T21:30:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853803+0000", 2024-09-17T21:30:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7620000000000003e-06 2024-09-17T21:30:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879035+0000", 2024-09-17T21:30:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025231535999999999 2024-09-17T21:30:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879107+0000", 2024-09-17T21:30:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2154000000000001e-05 2024-09-17T21:30:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879357+0000", 2024-09-17T21:30:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025055800000000001 2024-09-17T21:30:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26647 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 292864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853809+0000", 2024-09-17T21:30:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.762262952, 2024-09-17T21:30:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025774357000000001, 2024-09-17T21:30:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26647 2024-09-17T21:30:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853809+0000", 2024-09-17T21:30:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853809+0000", 2024-09-17T21:30:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853810+0000", 2024-09-17T21:30:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7500000000000001e-07 2024-09-17T21:30:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853811+0000", 2024-09-17T21:30:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7749999999999999e-06 2024-09-17T21:30:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853818+0000", 2024-09-17T21:30:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7519999999999999e-06 2024-09-17T21:30:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853823+0000", 2024-09-17T21:30:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2399999999999998e-06 2024-09-17T21:30:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879387+0000", 2024-09-17T21:30:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02556394 2024-09-17T21:30:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879462+0000", 2024-09-17T21:30:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4697000000000007e-05 2024-09-17T21:30:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879584+0000", 2024-09-17T21:30:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012157799999999999 2024-09-17T21:30:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26648 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 307200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853862+0000", 2024-09-17T21:30:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.762210492000001, 2024-09-17T21:30:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025888738000000001, 2024-09-17T21:30:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26648 2024-09-17T21:30:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853862+0000", 2024-09-17T21:30:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853862+0000", 2024-09-17T21:30:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853862+0000", 2024-09-17T21:30:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1799999999999995e-07 2024-09-17T21:30:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853865+0000", 2024-09-17T21:30:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5950000000000001e-06 2024-09-17T21:30:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853872+0000", 2024-09-17T21:30:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1749999999999999e-06 2024-09-17T21:30:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853877+0000", 2024-09-17T21:30:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0930000000000002e-06 2024-09-17T21:30:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879609+0000", 2024-09-17T21:30:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025731970999999999 2024-09-17T21:30:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879674+0000", 2024-09-17T21:30:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4958999999999999e-05 2024-09-17T21:30:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879750+0000", 2024-09-17T21:30:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6327000000000003e-05 2024-09-17T21:30:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26649 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 321536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853886+0000", 2024-09-17T21:30:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.762186399000001, 2024-09-17T21:30:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026043475999999999, 2024-09-17T21:30:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26649 2024-09-17T21:30:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853886+0000", 2024-09-17T21:30:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853886+0000", 2024-09-17T21:30:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853886+0000", 2024-09-17T21:30:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:30:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853888+0000", 2024-09-17T21:30:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0789999999999999e-06 2024-09-17T21:30:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853894+0000", 2024-09-17T21:30:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0209999999999996e-06 2024-09-17T21:30:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853900+0000", 2024-09-17T21:30:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2220000000000001e-06 2024-09-17T21:30:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879779+0000", 2024-09-17T21:30:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025879049000000001 2024-09-17T21:30:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879852+0000", 2024-09-17T21:30:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3244999999999994e-05 2024-09-17T21:30:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879929+0000", 2024-09-17T21:30:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7472999999999995e-05 2024-09-17T21:30:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26650 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 333824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853906+0000", 2024-09-17T21:30:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.762166281999999, 2024-09-17T21:30:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026202728000000002, 2024-09-17T21:30:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26650 2024-09-17T21:30:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853906+0000", 2024-09-17T21:30:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853906+0000", 2024-09-17T21:30:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853906+0000", 2024-09-17T21:30:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853908+0000", 2024-09-17T21:30:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0030000000000001e-06 2024-09-17T21:30:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853914+0000", 2024-09-17T21:30:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3759999999999999e-06 2024-09-17T21:30:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853919+0000", 2024-09-17T21:30:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9139999999999999e-06 2024-09-17T21:30:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.879956+0000", 2024-09-17T21:30:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026037523999999999 2024-09-17T21:30:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880027+0000", 2024-09-17T21:30:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0939000000000006e-05 2024-09-17T21:30:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880109+0000", 2024-09-17T21:30:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1594000000000005e-05 2024-09-17T21:30:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26651 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853925+0000", 2024-09-17T21:30:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.762147472999999, 2024-09-17T21:30:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026365942, 2024-09-17T21:30:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.682 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.682 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.682 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26651 2024-09-17T21:30:07.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.682 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853925+0000", 2024-09-17T21:30:07.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853925+0000", 2024-09-17T21:30:07.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853925+0000", 2024-09-17T21:30:07.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4799999999999999e-07 2024-09-17T21:30:07.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853927+0000", 2024-09-17T21:30:07.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9250000000000002e-06 2024-09-17T21:30:07.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853932+0000", 2024-09-17T21:30:07.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3489999999999999e-06 2024-09-17T21:30:07.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853937+0000", 2024-09-17T21:30:07.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4769999999999997e-06 2024-09-17T21:30:07.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880136+0000", 2024-09-17T21:30:07.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026199618000000001 2024-09-17T21:30:07.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880213+0000", 2024-09-17T21:30:07.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6594999999999994e-05 2024-09-17T21:30:07.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880291+0000", 2024-09-17T21:30:07.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.763e-05 2024-09-17T21:30:07.685 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.685 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.685 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.685 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26652 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.685 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853943+0000", 2024-09-17T21:30:07.686 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.762129011999999, 2024-09-17T21:30:07.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026527075000000001, 2024-09-17T21:30:07.686 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.686 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.686 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.686 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.686 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.686 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.686 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26652 2024-09-17T21:30:07.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.686 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853943+0000", 2024-09-17T21:30:07.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853943+0000", 2024-09-17T21:30:07.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853944+0000", 2024-09-17T21:30:07.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2599999999999998e-07 2024-09-17T21:30:07.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853946+0000", 2024-09-17T21:30:07.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9970000000000001e-06 2024-09-17T21:30:07.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853953+0000", 2024-09-17T21:30:07.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4080000000000001e-06 2024-09-17T21:30:07.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853958+0000", 2024-09-17T21:30:07.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7280000000000003e-06 2024-09-17T21:30:07.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880319+0000", 2024-09-17T21:30:07.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026360999 2024-09-17T21:30:07.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880391+0000", 2024-09-17T21:30:07.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2418999999999999e-05 2024-09-17T21:30:07.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880470+0000", 2024-09-17T21:30:07.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9197999999999996e-05 2024-09-17T21:30:07.689 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.690 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.690 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.690 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26653 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 374784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.690 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853964+0000", 2024-09-17T21:30:07.690 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.762108314999999, 2024-09-17T21:30:07.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026678573000000001, 2024-09-17T21:30:07.690 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.690 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.690 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.690 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26653 2024-09-17T21:30:07.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.691 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853964+0000", 2024-09-17T21:30:07.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853964+0000", 2024-09-17T21:30:07.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853964+0000", 2024-09-17T21:30:07.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:30:07.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853966+0000", 2024-09-17T21:30:07.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.888e-06 2024-09-17T21:30:07.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853972+0000", 2024-09-17T21:30:07.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T21:30:07.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853976+0000", 2024-09-17T21:30:07.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7060000000000003e-06 2024-09-17T21:30:07.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880499+0000", 2024-09-17T21:30:07.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026523105000000002 2024-09-17T21:30:07.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880570+0000", 2024-09-17T21:30:07.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0207000000000001e-05 2024-09-17T21:30:07.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880642+0000", 2024-09-17T21:30:07.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2886000000000007e-05 2024-09-17T21:30:07.694 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.694 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.694 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.694 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26654 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 389120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.694 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.853982+0000", 2024-09-17T21:30:07.694 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.762090251, 2024-09-17T21:30:07.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026832110999999999, 2024-09-17T21:30:07.695 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.695 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.695 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.695 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.695 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.695 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.695 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26654 2024-09-17T21:30:07.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.695 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853982+0000", 2024-09-17T21:30:07.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853982+0000", 2024-09-17T21:30:07.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853982+0000", 2024-09-17T21:30:07.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8099999999999998e-07 2024-09-17T21:30:07.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853984+0000", 2024-09-17T21:30:07.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7460000000000001e-06 2024-09-17T21:30:07.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853990+0000", 2024-09-17T21:30:07.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1179999999999997e-06 2024-09-17T21:30:07.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.853995+0000", 2024-09-17T21:30:07.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4789999999999998e-06 2024-09-17T21:30:07.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880670+0000", 2024-09-17T21:30:07.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026675299 2024-09-17T21:30:07.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880736+0000", 2024-09-17T21:30:07.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6476000000000005e-05 2024-09-17T21:30:07.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880814+0000", 2024-09-17T21:30:07.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7612000000000002e-05 2024-09-17T21:30:07.698 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.698 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.699 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.699 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26655 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 403456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.699 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.854001+0000", 2024-09-17T21:30:07.699 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.762071572, 2024-09-17T21:30:07.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026991347999999998, 2024-09-17T21:30:07.699 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.699 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.699 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.699 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26655 2024-09-17T21:30:07.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.700 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854001+0000", 2024-09-17T21:30:07.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854001+0000", 2024-09-17T21:30:07.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854001+0000", 2024-09-17T21:30:07.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.89e-07 2024-09-17T21:30:07.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854003+0000", 2024-09-17T21:30:07.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.826e-06 2024-09-17T21:30:07.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854009+0000", 2024-09-17T21:30:07.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7150000000000003e-06 2024-09-17T21:30:07.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854014+0000", 2024-09-17T21:30:07.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1109999999999999e-06 2024-09-17T21:30:07.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880842+0000", 2024-09-17T21:30:07.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026828161999999999 2024-09-17T21:30:07.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880912+0000", 2024-09-17T21:30:07.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0067000000000005e-05 2024-09-17T21:30:07.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.880992+0000", 2024-09-17T21:30:07.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0078000000000001e-05 2024-09-17T21:30:07.703 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.703 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.703 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.703 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26656 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 415744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.703 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.854020+0000", 2024-09-17T21:30:07.703 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.762052112999999, 2024-09-17T21:30:07.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027156040999999999, 2024-09-17T21:30:07.704 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.704 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.704 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.704 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26656 2024-09-17T21:30:07.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.704 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854020+0000", 2024-09-17T21:30:07.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854020+0000", 2024-09-17T21:30:07.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854020+0000", 2024-09-17T21:30:07.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:30:07.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854023+0000", 2024-09-17T21:30:07.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2139999999999999e-06 2024-09-17T21:30:07.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854028+0000", 2024-09-17T21:30:07.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7570000000000001e-06 2024-09-17T21:30:07.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854033+0000", 2024-09-17T21:30:07.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3409999999999996e-06 2024-09-17T21:30:07.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.881018+0000", 2024-09-17T21:30:07.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026985529000000001 2024-09-17T21:30:07.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.881090+0000", 2024-09-17T21:30:07.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1946000000000005e-05 2024-09-17T21:30:07.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.881176+0000", 2024-09-17T21:30:07.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5886999999999999e-05 2024-09-17T21:30:07.707 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.708 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.708 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.708 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:26657 3.b 3:d03bf7c2:::rbd_data.11202c1b6cb4.0000000000000039:4 [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:07.708 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:29:49.854123+0000", 2024-09-17T21:30:07.708 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.761948826000001, 2024-09-17T21:30:07.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027224600000000002, 2024-09-17T21:30:07.708 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:07.708 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:07.708 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:07.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:07.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:07.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:07.709 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 26657 2024-09-17T21:30:07.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.709 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:07.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:07.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854123+0000", 2024-09-17T21:30:07.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:07.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854123+0000", 2024-09-17T21:30:07.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:07.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:07.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854124+0000", 2024-09-17T21:30:07.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:30:07.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:07.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854127+0000", 2024-09-17T21:30:07.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0750000000000002e-06 2024-09-17T21:30:07.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:07.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854134+0000", 2024-09-17T21:30:07.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4630000000000004e-06 2024-09-17T21:30:07.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:07.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.854141+0000", 2024-09-17T21:30:07.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:30:07.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:07.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.881204+0000", 2024-09-17T21:30:07.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027062685999999999 2024-09-17T21:30:07.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:07.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.881273+0000", 2024-09-17T21:30:07.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8841e-05 2024-09-17T21:30:07.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:07.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:07.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:07.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:29:49.881348+0000", 2024-09-17T21:30:07.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5265999999999994e-05 2024-09-17T21:30:07.712 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.712 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.712 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.712 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:07.712 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:07.712 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:07.714 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:30:07.714 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:07.711+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:07.799 INFO:teuthology.orchestra.run.smithi129.stdout:1680 punch from 0x9144abd to 0x9147e99, (0x33dc bytes) 2024-09-17T21:30:07.799 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:07.799 INFO:teuthology.orchestra.run.smithi129.stdout:1682 clone 14 order 25 su 262144 sc 10 2024-09-17T21:30:07.814 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:07.812+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:07.821 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:07.821 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:07.821 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:07.821 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:07.830 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:30:07.915 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:07.913+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:07.972 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:07.973 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:07.981 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:30:08.022 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:08.020+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:08.117 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:08.115+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:08.126 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:08.126 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:08.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:08.126 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:08.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.126 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21342 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2082816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:08.126 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:08.127 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.446851258999999, 2024-09-17T21:30:08.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022083523000000001, 2024-09-17T21:30:08.127 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.127 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.127 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.127 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21342 2024-09-17T21:30:08.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.127 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:08.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:08.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:08.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:30:08.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676659+0000", 2024-09-17T21:30:08.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0950000000000001e-06 2024-09-17T21:30:08.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676662+0000", 2024-09-17T21:30:08.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.971e-06 2024-09-17T21:30:08.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676665+0000", 2024-09-17T21:30:08.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6189999999999998e-06 2024-09-17T21:30:08.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698568+0000", 2024-09-17T21:30:08.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021903549000000001 2024-09-17T21:30:08.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698668+0000", 2024-09-17T21:30:08.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9778999999999995e-05 2024-09-17T21:30:08.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698741+0000", 2024-09-17T21:30:08.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000006e-05 2024-09-17T21:30:08.130 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.131 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.131 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.131 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21343 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2095104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:08.131 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:08.131 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.446841321000001, 2024-09-17T21:30:08.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022504024000000001, 2024-09-17T21:30:08.131 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.131 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.131 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.132 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21343 2024-09-17T21:30:08.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.132 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:08.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:08.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:08.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:30:08.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676669+0000", 2024-09-17T21:30:08.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.003e-06 2024-09-17T21:30:08.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676672+0000", 2024-09-17T21:30:08.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9280000000000002e-06 2024-09-17T21:30:08.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676674+0000", 2024-09-17T21:30:08.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5469999999999999e-06 2024-09-17T21:30:08.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699015+0000", 2024-09-17T21:30:08.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022340563000000001 2024-09-17T21:30:08.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699102+0000", 2024-09-17T21:30:08.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7425999999999999e-05 2024-09-17T21:30:08.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699172+0000", 2024-09-17T21:30:08.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9346999999999999e-05 2024-09-17T21:30:08.135 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.135 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.135 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.135 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45576 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:08.135 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:08.136 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.418607521, 2024-09-17T21:30:08.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022024660000000001, 2024-09-17T21:30:08.136 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.136 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.136 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.136 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45576 2024-09-17T21:30:08.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.136 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:08.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:08.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704902+0000", 2024-09-17T21:30:08.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9100000000000004e-07 2024-09-17T21:30:08.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704905+0000", 2024-09-17T21:30:08.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6139999999999999e-06 2024-09-17T21:30:08.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704911+0000", 2024-09-17T21:30:08.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8619999999999997e-06 2024-09-17T21:30:08.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704917+0000", 2024-09-17T21:30:08.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:30:08.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726681+0000", 2024-09-17T21:30:08.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021764569000000001 2024-09-17T21:30:08.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726726+0000", 2024-09-17T21:30:08.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4218999999999998e-05 2024-09-17T21:30:08.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726926+0000", 2024-09-17T21:30:08.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000200466 2024-09-17T21:30:08.139 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.140 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.140 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.140 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45577 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:08.140 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:08.140 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.418583865, 2024-09-17T21:30:08.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022269694999999999, 2024-09-17T21:30:08.140 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.140 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.140 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.141 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.141 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45577 2024-09-17T21:30:08.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.141 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:08.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:08.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704926+0000", 2024-09-17T21:30:08.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:30:08.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704928+0000", 2024-09-17T21:30:08.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9989999999999998e-06 2024-09-17T21:30:08.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704934+0000", 2024-09-17T21:30:08.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:30:08.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704939+0000", 2024-09-17T21:30:08.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5909999999999998e-06 2024-09-17T21:30:08.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726954+0000", 2024-09-17T21:30:08.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022014885000000001 2024-09-17T21:30:08.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726995+0000", 2024-09-17T21:30:08.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1474000000000003e-05 2024-09-17T21:30:08.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727195+0000", 2024-09-17T21:30:08.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019968300000000001 2024-09-17T21:30:08.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.144 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.144 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45578 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:08.144 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:08.145 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.418562621, 2024-09-17T21:30:08.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022484006000000001, 2024-09-17T21:30:08.145 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.145 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.145 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.145 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.145 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.145 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.145 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45578 2024-09-17T21:30:08.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.145 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:08.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:08.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704947+0000", 2024-09-17T21:30:08.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1399999999999997e-07 2024-09-17T21:30:08.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704949+0000", 2024-09-17T21:30:08.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0959999999999999e-06 2024-09-17T21:30:08.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704956+0000", 2024-09-17T21:30:08.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9330000000000004e-06 2024-09-17T21:30:08.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704961+0000", 2024-09-17T21:30:08.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6759999999999998e-06 2024-09-17T21:30:08.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727213+0000", 2024-09-17T21:30:08.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022252568 2024-09-17T21:30:08.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727259+0000", 2024-09-17T21:30:08.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5238000000000003e-05 2024-09-17T21:30:08.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727430+0000", 2024-09-17T21:30:08.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000171881 2024-09-17T21:30:08.148 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.148 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.149 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.149 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45579 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:08.149 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:08.149 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.418540693, 2024-09-17T21:30:08.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022749080000000001, 2024-09-17T21:30:08.149 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.149 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.149 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.149 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.149 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.149 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.149 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45579 2024-09-17T21:30:08.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.150 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:08.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:08.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704969+0000", 2024-09-17T21:30:08.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:30:08.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704971+0000", 2024-09-17T21:30:08.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.097e-06 2024-09-17T21:30:08.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704977+0000", 2024-09-17T21:30:08.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6329999999999999e-06 2024-09-17T21:30:08.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704983+0000", 2024-09-17T21:30:08.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:30:08.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727457+0000", 2024-09-17T21:30:08.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022474521000000001 2024-09-17T21:30:08.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727498+0000", 2024-09-17T21:30:08.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0558e-05 2024-09-17T21:30:08.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727717+0000", 2024-09-17T21:30:08.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219805 2024-09-17T21:30:08.153 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.153 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.153 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.153 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45650 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 20480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:08.153 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:08.153 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.4174173859999999, 2024-09-17T21:30:08.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021996219000000001, 2024-09-17T21:30:08.153 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.154 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.154 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.154 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.154 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.154 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.154 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45650 2024-09-17T21:30:08.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.154 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:08.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:08.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:08.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5199999999999997e-07 2024-09-17T21:30:08.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706095+0000", 2024-09-17T21:30:08.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6079999999999998e-06 2024-09-17T21:30:08.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706101+0000", 2024-09-17T21:30:08.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:30:08.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706106+0000", 2024-09-17T21:30:08.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T21:30:08.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728000+0000", 2024-09-17T21:30:08.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021893378000000002 2024-09-17T21:30:08.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728039+0000", 2024-09-17T21:30:08.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9249000000000003e-05 2024-09-17T21:30:08.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728088+0000", 2024-09-17T21:30:08.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8785000000000001e-05 2024-09-17T21:30:08.157 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.157 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.158 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.158 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45652 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:08.158 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:08.158 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.417394555, 2024-09-17T21:30:08.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022100465999999999, 2024-09-17T21:30:08.158 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.158 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.158 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.158 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.158 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.158 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.158 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45652 2024-09-17T21:30:08.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.159 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:08.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:08.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706115+0000", 2024-09-17T21:30:08.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:30:08.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706117+0000", 2024-09-17T21:30:08.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-06 2024-09-17T21:30:08.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706122+0000", 2024-09-17T21:30:08.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.288e-06 2024-09-17T21:30:08.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706136+0000", 2024-09-17T21:30:08.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3409e-05 2024-09-17T21:30:08.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728112+0000", 2024-09-17T21:30:08.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021976724999999999 2024-09-17T21:30:08.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728151+0000", 2024-09-17T21:30:08.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8318999999999997e-05 2024-09-17T21:30:08.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728215+0000", 2024-09-17T21:30:08.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4172999999999997e-05 2024-09-17T21:30:08.162 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.162 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.162 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.162 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45653 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:08.162 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:08.162 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.4173653449999999, 2024-09-17T21:30:08.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022249517999999999, 2024-09-17T21:30:08.162 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.163 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.163 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.163 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.163 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.163 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.163 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45653 2024-09-17T21:30:08.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.163 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:08.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T21:30:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706147+0000", 2024-09-17T21:30:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4430000000000002e-06 2024-09-17T21:30:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706152+0000", 2024-09-17T21:30:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5350000000000004e-06 2024-09-17T21:30:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706157+0000", 2024-09-17T21:30:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8420000000000004e-06 2024-09-17T21:30:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728246+0000", 2024-09-17T21:30:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022089237000000001 2024-09-17T21:30:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728309+0000", 2024-09-17T21:30:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2356999999999999e-05 2024-09-17T21:30:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:30:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4596000000000001e-05 2024-09-17T21:30:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45657 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 36864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.4173340809999999, 2024-09-17T21:30:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022218252000000001, 2024-09-17T21:30:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45657 2024-09-17T21:30:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8299999999999997e-07 2024-09-17T21:30:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706178+0000", 2024-09-17T21:30:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3700000000000002e-06 2024-09-17T21:30:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706184+0000", 2024-09-17T21:30:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.144e-06 2024-09-17T21:30:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706190+0000", 2024-09-17T21:30:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8590000000000001e-06 2024-09-17T21:30:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728240+0000", 2024-09-17T21:30:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022049804999999999 2024-09-17T21:30:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728307+0000", 2024-09-17T21:30:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7477999999999993e-05 2024-09-17T21:30:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:30:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6112999999999994e-05 2024-09-17T21:30:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45658 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.4173109610000001, 2024-09-17T21:30:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022376455, 2024-09-17T21:30:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45658 2024-09-17T21:30:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:08.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:08.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:30:08.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706200+0000", 2024-09-17T21:30:08.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:30:08.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706206+0000", 2024-09-17T21:30:08.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7039999999999999e-06 2024-09-17T21:30:08.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706211+0000", 2024-09-17T21:30:08.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9429999999999999e-06 2024-09-17T21:30:08.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728418+0000", 2024-09-17T21:30:08.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022206760999999998 2024-09-17T21:30:08.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728497+0000", 2024-09-17T21:30:08.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9493999999999995e-05 2024-09-17T21:30:08.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728574+0000", 2024-09-17T21:30:08.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7180000000000003e-05 2024-09-17T21:30:08.175 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.175 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.176 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.176 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45706 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 172032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:08.176 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707876+0000", 2024-09-17T21:30:08.176 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.415633216, 2024-09-17T21:30:08.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021993196999999999, 2024-09-17T21:30:08.176 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.176 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.176 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.176 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.176 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.176 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.176 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45706 2024-09-17T21:30:08.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.177 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707876+0000", 2024-09-17T21:30:08.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707876+0000", 2024-09-17T21:30:08.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707876+0000", 2024-09-17T21:30:08.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999998e-07 2024-09-17T21:30:08.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707878+0000", 2024-09-17T21:30:08.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7689999999999999e-06 2024-09-17T21:30:08.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707884+0000", 2024-09-17T21:30:08.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3990000000000001e-06 2024-09-17T21:30:08.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707895+0000", 2024-09-17T21:30:08.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0577000000000001e-05 2024-09-17T21:30:08.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729606+0000", 2024-09-17T21:30:08.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021711034000000001 2024-09-17T21:30:08.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729651+0000", 2024-09-17T21:30:08.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5337999999999999e-05 2024-09-17T21:30:08.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729869+0000", 2024-09-17T21:30:08.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00021769899999999999 2024-09-17T21:30:08.180 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.180 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.180 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.180 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45707 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 212992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:08.180 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:08.180 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.415606895, 2024-09-17T21:30:08.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022235550999999999, 2024-09-17T21:30:08.181 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.181 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.181 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.181 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.181 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.181 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.181 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45707 2024-09-17T21:30:08.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.181 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:08.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:08.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:08.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:30:08.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707905+0000", 2024-09-17T21:30:08.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2120000000000002e-06 2024-09-17T21:30:08.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707911+0000", 2024-09-17T21:30:08.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1929999999999998e-06 2024-09-17T21:30:08.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707915+0000", 2024-09-17T21:30:08.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:30:08.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729897+0000", 2024-09-17T21:30:08.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021981186 2024-09-17T21:30:08.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729944+0000", 2024-09-17T21:30:08.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571000000000002e-05 2024-09-17T21:30:08.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730138+0000", 2024-09-17T21:30:08.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000193451 2024-09-17T21:30:08.184 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.184 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.184 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.185 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45708 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:08.185 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:08.185 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.4155867630000001, 2024-09-17T21:30:08.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022533742999999998, 2024-09-17T21:30:08.185 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.185 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.185 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.185 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.185 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.185 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.185 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45708 2024-09-17T21:30:08.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.186 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:08.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:08.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707923+0000", 2024-09-17T21:30:08.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:30:08.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707932+0000", 2024-09-17T21:30:08.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6430000000000007e-06 2024-09-17T21:30:08.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707939+0000", 2024-09-17T21:30:08.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3539999999999996e-06 2024-09-17T21:30:08.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707944+0000", 2024-09-17T21:30:08.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3249999999999998e-06 2024-09-17T21:30:08.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730176+0000", 2024-09-17T21:30:08.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022232393 2024-09-17T21:30:08.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730263+0000", 2024-09-17T21:30:08.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6638999999999995e-05 2024-09-17T21:30:08.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730456+0000", 2024-09-17T21:30:08.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019304899999999999 2024-09-17T21:30:08.189 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.189 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.189 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.189 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45752 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:08.189 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:08.189 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.415331543, 2024-09-17T21:30:08.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022426878000000001, 2024-09-17T21:30:08.190 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.190 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.190 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.190 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.190 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.190 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.190 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45752 2024-09-17T21:30:08.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.190 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:08.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:08.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708178+0000", 2024-09-17T21:30:08.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:30:08.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708181+0000", 2024-09-17T21:30:08.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.909e-06 2024-09-17T21:30:08.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708187+0000", 2024-09-17T21:30:08.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4999999999999996e-06 2024-09-17T21:30:08.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708193+0000", 2024-09-17T21:30:08.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:30:08.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730484+0000", 2024-09-17T21:30:08.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022291765000000002 2024-09-17T21:30:08.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730541+0000", 2024-09-17T21:30:08.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6483000000000001e-05 2024-09-17T21:30:08.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730604+0000", 2024-09-17T21:30:08.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3429000000000005e-05 2024-09-17T21:30:08.194 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.194 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.194 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.194 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45761 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:08.194 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:08.194 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.4153036329999999, 2024-09-17T21:30:08.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022614408999999999, 2024-09-17T21:30:08.194 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.194 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.194 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.194 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.195 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.195 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.195 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45761 2024-09-17T21:30:08.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.195 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:08.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:08.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708206+0000", 2024-09-17T21:30:08.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4700000000000001e-07 2024-09-17T21:30:08.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708208+0000", 2024-09-17T21:30:08.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1509999999999998e-06 2024-09-17T21:30:08.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708214+0000", 2024-09-17T21:30:08.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.49e-06 2024-09-17T21:30:08.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708219+0000", 2024-09-17T21:30:08.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8119999999999998e-06 2024-09-17T21:30:08.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730641+0000", 2024-09-17T21:30:08.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022422765000000001 2024-09-17T21:30:08.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730731+0000", 2024-09-17T21:30:08.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9598999999999994e-05 2024-09-17T21:30:08.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730820+0000", 2024-09-17T21:30:08.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8844999999999994e-05 2024-09-17T21:30:08.198 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.198 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.198 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.198 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45778 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:08.199 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:08.199 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.414560029, 2024-09-17T21:30:08.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022018544000000001, 2024-09-17T21:30:08.199 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.199 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.199 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.199 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.199 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.199 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.199 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45778 2024-09-17T21:30:08.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.199 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:08.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:08.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708950+0000", 2024-09-17T21:30:08.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-17T21:30:08.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709307+0000", 2024-09-17T21:30:08.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000356848 2024-09-17T21:30:08.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709316+0000", 2024-09-17T21:30:08.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1430000000000004e-06 2024-09-17T21:30:08.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709328+0000", 2024-09-17T21:30:08.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2629999999999999e-05 2024-09-17T21:30:08.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730861+0000", 2024-09-17T21:30:08.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021532057 2024-09-17T21:30:08.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730905+0000", 2024-09-17T21:30:08.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4907000000000002e-05 2024-09-17T21:30:08.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730968+0000", 2024-09-17T21:30:08.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2108999999999998e-05 2024-09-17T21:30:08.203 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.203 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.203 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.203 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45780 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 90112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:08.203 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709383+0000", 2024-09-17T21:30:08.203 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.414126319, 2024-09-17T21:30:08.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021946739999999999, 2024-09-17T21:30:08.203 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.203 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.203 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.204 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.204 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.204 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.204 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45780 2024-09-17T21:30:08.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.204 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709383+0000", 2024-09-17T21:30:08.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709383+0000", 2024-09-17T21:30:08.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709383+0000", 2024-09-17T21:30:08.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9200000000000001e-07 2024-09-17T21:30:08.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709386+0000", 2024-09-17T21:30:08.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:30:08.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709392+0000", 2024-09-17T21:30:08.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9369999999999998e-06 2024-09-17T21:30:08.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709398+0000", 2024-09-17T21:30:08.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3489999999999999e-06 2024-09-17T21:30:08.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731170+0000", 2024-09-17T21:30:08.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021772657000000001 2024-09-17T21:30:08.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731244+0000", 2024-09-17T21:30:08.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3466000000000004e-05 2024-09-17T21:30:08.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731329+0000", 2024-09-17T21:30:08.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5498e-05 2024-09-17T21:30:08.207 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.207 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.207 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.208 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45785 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:08.208 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:08.208 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.414081846, 2024-09-17T21:30:08.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022044113000000001, 2024-09-17T21:30:08.208 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.208 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.208 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.208 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.208 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.208 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.208 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45785 2024-09-17T21:30:08.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.208 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:08.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:08.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709428+0000", 2024-09-17T21:30:08.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:30:08.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709430+0000", 2024-09-17T21:30:08.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:30:08.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709451+0000", 2024-09-17T21:30:08.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1372000000000001e-05 2024-09-17T21:30:08.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709472+0000", 2024-09-17T21:30:08.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1007999999999999e-05 2024-09-17T21:30:08.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731293+0000", 2024-09-17T21:30:08.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021821320000000002 2024-09-17T21:30:08.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731378+0000", 2024-09-17T21:30:08.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5059e-05 2024-09-17T21:30:08.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731471+0000", 2024-09-17T21:30:08.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2867000000000004e-05 2024-09-17T21:30:08.212 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.212 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.212 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.212 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45786 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:08.212 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:08.212 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.4140230489999999, 2024-09-17T21:30:08.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022181821000000001, 2024-09-17T21:30:08.212 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:08.212 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:08.212 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:08.213 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:08.213 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:08.213 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:08.213 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45786 2024-09-17T21:30:08.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.213 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:08.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:08.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:08.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:08.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:08.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:08.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:08.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709487+0000", 2024-09-17T21:30:08.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1299999999999999e-07 2024-09-17T21:30:08.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:08.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709491+0000", 2024-09-17T21:30:08.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2089999999999999e-06 2024-09-17T21:30:08.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:08.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709498+0000", 2024-09-17T21:30:08.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3980000000000002e-06 2024-09-17T21:30:08.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:08.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709504+0000", 2024-09-17T21:30:08.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7219999999999996e-06 2024-09-17T21:30:08.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:08.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731499+0000", 2024-09-17T21:30:08.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021995297 2024-09-17T21:30:08.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:08.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731591+0000", 2024-09-17T21:30:08.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1373999999999996e-05 2024-09-17T21:30:08.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:08.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:08.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:08.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731668+0000", 2024-09-17T21:30:08.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7107999999999994e-05 2024-09-17T21:30:08.216 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.216 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.216 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.216 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:08.216 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:08.217 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:08.218 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:30:08.218 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:08.215+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:08.223 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:30:08.318 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:08.316+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:08.355 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:08.355 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:08.364 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:08.364 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:08.364 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:08.364 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:08.371 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:30:08.372 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:30:08.373 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:30:08.387 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:08.387 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:08.403 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:08.403 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:08.419 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:08.420 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:08.436 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:08.437 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:08.452 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:08.453 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:08.457 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:08.455+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:08.468 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:08.469 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:08.516 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:08.516 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:08.519 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:08.517+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:08.524 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:30:08.620 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:08.618+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:08.677 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:08.677 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:08.686 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:30:08.697 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T21:30:08.697 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:30:08.698 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T21:30:08.698 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T21:30:08.698 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:30:08.700 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone13 from 0xcac9e07 (overlap 0xcac9e07) to 0x288ec67 2024-09-17T21:30:08.727 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:08.725+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:08.734 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T21:30:08.734 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.1 2024-09-17T21:30:08.734 INFO:tasks.thrashosds.thrasher:remote for osd 1 is ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T21:30:08.735 DEBUG:teuthology.orchestra.run.smithi129:> type ceph-objectstore-tool 2024-09-17T21:30:08.789 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T21:30:08.789 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op list-pgs 2024-09-17T21:30:08.830 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:08.827+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:08.830 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:08.830 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:08.838 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:30:08.923 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:08.920+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:08.943 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:08.943 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:08.951 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:30:09.024 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:09.022+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:09.054 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:09.054 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:09.063 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:30:09.127 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:09.125+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:09.168 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:09.168 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:09.178 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:30:09.228 INFO:teuthology.orchestra.run.smithi129.stdout:3.6 2024-09-17T21:30:09.229 INFO:teuthology.orchestra.run.smithi129.stdout:3.1c 2024-09-17T21:30:09.229 INFO:teuthology.orchestra.run.smithi129.stdout:3.1a 2024-09-17T21:30:09.229 INFO:teuthology.orchestra.run.smithi129.stdout:3.8 2024-09-17T21:30:09.229 INFO:teuthology.orchestra.run.smithi129.stdout:3.4 2024-09-17T21:30:09.229 INFO:teuthology.orchestra.run.smithi129.stdout:3.16 2024-09-17T21:30:09.229 INFO:teuthology.orchestra.run.smithi129.stdout:3.19 2024-09-17T21:30:09.229 INFO:teuthology.orchestra.run.smithi129.stdout:3.0 2024-09-17T21:30:09.229 INFO:teuthology.orchestra.run.smithi129.stdout:2.6 2024-09-17T21:30:09.229 INFO:teuthology.orchestra.run.smithi129.stdout:2.4 2024-09-17T21:30:09.230 INFO:teuthology.orchestra.run.smithi129.stdout:2.2 2024-09-17T21:30:09.230 INFO:teuthology.orchestra.run.smithi129.stdout:2.0 2024-09-17T21:30:09.230 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:09.226+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:09.324 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:09.324 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:09.326 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:09.323+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:09.333 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:30:09.427 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:09.425+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:09.476 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:09.476 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:09.485 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:30:09.503 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op export --pgid 3.1a --file /var/log/ceph/exp.3.1a.1 2024-09-17T21:30:09.528 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:09.526+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:09.613 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:30:09.652 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:09.652 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:09.652 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:09.649+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:09.661 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:30:09.747 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:09.744+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:09.806 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:09.806 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:09.815 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:30:09.857 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:09.854+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:09.931 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:09.929+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:09.943 INFO:teuthology.orchestra.run.smithi129.stderr:Exporting 3.1a info 3.1a( v 45'239 (0'0,45'239] local-lis/les=25/26 n=36 ec=25/25 lis/c=25/25 les/c/f=26/26/0 sis=25) 2024-09-17T21:30:09.945 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:582fa413:::rbd_data.11207744cc2c.0000000000000149:head# 2024-09-17T21:30:09.946 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:58668c6d:::rbd_data.1120b21c3309.000000000000006e:head# 2024-09-17T21:30:09.947 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:587ac12c:::rbd_data.1120b21c3309.00000000000000f5:head# 2024-09-17T21:30:09.947 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:592066d8:::rbd_data.11207744cc2c.0000000000000087:5# 2024-09-17T21:30:09.947 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:592066d8:::rbd_data.11207744cc2c.0000000000000087:head# 2024-09-17T21:30:09.947 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4# 2024-09-17T21:30:09.948 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:head# 2024-09-17T21:30:09.948 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5a0f90aa:::rbd_data.11207744cc2c.0000000000000145:head# 2024-09-17T21:30:09.951 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5a30f936:::rbd_data.1120b21c3309.00000000000000b8:head# 2024-09-17T21:30:09.952 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5a4e2dc1:::rbd_data.1120b21c3309.00000000000000d4:head# 2024-09-17T21:30:09.952 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5a6712ae:::rbd_id.image_client.0-clone11:head# 2024-09-17T21:30:09.952 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5a9e26c2:::rbd_object_map.1120555f8258:head# 2024-09-17T21:30:09.952 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5ac70768:::rbd_data.1120b21c3309.0000000000000163:head# 2024-09-17T21:30:09.952 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5b6d4ec2:::rbd_data.1120d7424d8b.000000000000000a:e# 2024-09-17T21:30:09.952 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5b6d4ec2:::rbd_data.1120d7424d8b.000000000000000a:head# 2024-09-17T21:30:09.952 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5c2e077a:::rbd_data.1120b21c3309.0000000000000155:head# 2024-09-17T21:30:09.952 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5c360ae6:::rbd_data.1120b21c3309.000000000000015d:head# 2024-09-17T21:30:09.952 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5ca68579:::rbd_data.1120d7424d8b.000000000000007c:e# 2024-09-17T21:30:09.952 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5ca68579:::rbd_data.1120d7424d8b.000000000000007c:head# 2024-09-17T21:30:09.952 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5cea7035:::rbd_info:head# 2024-09-17T21:30:09.952 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5cfb4ea8:::rbd_data.1120b21c3309.0000000000000020:f# 2024-09-17T21:30:09.953 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5cfb4ea8:::rbd_data.1120b21c3309.0000000000000020:head# 2024-09-17T21:30:09.953 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5d18af01:::rbd_data.1120b21c3309.000000000000009c:head# 2024-09-17T21:30:09.953 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5d1e6b55:::rbd_data.1120b21c3309.0000000000000029:head# 2024-09-17T21:30:09.953 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5d3d0a43:::rbd_data.1120d7424d8b.000000000000001e:e# 2024-09-17T21:30:09.953 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5d3d0a43:::rbd_data.1120d7424d8b.000000000000001e:head# 2024-09-17T21:30:09.953 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5da731dc:::rbd_data.1120818f9bfd.000000000000000b:head# 2024-09-17T21:30:09.955 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5dde59b0:::rbd_data.1120b21c3309.000000000000009e:head# 2024-09-17T21:30:09.955 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5e28433d:::rbd_data.1120e62eea89.0000000000000005:head# 2024-09-17T21:30:09.956 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5e7dbd4c:::rbd_data.11207744cc2c.0000000000000022:5# 2024-09-17T21:30:09.957 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5e7dbd4c:::rbd_data.11207744cc2c.0000000000000022:head# 2024-09-17T21:30:09.959 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5f070e49:::rbd_data.1120b21c3309.0000000000000125:head# 2024-09-17T21:30:09.959 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5f6403b3:::rbd_data.1120b21c3309.0000000000000051:head# 2024-09-17T21:30:09.959 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5f730104:::rbd_data.1120b21c3309.0000000000000179:head# 2024-09-17T21:30:09.959 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5f7349db:::rbd_data.1120b21c3309.00000000000000ff:head# 2024-09-17T21:30:09.959 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:5fdea744:::rbd_data.1120b21c3309.0000000000000096:head# 2024-09-17T21:30:09.959 INFO:teuthology.orchestra.run.smithi129.stderr:Export successful 2024-09-17T21:30:09.962 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:09.962 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:09.972 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:30:10.032 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:10.030+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:10.115 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:10.115 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:10.123 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:30:10.167 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:10.165+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:10.226 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:10.226 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:10.232 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --force --op remove --pgid 3.1a 2024-09-17T21:30:10.236 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:30:10.237 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:10.233+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:10.334 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:10.332+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:10.381 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:10.381 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:10.389 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:30:10.435 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:10.433+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:10.480 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:30:10.504 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:10.504 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:10.513 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:30:10.535 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:10.533+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:10.600 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:10.600 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:10.601 INFO:teuthology.orchestra.run.smithi129.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-17T21:30:10.617 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is unset 2024-09-17T21:30:10.619 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:10.619 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:10.621 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:10.621 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:10.629 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:30:10.639 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:10.639 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:10.655 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:10.655 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:10.671 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:10.672 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:10.673 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:10.671+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:10.688 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:10.688 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:10.704 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:10.704 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:10.707 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:30:10.707 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:30:10.736 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:10.736 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:10.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:10.736 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:10.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.737 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21342 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2082816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:10.737 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:10.737 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 21.056956231000001, 2024-09-17T21:30:10.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022083523000000001, 2024-09-17T21:30:10.737 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.737 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.737 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.737 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.737 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.737 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.737 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21342 2024-09-17T21:30:10.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.738 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:10.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:10.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:10.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:30:10.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676659+0000", 2024-09-17T21:30:10.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0950000000000001e-06 2024-09-17T21:30:10.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676662+0000", 2024-09-17T21:30:10.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.971e-06 2024-09-17T21:30:10.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676665+0000", 2024-09-17T21:30:10.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6189999999999998e-06 2024-09-17T21:30:10.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698568+0000", 2024-09-17T21:30:10.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021903549000000001 2024-09-17T21:30:10.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698668+0000", 2024-09-17T21:30:10.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9778999999999995e-05 2024-09-17T21:30:10.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698741+0000", 2024-09-17T21:30:10.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000006e-05 2024-09-17T21:30:10.741 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.741 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.741 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.742 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21343 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2095104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:10.742 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:10.742 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 21.056946292999999, 2024-09-17T21:30:10.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022504024000000001, 2024-09-17T21:30:10.742 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.742 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.742 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.742 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.742 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.742 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.742 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21343 2024-09-17T21:30:10.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.743 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:10.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:10.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:10.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:30:10.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676669+0000", 2024-09-17T21:30:10.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.003e-06 2024-09-17T21:30:10.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676672+0000", 2024-09-17T21:30:10.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9280000000000002e-06 2024-09-17T21:30:10.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676674+0000", 2024-09-17T21:30:10.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5469999999999999e-06 2024-09-17T21:30:10.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699015+0000", 2024-09-17T21:30:10.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022340563000000001 2024-09-17T21:30:10.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699102+0000", 2024-09-17T21:30:10.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7425999999999999e-05 2024-09-17T21:30:10.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699172+0000", 2024-09-17T21:30:10.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9346999999999999e-05 2024-09-17T21:30:10.746 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.746 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.746 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.746 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45576 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:10.747 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:10.747 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.0287124929999996, 2024-09-17T21:30:10.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022024660000000001, 2024-09-17T21:30:10.747 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.747 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.747 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.747 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.747 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.747 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.747 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45576 2024-09-17T21:30:10.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.747 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:10.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:10.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704902+0000", 2024-09-17T21:30:10.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9100000000000004e-07 2024-09-17T21:30:10.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704905+0000", 2024-09-17T21:30:10.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6139999999999999e-06 2024-09-17T21:30:10.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704911+0000", 2024-09-17T21:30:10.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8619999999999997e-06 2024-09-17T21:30:10.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704917+0000", 2024-09-17T21:30:10.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:30:10.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726681+0000", 2024-09-17T21:30:10.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021764569000000001 2024-09-17T21:30:10.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726726+0000", 2024-09-17T21:30:10.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4218999999999998e-05 2024-09-17T21:30:10.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726926+0000", 2024-09-17T21:30:10.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000200466 2024-09-17T21:30:10.751 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.751 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.751 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.751 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45577 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:10.751 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:10.751 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.0286888369999998, 2024-09-17T21:30:10.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022269694999999999, 2024-09-17T21:30:10.751 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.752 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.752 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.752 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.752 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.752 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.752 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45577 2024-09-17T21:30:10.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.752 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:10.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:10.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704926+0000", 2024-09-17T21:30:10.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:30:10.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704928+0000", 2024-09-17T21:30:10.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9989999999999998e-06 2024-09-17T21:30:10.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704934+0000", 2024-09-17T21:30:10.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:30:10.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704939+0000", 2024-09-17T21:30:10.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5909999999999998e-06 2024-09-17T21:30:10.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726954+0000", 2024-09-17T21:30:10.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022014885000000001 2024-09-17T21:30:10.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726995+0000", 2024-09-17T21:30:10.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1474000000000003e-05 2024-09-17T21:30:10.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727195+0000", 2024-09-17T21:30:10.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019968300000000001 2024-09-17T21:30:10.755 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.756 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.756 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.756 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45578 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:10.756 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:10.756 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.0286675929999998, 2024-09-17T21:30:10.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022484006000000001, 2024-09-17T21:30:10.756 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.756 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.756 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.756 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.757 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.757 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.757 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45578 2024-09-17T21:30:10.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.757 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:10.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:10.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704947+0000", 2024-09-17T21:30:10.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1399999999999997e-07 2024-09-17T21:30:10.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704949+0000", 2024-09-17T21:30:10.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0959999999999999e-06 2024-09-17T21:30:10.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704956+0000", 2024-09-17T21:30:10.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9330000000000004e-06 2024-09-17T21:30:10.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704961+0000", 2024-09-17T21:30:10.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6759999999999998e-06 2024-09-17T21:30:10.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727213+0000", 2024-09-17T21:30:10.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022252568 2024-09-17T21:30:10.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727259+0000", 2024-09-17T21:30:10.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5238000000000003e-05 2024-09-17T21:30:10.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727430+0000", 2024-09-17T21:30:10.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000171881 2024-09-17T21:30:10.760 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.760 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.760 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.761 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45579 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:10.761 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:10.761 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.028645665, 2024-09-17T21:30:10.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022749080000000001, 2024-09-17T21:30:10.761 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.761 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.761 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.761 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.761 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.761 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.761 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45579 2024-09-17T21:30:10.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.762 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:10.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:10.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704969+0000", 2024-09-17T21:30:10.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:30:10.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704971+0000", 2024-09-17T21:30:10.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.097e-06 2024-09-17T21:30:10.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704977+0000", 2024-09-17T21:30:10.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6329999999999999e-06 2024-09-17T21:30:10.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704983+0000", 2024-09-17T21:30:10.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:30:10.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727457+0000", 2024-09-17T21:30:10.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022474521000000001 2024-09-17T21:30:10.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727498+0000", 2024-09-17T21:30:10.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0558e-05 2024-09-17T21:30:10.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727717+0000", 2024-09-17T21:30:10.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219805 2024-09-17T21:30:10.765 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.765 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.766 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.766 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45650 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 20480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:10.766 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:10.766 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.0275223579999997, 2024-09-17T21:30:10.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021996219000000001, 2024-09-17T21:30:10.766 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.766 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.766 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.766 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.766 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.766 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.766 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45650 2024-09-17T21:30:10.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.767 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:10.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:10.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:10.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5199999999999997e-07 2024-09-17T21:30:10.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706095+0000", 2024-09-17T21:30:10.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6079999999999998e-06 2024-09-17T21:30:10.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706101+0000", 2024-09-17T21:30:10.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:30:10.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706106+0000", 2024-09-17T21:30:10.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T21:30:10.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728000+0000", 2024-09-17T21:30:10.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021893378000000002 2024-09-17T21:30:10.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728039+0000", 2024-09-17T21:30:10.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9249000000000003e-05 2024-09-17T21:30:10.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728088+0000", 2024-09-17T21:30:10.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8785000000000001e-05 2024-09-17T21:30:10.770 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.770 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.771 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.771 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45652 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:10.771 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:10.771 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.0274995269999998, 2024-09-17T21:30:10.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022100465999999999, 2024-09-17T21:30:10.771 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.771 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.771 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.771 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.771 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.771 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.772 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45652 2024-09-17T21:30:10.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.772 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:10.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:10.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706115+0000", 2024-09-17T21:30:10.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:30:10.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706117+0000", 2024-09-17T21:30:10.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-06 2024-09-17T21:30:10.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706122+0000", 2024-09-17T21:30:10.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.288e-06 2024-09-17T21:30:10.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706136+0000", 2024-09-17T21:30:10.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3409e-05 2024-09-17T21:30:10.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728112+0000", 2024-09-17T21:30:10.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021976724999999999 2024-09-17T21:30:10.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728151+0000", 2024-09-17T21:30:10.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8318999999999997e-05 2024-09-17T21:30:10.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728215+0000", 2024-09-17T21:30:10.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4172999999999997e-05 2024-09-17T21:30:10.775 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.775 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.775 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.775 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45653 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:10.775 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:10.776 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.0274703169999997, 2024-09-17T21:30:10.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022249517999999999, 2024-09-17T21:30:10.776 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.776 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.776 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.776 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.776 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.776 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.776 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45653 2024-09-17T21:30:10.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.776 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:10.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:10.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:10.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T21:30:10.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706147+0000", 2024-09-17T21:30:10.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4430000000000002e-06 2024-09-17T21:30:10.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706152+0000", 2024-09-17T21:30:10.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5350000000000004e-06 2024-09-17T21:30:10.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706157+0000", 2024-09-17T21:30:10.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8420000000000004e-06 2024-09-17T21:30:10.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728246+0000", 2024-09-17T21:30:10.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022089237000000001 2024-09-17T21:30:10.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728309+0000", 2024-09-17T21:30:10.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2356999999999999e-05 2024-09-17T21:30:10.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:30:10.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4596000000000001e-05 2024-09-17T21:30:10.780 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.780 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.780 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.780 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45657 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 36864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:10.780 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:10.780 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.0274390530000002, 2024-09-17T21:30:10.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022218252000000001, 2024-09-17T21:30:10.780 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.780 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.781 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.781 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.781 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.781 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.781 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45657 2024-09-17T21:30:10.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.781 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:10.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:10.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:10.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8299999999999997e-07 2024-09-17T21:30:10.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706178+0000", 2024-09-17T21:30:10.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3700000000000002e-06 2024-09-17T21:30:10.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706184+0000", 2024-09-17T21:30:10.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.144e-06 2024-09-17T21:30:10.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706190+0000", 2024-09-17T21:30:10.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8590000000000001e-06 2024-09-17T21:30:10.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728240+0000", 2024-09-17T21:30:10.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022049804999999999 2024-09-17T21:30:10.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728307+0000", 2024-09-17T21:30:10.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7477999999999993e-05 2024-09-17T21:30:10.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:30:10.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6112999999999994e-05 2024-09-17T21:30:10.784 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.784 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.784 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.785 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45658 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:10.785 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:10.785 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.0274159330000003, 2024-09-17T21:30:10.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022376455, 2024-09-17T21:30:10.785 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.785 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.785 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.785 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.785 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.785 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.785 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45658 2024-09-17T21:30:10.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.786 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:10.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:10.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:10.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:30:10.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706200+0000", 2024-09-17T21:30:10.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:30:10.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706206+0000", 2024-09-17T21:30:10.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7039999999999999e-06 2024-09-17T21:30:10.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706211+0000", 2024-09-17T21:30:10.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9429999999999999e-06 2024-09-17T21:30:10.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728418+0000", 2024-09-17T21:30:10.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022206760999999998 2024-09-17T21:30:10.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728497+0000", 2024-09-17T21:30:10.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9493999999999995e-05 2024-09-17T21:30:10.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728574+0000", 2024-09-17T21:30:10.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7180000000000003e-05 2024-09-17T21:30:10.789 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.789 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.789 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.789 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45706 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 172032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:10.789 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707876+0000", 2024-09-17T21:30:10.789 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.0257381880000001, 2024-09-17T21:30:10.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021993196999999999, 2024-09-17T21:30:10.790 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.790 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.790 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.790 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.790 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.790 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.790 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45706 2024-09-17T21:30:10.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.790 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707876+0000", 2024-09-17T21:30:10.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707876+0000", 2024-09-17T21:30:10.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707876+0000", 2024-09-17T21:30:10.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999998e-07 2024-09-17T21:30:10.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707878+0000", 2024-09-17T21:30:10.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7689999999999999e-06 2024-09-17T21:30:10.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707884+0000", 2024-09-17T21:30:10.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3990000000000001e-06 2024-09-17T21:30:10.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707895+0000", 2024-09-17T21:30:10.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0577000000000001e-05 2024-09-17T21:30:10.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729606+0000", 2024-09-17T21:30:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021711034000000001 2024-09-17T21:30:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729651+0000", 2024-09-17T21:30:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5337999999999999e-05 2024-09-17T21:30:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729869+0000", 2024-09-17T21:30:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00021769899999999999 2024-09-17T21:30:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.794 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.794 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.794 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45707 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 212992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:10.794 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:10.794 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.0257118670000001, 2024-09-17T21:30:10.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022235550999999999, 2024-09-17T21:30:10.794 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.794 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.794 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.794 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.794 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.794 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.795 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45707 2024-09-17T21:30:10.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.795 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:10.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:10.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:10.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:30:10.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707905+0000", 2024-09-17T21:30:10.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2120000000000002e-06 2024-09-17T21:30:10.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707911+0000", 2024-09-17T21:30:10.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1929999999999998e-06 2024-09-17T21:30:10.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707915+0000", 2024-09-17T21:30:10.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:30:10.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729897+0000", 2024-09-17T21:30:10.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021981186 2024-09-17T21:30:10.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729944+0000", 2024-09-17T21:30:10.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571000000000002e-05 2024-09-17T21:30:10.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730138+0000", 2024-09-17T21:30:10.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000193451 2024-09-17T21:30:10.798 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.798 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.799 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.799 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45708 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:10.799 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:10.799 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.0256917349999997, 2024-09-17T21:30:10.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022533742999999998, 2024-09-17T21:30:10.799 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.799 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.799 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.799 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.799 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.800 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.800 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45708 2024-09-17T21:30:10.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.800 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:10.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:10.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707923+0000", 2024-09-17T21:30:10.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:30:10.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707932+0000", 2024-09-17T21:30:10.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6430000000000007e-06 2024-09-17T21:30:10.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707939+0000", 2024-09-17T21:30:10.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3539999999999996e-06 2024-09-17T21:30:10.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707944+0000", 2024-09-17T21:30:10.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3249999999999998e-06 2024-09-17T21:30:10.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730176+0000", 2024-09-17T21:30:10.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022232393 2024-09-17T21:30:10.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730263+0000", 2024-09-17T21:30:10.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6638999999999995e-05 2024-09-17T21:30:10.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730456+0000", 2024-09-17T21:30:10.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019304899999999999 2024-09-17T21:30:10.803 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.803 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.803 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.803 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45752 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:10.804 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:10.804 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.025436515, 2024-09-17T21:30:10.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022426878000000001, 2024-09-17T21:30:10.804 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.804 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.804 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.804 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.804 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.804 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.804 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45752 2024-09-17T21:30:10.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.804 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:10.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:10.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708178+0000", 2024-09-17T21:30:10.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:30:10.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708181+0000", 2024-09-17T21:30:10.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.909e-06 2024-09-17T21:30:10.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708187+0000", 2024-09-17T21:30:10.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4999999999999996e-06 2024-09-17T21:30:10.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708193+0000", 2024-09-17T21:30:10.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:30:10.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730484+0000", 2024-09-17T21:30:10.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022291765000000002 2024-09-17T21:30:10.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730541+0000", 2024-09-17T21:30:10.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6483000000000001e-05 2024-09-17T21:30:10.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730604+0000", 2024-09-17T21:30:10.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3429000000000005e-05 2024-09-17T21:30:10.808 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.808 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.808 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.808 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45761 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:10.808 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:10.808 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.025408605, 2024-09-17T21:30:10.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022614408999999999, 2024-09-17T21:30:10.808 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.809 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.809 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.809 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.809 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.809 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.809 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45761 2024-09-17T21:30:10.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.809 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:10.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:10.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708206+0000", 2024-09-17T21:30:10.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4700000000000001e-07 2024-09-17T21:30:10.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708208+0000", 2024-09-17T21:30:10.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1509999999999998e-06 2024-09-17T21:30:10.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708214+0000", 2024-09-17T21:30:10.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.49e-06 2024-09-17T21:30:10.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708219+0000", 2024-09-17T21:30:10.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8119999999999998e-06 2024-09-17T21:30:10.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730641+0000", 2024-09-17T21:30:10.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022422765000000001 2024-09-17T21:30:10.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730731+0000", 2024-09-17T21:30:10.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9598999999999994e-05 2024-09-17T21:30:10.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730820+0000", 2024-09-17T21:30:10.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8844999999999994e-05 2024-09-17T21:30:10.813 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.813 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.813 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.813 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45778 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:10.813 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:10.813 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.0246650009999998, 2024-09-17T21:30:10.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022018544000000001, 2024-09-17T21:30:10.814 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.814 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.814 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.814 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.814 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.814 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.814 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45778 2024-09-17T21:30:10.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.814 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:10.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:10.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708950+0000", 2024-09-17T21:30:10.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-17T21:30:10.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709307+0000", 2024-09-17T21:30:10.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000356848 2024-09-17T21:30:10.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709316+0000", 2024-09-17T21:30:10.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1430000000000004e-06 2024-09-17T21:30:10.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709328+0000", 2024-09-17T21:30:10.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2629999999999999e-05 2024-09-17T21:30:10.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730861+0000", 2024-09-17T21:30:10.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021532057 2024-09-17T21:30:10.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730905+0000", 2024-09-17T21:30:10.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4907000000000002e-05 2024-09-17T21:30:10.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730968+0000", 2024-09-17T21:30:10.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2108999999999998e-05 2024-09-17T21:30:10.817 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.818 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.818 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.818 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45780 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 90112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:10.818 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709383+0000", 2024-09-17T21:30:10.818 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.0242312910000004, 2024-09-17T21:30:10.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021946739999999999, 2024-09-17T21:30:10.818 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.818 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.818 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.818 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.818 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.819 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.819 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45780 2024-09-17T21:30:10.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.819 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709383+0000", 2024-09-17T21:30:10.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709383+0000", 2024-09-17T21:30:10.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709383+0000", 2024-09-17T21:30:10.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9200000000000001e-07 2024-09-17T21:30:10.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709386+0000", 2024-09-17T21:30:10.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:30:10.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709392+0000", 2024-09-17T21:30:10.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9369999999999998e-06 2024-09-17T21:30:10.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709398+0000", 2024-09-17T21:30:10.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3489999999999999e-06 2024-09-17T21:30:10.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731170+0000", 2024-09-17T21:30:10.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021772657000000001 2024-09-17T21:30:10.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731244+0000", 2024-09-17T21:30:10.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3466000000000004e-05 2024-09-17T21:30:10.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731329+0000", 2024-09-17T21:30:10.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5498e-05 2024-09-17T21:30:10.822 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.822 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.822 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.822 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45785 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:10.822 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:10.823 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.0241868180000004, 2024-09-17T21:30:10.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022044113000000001, 2024-09-17T21:30:10.823 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.823 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.823 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.823 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.823 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.823 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.823 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45785 2024-09-17T21:30:10.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.823 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:10.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:10.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709428+0000", 2024-09-17T21:30:10.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:30:10.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709430+0000", 2024-09-17T21:30:10.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:30:10.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709451+0000", 2024-09-17T21:30:10.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1372000000000001e-05 2024-09-17T21:30:10.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709472+0000", 2024-09-17T21:30:10.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1007999999999999e-05 2024-09-17T21:30:10.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731293+0000", 2024-09-17T21:30:10.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021821320000000002 2024-09-17T21:30:10.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731378+0000", 2024-09-17T21:30:10.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5059e-05 2024-09-17T21:30:10.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731471+0000", 2024-09-17T21:30:10.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2867000000000004e-05 2024-09-17T21:30:10.827 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.827 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.827 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.827 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45786 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:10.827 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:10.827 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.0241280210000001, 2024-09-17T21:30:10.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022181821000000001, 2024-09-17T21:30:10.828 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:10.828 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:10.828 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:10.828 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:10.828 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:10.828 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:10.828 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45786 2024-09-17T21:30:10.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.828 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:10.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:10.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:10.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:10.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:10.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:10.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:10.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709487+0000", 2024-09-17T21:30:10.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1299999999999999e-07 2024-09-17T21:30:10.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:10.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709491+0000", 2024-09-17T21:30:10.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2089999999999999e-06 2024-09-17T21:30:10.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:10.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709498+0000", 2024-09-17T21:30:10.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3980000000000002e-06 2024-09-17T21:30:10.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:10.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709504+0000", 2024-09-17T21:30:10.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7219999999999996e-06 2024-09-17T21:30:10.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:10.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731499+0000", 2024-09-17T21:30:10.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021995297 2024-09-17T21:30:10.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:10.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731591+0000", 2024-09-17T21:30:10.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1373999999999996e-05 2024-09-17T21:30:10.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:10.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:10.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:10.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731668+0000", 2024-09-17T21:30:10.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7107999999999994e-05 2024-09-17T21:30:10.831 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.832 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.832 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.832 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:10.832 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:10.832 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:10.833 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:30:10.834 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:10.830+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:10.934 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:10.931+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:10.986 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:10.986 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:10.986 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:10.986 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:10.994 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:30:11.035 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:11.032+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:11.101 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:11.101 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:11.101 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:11.102 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:11.102 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:11.110 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:30:11.154 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:11.152+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:11.221 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:11.221 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:11.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:11.222 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:11.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.222 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10799 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.222 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:30:11.222 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.153411563999999, 2024-09-17T21:30:11.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039122745, 2024-09-17T21:30:11.222 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.222 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.222 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:11.222 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.222 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.222 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.222 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10799 2024-09-17T21:30:11.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.223 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:30:11.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:30:11.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065469+0000", 2024-09-17T21:30:11.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:30:11.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.224 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065477+0000", 2024-09-17T21:30:11.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1319999999999994e-06 2024-09-17T21:30:11.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.224 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065483+0000", 2024-09-17T21:30:11.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2040000000000002e-06 2024-09-17T21:30:11.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065487+0000", 2024-09-17T21:30:11.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.42e-06 2024-09-17T21:30:11.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099984+0000", 2024-09-17T21:30:11.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034496484000000001 2024-09-17T21:30:11.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100065+0000", 2024-09-17T21:30:11.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1200999999999997e-05 2024-09-17T21:30:11.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:11.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100422+0000", 2024-09-17T21:30:11.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035657700000000001 2024-09-17T21:30:11.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:11.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104214+0000", 2024-09-17T21:30:11.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003792045 2024-09-17T21:30:11.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:11.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104505+0000", 2024-09-17T21:30:11.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029141399999999999 2024-09-17T21:30:11.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:11.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104556+0000", 2024-09-17T21:30:11.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1388999999999999e-05 2024-09-17T21:30:11.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104591+0000", 2024-09-17T21:30:11.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4455999999999997e-05 2024-09-17T21:30:11.227 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.227 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.227 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.228 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12187 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.228 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:11.228 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.079318832999999, 2024-09-17T21:30:11.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038825750999999999, 2024-09-17T21:30:11.228 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.228 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.228 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.228 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.228 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.228 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.228 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12187 2024-09-17T21:30:11.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.228 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:11.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:11.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:11.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:30:11.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139563+0000", 2024-09-17T21:30:11.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.624e-06 2024-09-17T21:30:11.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139569+0000", 2024-09-17T21:30:11.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1090000000000003e-06 2024-09-17T21:30:11.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139573+0000", 2024-09-17T21:30:11.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4429999999999996e-06 2024-09-17T21:30:11.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178206+0000", 2024-09-17T21:30:11.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038632376000000003 2024-09-17T21:30:11.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178282+0000", 2024-09-17T21:30:11.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6076999999999997e-05 2024-09-17T21:30:11.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178387+0000", 2024-09-17T21:30:11.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104682 2024-09-17T21:30:11.231 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.231 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.232 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.232 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12188 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.232 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:30:11.232 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.079300652000001, 2024-09-17T21:30:11.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039053826999999999, 2024-09-17T21:30:11.232 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.232 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.232 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.232 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.232 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.233 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.233 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12188 2024-09-17T21:30:11.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.233 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:30:11.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:30:11.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139746+0000", 2024-09-17T21:30:11.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000167281 2024-09-17T21:30:11.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139749+0000", 2024-09-17T21:30:11.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9660000000000001e-06 2024-09-17T21:30:11.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139757+0000", 2024-09-17T21:30:11.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8029999999999997e-06 2024-09-17T21:30:11.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139771+0000", 2024-09-17T21:30:11.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3541e-05 2024-09-17T21:30:11.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178415+0000", 2024-09-17T21:30:11.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038644803999999998 2024-09-17T21:30:11.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178517+0000", 2024-09-17T21:30:11.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101298 2024-09-17T21:30:11.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178633+0000", 2024-09-17T21:30:11.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116134 2024-09-17T21:30:11.236 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.236 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.236 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.236 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12189 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.236 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:30:11.236 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.079100175000001, 2024-09-17T21:30:11.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039135991000000002, 2024-09-17T21:30:11.236 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.236 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.236 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.237 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.237 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.237 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.237 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12189 2024-09-17T21:30:11.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.237 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:30:11.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:30:11.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139780+0000", 2024-09-17T21:30:11.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:30:11.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139782+0000", 2024-09-17T21:30:11.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1950000000000002e-06 2024-09-17T21:30:11.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139788+0000", 2024-09-17T21:30:11.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0639999999999999e-06 2024-09-17T21:30:11.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139792+0000", 2024-09-17T21:30:11.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1960000000000001e-06 2024-09-17T21:30:11.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178671+0000", 2024-09-17T21:30:11.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038878176 2024-09-17T21:30:11.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178778+0000", 2024-09-17T21:30:11.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107278 2024-09-17T21:30:11.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178915+0000", 2024-09-17T21:30:11.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013757900000000001 2024-09-17T21:30:11.240 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.240 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.240 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.240 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12190 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.241 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:11.241 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.079080992000002, 2024-09-17T21:30:11.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039343781000000001, 2024-09-17T21:30:11.241 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.241 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.241 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.241 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12190 2024-09-17T21:30:11.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.241 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:11.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:11.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:11.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:11.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139801+0000", 2024-09-17T21:30:11.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.916e-06 2024-09-17T21:30:11.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139807+0000", 2024-09-17T21:30:11.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4540000000000004e-06 2024-09-17T21:30:11.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139812+0000", 2024-09-17T21:30:11.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9729999999999997e-06 2024-09-17T21:30:11.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178960+0000", 2024-09-17T21:30:11.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039147567000000001 2024-09-17T21:30:11.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179051+0000", 2024-09-17T21:30:11.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0754000000000006e-05 2024-09-17T21:30:11.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179142+0000", 2024-09-17T21:30:11.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1712999999999994e-05 2024-09-17T21:30:11.244 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.244 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.245 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.245 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12191 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.245 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:11.245 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.079061134, 2024-09-17T21:30:11.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039464336000000003, 2024-09-17T21:30:11.245 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.245 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.245 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.246 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.246 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.246 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.246 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12191 2024-09-17T21:30:11.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.246 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:11.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:11.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:11.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:30:11.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139821+0000", 2024-09-17T21:30:11.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1260000000000001e-06 2024-09-17T21:30:11.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139827+0000", 2024-09-17T21:30:11.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T21:30:11.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139840+0000", 2024-09-17T21:30:11.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2607e-05 2024-09-17T21:30:11.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179162+0000", 2024-09-17T21:30:11.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039322863999999999 2024-09-17T21:30:11.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179204+0000", 2024-09-17T21:30:11.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1938999999999999e-05 2024-09-17T21:30:11.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179283+0000", 2024-09-17T21:30:11.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8547000000000005e-05 2024-09-17T21:30:11.249 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.249 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.250 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.250 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12192 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.250 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:30:11.250 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.079033421999998, 2024-09-17T21:30:11.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039652318999999998, 2024-09-17T21:30:11.250 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.250 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.250 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.250 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.250 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.250 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.250 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12192 2024-09-17T21:30:11.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.251 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:30:11.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:30:11.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141458+0000", 2024-09-17T21:30:11.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001612098 2024-09-17T21:30:11.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141469+0000", 2024-09-17T21:30:11.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-17T21:30:11.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141478+0000", 2024-09-17T21:30:11.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T21:30:11.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141485+0000", 2024-09-17T21:30:11.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4760000000000003e-06 2024-09-17T21:30:11.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179317+0000", 2024-09-17T21:30:11.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037831832000000003 2024-09-17T21:30:11.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179395+0000", 2024-09-17T21:30:11.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8349000000000005e-05 2024-09-17T21:30:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179499+0000", 2024-09-17T21:30:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103628 2024-09-17T21:30:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12257 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.076844613999999, 2024-09-17T21:30:11.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038862751000000001, 2024-09-17T21:30:11.255 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.255 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.255 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.255 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.255 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.255 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.255 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12257 2024-09-17T21:30:11.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.255 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:11.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:11.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:11.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T21:30:11.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142037+0000", 2024-09-17T21:30:11.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.03e-06 2024-09-17T21:30:11.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142043+0000", 2024-09-17T21:30:11.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6319999999999996e-06 2024-09-17T21:30:11.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142052+0000", 2024-09-17T21:30:11.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2709999999999998e-06 2024-09-17T21:30:11.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180794+0000", 2024-09-17T21:30:11.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038741758000000001 2024-09-17T21:30:11.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180843+0000", 2024-09-17T21:30:11.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9021999999999999e-05 2024-09-17T21:30:11.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180898+0000", 2024-09-17T21:30:11.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673000000000003e-05 2024-09-17T21:30:11.259 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.259 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.259 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.259 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12258 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.259 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:11.259 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.076821097, 2024-09-17T21:30:11.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038950717000000003, 2024-09-17T21:30:11.259 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.259 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.259 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.259 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.260 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.260 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.260 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12258 2024-09-17T21:30:11.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.260 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:11.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:11.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:11.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199999999999999e-07 2024-09-17T21:30:11.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142061+0000", 2024-09-17T21:30:11.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8279999999999999e-06 2024-09-17T21:30:11.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142067+0000", 2024-09-17T21:30:11.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5600000000000001e-06 2024-09-17T21:30:11.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142071+0000", 2024-09-17T21:30:11.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6240000000000001e-06 2024-09-17T21:30:11.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180921+0000", 2024-09-17T21:30:11.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038849366000000003 2024-09-17T21:30:11.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180961+0000", 2024-09-17T21:30:11.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0219000000000002e-05 2024-09-17T21:30:11.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181009+0000", 2024-09-17T21:30:11.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8437999999999999e-05 2024-09-17T21:30:11.263 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.263 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.263 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.263 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12259 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.264 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:11.264 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.076793850000001, 2024-09-17T21:30:11.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039116832999999997, 2024-09-17T21:30:11.264 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.264 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.264 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.264 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.264 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.264 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.264 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12259 2024-09-17T21:30:11.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.264 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:11.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:11.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:11.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2200000000000004e-07 2024-09-17T21:30:11.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142088+0000", 2024-09-17T21:30:11.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0329999999999998e-06 2024-09-17T21:30:11.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142095+0000", 2024-09-17T21:30:11.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4320000000000004e-06 2024-09-17T21:30:11.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142100+0000", 2024-09-17T21:30:11.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:30:11.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181037+0000", 2024-09-17T21:30:11.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038937601000000002 2024-09-17T21:30:11.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181114+0000", 2024-09-17T21:30:11.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6857e-05 2024-09-17T21:30:11.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181203+0000", 2024-09-17T21:30:11.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8437999999999996e-05 2024-09-17T21:30:11.268 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.268 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.268 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.268 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12260 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.268 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:30:11.268 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.076773563, 2024-09-17T21:30:11.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039204184000000003, 2024-09-17T21:30:11.268 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.268 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.269 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.269 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.269 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.269 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.269 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12260 2024-09-17T21:30:11.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.270 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:30:11.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:30:11.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142107+0000", 2024-09-17T21:30:11.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:30:11.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142108+0000", 2024-09-17T21:30:11.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.753e-06 2024-09-17T21:30:11.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142114+0000", 2024-09-17T21:30:11.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7660000000000004e-06 2024-09-17T21:30:11.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142119+0000", 2024-09-17T21:30:11.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:30:11.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181216+0000", 2024-09-17T21:30:11.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039097191000000003 2024-09-17T21:30:11.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181260+0000", 2024-09-17T21:30:11.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4731000000000003e-05 2024-09-17T21:30:11.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181310+0000", 2024-09-17T21:30:11.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9818000000000003e-05 2024-09-17T21:30:11.273 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.273 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.273 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.273 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12261 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.273 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:30:11.273 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.076755239000001, 2024-09-17T21:30:11.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039292200999999999, 2024-09-17T21:30:11.273 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.274 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.274 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.274 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.274 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.274 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.274 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12261 2024-09-17T21:30:11.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.274 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:30:11.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:30:11.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142125+0000", 2024-09-17T21:30:11.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:30:11.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142127+0000", 2024-09-17T21:30:11.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.792e-06 2024-09-17T21:30:11.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142132+0000", 2024-09-17T21:30:11.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3979999999999998e-06 2024-09-17T21:30:11.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142136+0000", 2024-09-17T21:30:11.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.211e-06 2024-09-17T21:30:11.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181324+0000", 2024-09-17T21:30:11.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039187560000000003 2024-09-17T21:30:11.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181363+0000", 2024-09-17T21:30:11.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9375000000000002e-05 2024-09-17T21:30:11.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181417+0000", 2024-09-17T21:30:11.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3372000000000003e-05 2024-09-17T21:30:11.277 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.278 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.278 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.278 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12262 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.278 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:30:11.278 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.076737445999999, 2024-09-17T21:30:11.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039384875, 2024-09-17T21:30:11.278 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.278 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.278 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.278 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.278 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.279 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.279 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12262 2024-09-17T21:30:11.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.279 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:30:11.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:30:11.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142143+0000", 2024-09-17T21:30:11.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:30:11.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142148+0000", 2024-09-17T21:30:11.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4360000000000001e-06 2024-09-17T21:30:11.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142154+0000", 2024-09-17T21:30:11.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8719999999999999e-06 2024-09-17T21:30:11.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142158+0000", 2024-09-17T21:30:11.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.476e-06 2024-09-17T21:30:11.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181436+0000", 2024-09-17T21:30:11.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039277919000000001 2024-09-17T21:30:11.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181480+0000", 2024-09-17T21:30:11.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3649999999999997e-05 2024-09-17T21:30:11.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181527+0000", 2024-09-17T21:30:11.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7131e-05 2024-09-17T21:30:11.282 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.282 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.282 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.282 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12263 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1079296~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.282 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:11.282 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.076715011000001, 2024-09-17T21:30:11.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039473339000000003, 2024-09-17T21:30:11.283 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.283 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.283 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.283 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.283 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.283 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.283 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12263 2024-09-17T21:30:11.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.283 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:11.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:11.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:11.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:30:11.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142167+0000", 2024-09-17T21:30:11.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.81e-06 2024-09-17T21:30:11.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142172+0000", 2024-09-17T21:30:11.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:30:11.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142177+0000", 2024-09-17T21:30:11.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T21:30:11.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181553+0000", 2024-09-17T21:30:11.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039376130000000002 2024-09-17T21:30:11.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181592+0000", 2024-09-17T21:30:11.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9153999999999998e-05 2024-09-17T21:30:11.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181638+0000", 2024-09-17T21:30:11.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5864e-05 2024-09-17T21:30:11.286 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.287 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.287 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.287 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12278 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1095680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.287 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:11.287 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.076697035999999, 2024-09-17T21:30:11.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039454222999999997, 2024-09-17T21:30:11.287 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.287 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.287 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.287 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.287 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.288 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.288 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12278 2024-09-17T21:30:11.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.288 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:11.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:11.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:11.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:30:11.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142185+0000", 2024-09-17T21:30:11.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.697e-06 2024-09-17T21:30:11.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142191+0000", 2024-09-17T21:30:11.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:30:11.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142196+0000", 2024-09-17T21:30:11.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4829999999999999e-06 2024-09-17T21:30:11.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181548+0000", 2024-09-17T21:30:11.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039351200000000003 2024-09-17T21:30:11.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181591+0000", 2024-09-17T21:30:11.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3492000000000003e-05 2024-09-17T21:30:11.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181637+0000", 2024-09-17T21:30:11.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5847000000000003e-05 2024-09-17T21:30:11.291 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.291 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.291 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.291 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12279 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.292 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:11.292 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.076673747000001, 2024-09-17T21:30:11.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039539801999999999, 2024-09-17T21:30:11.292 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.292 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.292 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.292 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.292 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.292 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.292 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12279 2024-09-17T21:30:11.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.292 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:11.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:11.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:11.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.89e-07 2024-09-17T21:30:11.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142208+0000", 2024-09-17T21:30:11.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7579999999999999e-06 2024-09-17T21:30:11.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142214+0000", 2024-09-17T21:30:11.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:30:11.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142218+0000", 2024-09-17T21:30:11.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3429999999999998e-06 2024-09-17T21:30:11.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181653+0000", 2024-09-17T21:30:11.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039435322000000002 2024-09-17T21:30:11.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181698+0000", 2024-09-17T21:30:11.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4978999999999997e-05 2024-09-17T21:30:11.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181746+0000", 2024-09-17T21:30:11.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571999999999998e-05 2024-09-17T21:30:11.296 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.296 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.296 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.296 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12280 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1124352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.296 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:30:11.296 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.076656228000001, 2024-09-17T21:30:11.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039704358000000002, 2024-09-17T21:30:11.296 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.297 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.297 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.297 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.297 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.297 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.297 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12280 2024-09-17T21:30:11.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.297 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:30:11.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:30:11.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142224+0000", 2024-09-17T21:30:11.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:30:11.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142226+0000", 2024-09-17T21:30:11.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:30:11.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142231+0000", 2024-09-17T21:30:11.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:30:11.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142240+0000", 2024-09-17T21:30:11.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4590000000000004e-06 2024-09-17T21:30:11.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181774+0000", 2024-09-17T21:30:11.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039533833999999997 2024-09-17T21:30:11.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181859+0000", 2024-09-17T21:30:11.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4328999999999998e-05 2024-09-17T21:30:11.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181928+0000", 2024-09-17T21:30:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9139000000000003e-05 2024-09-17T21:30:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12281 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1136640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:30:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.076633321999999, 2024-09-17T21:30:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039787799999999998, 2024-09-17T21:30:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12281 2024-09-17T21:30:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:30:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:30:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142309+0000", 2024-09-17T21:30:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2323999999999995e-05 2024-09-17T21:30:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142317+0000", 2024-09-17T21:30:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1859999999999992e-06 2024-09-17T21:30:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142324+0000", 2024-09-17T21:30:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4699999999999996e-06 2024-09-17T21:30:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142330+0000", 2024-09-17T21:30:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4650000000000002e-06 2024-09-17T21:30:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181948+0000", 2024-09-17T21:30:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039618056999999998 2024-09-17T21:30:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181986+0000", 2024-09-17T21:30:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8442999999999998e-05 2024-09-17T21:30:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182034+0000", 2024-09-17T21:30:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7855000000000002e-05 2024-09-17T21:30:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12282 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1150976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.076541092999999, 2024-09-17T21:30:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039871919999999998, 2024-09-17T21:30:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12282 2024-09-17T21:30:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:30:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142341+0000", 2024-09-17T21:30:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.232e-06 2024-09-17T21:30:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142348+0000", 2024-09-17T21:30:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7320000000000001e-06 2024-09-17T21:30:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142353+0000", 2024-09-17T21:30:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:30:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182062+0000", 2024-09-17T21:30:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039709739000000001 2024-09-17T21:30:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182128+0000", 2024-09-17T21:30:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5739000000000002e-05 2024-09-17T21:30:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182210+0000", 2024-09-17T21:30:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:30:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12283 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:30:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.076520682000002, 2024-09-17T21:30:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039896434000000001, 2024-09-17T21:30:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:30:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12283 2024-09-17T21:30:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:30:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:30:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144852+0000", 2024-09-17T21:30:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024934079999999999 2024-09-17T21:30:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144856+0000", 2024-09-17T21:30:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:30:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144864+0000", 2024-09-17T21:30:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0160000000000008e-06 2024-09-17T21:30:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144869+0000", 2024-09-17T21:30:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:30:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182246+0000", 2024-09-17T21:30:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037377207000000003 2024-09-17T21:30:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182255+0000", 2024-09-17T21:30:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1840000000000005e-06 2024-09-17T21:30:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.314 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:11.316 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:30:11.316 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:11.313+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:11.345 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-17T21:30:11.346 INFO:teuthology.orchestra.run.smithi129.stdout:1683 read 0x9db8307 thru 0x9db99b2 (0x16ac bytes) 2024-09-17T21:30:11.347 INFO:teuthology.orchestra.run.smithi129.stdout:1684 write 0x9a773d thru 0x9b75a9 (0xfe6d bytes) 2024-09-17T21:30:11.365 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:11.365 INFO:teuthology.orchestra.run.smithi129.stdout:1686 punch from 0x4cedc8f to 0x4cfc670, (0xe9e1 bytes) 2024-09-17T21:30:11.365 INFO:teuthology.orchestra.run.smithi129.stdout:1689 write 0x192f591 thru 0x1933c69 (0x46d9 bytes) 2024-09-17T21:30:11.408 INFO:teuthology.orchestra.run.smithi129.stdout:1691 punch from 0x4b95e08 to 0x4b9a757, (0x494f bytes) 2024-09-17T21:30:11.408 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:11.408 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:11.408 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:11.408 INFO:teuthology.orchestra.run.smithi129.stdout:1696 punch from 0x78e4611 to 0x78f15fc, (0xcfeb bytes) 2024-09-17T21:30:11.408 INFO:teuthology.orchestra.run.smithi129.stdout:1699 punch from 0x5d4dceb to 0x5d50684, (0x2999 bytes) 2024-09-17T21:30:11.409 INFO:teuthology.orchestra.run.smithi129.stdout:1700 read 0x31781d6 thru 0x317e9b1 (0x67dc bytes) 2024-09-17T21:30:11.409 INFO:teuthology.orchestra.run.smithi129.stdout:1704 write 0x983fc64 thru 0x9841bb6 (0x1f53 bytes) 2024-09-17T21:30:11.416 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:11.414+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:11.459 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:11.464 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:11.464 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:11.464 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:11.464 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:11.473 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:30:11.473 INFO:teuthology.orchestra.run.smithi129.stdout:1707 trunc from 0xcac9e07 to 0x1307d6f 2024-09-17T21:30:11.517 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:11.515+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:11.572 INFO:teuthology.orchestra.run.smithi129.stdout:1709 writesame 0xe1d6a69 thru 0xe1db0ed data_size 0x1(0x4685 bytes) 2024-09-17T21:30:11.573 INFO:teuthology.orchestra.run.smithi129.stdout:1710 punch from 0x193e630 to 0x194276b, (0x413b bytes) 2024-09-17T21:30:11.580 INFO:teuthology.orchestra.run.smithi129.stdout:1711 trunc from 0xe1db0ee to 0x8829802 2024-09-17T21:30:11.580 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:11.580 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:11.580 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:11.580 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:11.581 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:11.587 INFO:teuthology.orchestra.run.smithi129.stdout:1712 write 0x3e964f8 thru 0x3ea3677 (0xd180 bytes) 2024-09-17T21:30:11.589 INFO:teuthology.orchestra.run.smithi129.stdout:1713 read 0x4383f24 thru 0x438e0e4 (0xa1c1 bytes) 2024-09-17T21:30:11.590 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:30:11.590 INFO:teuthology.orchestra.run.smithi129.stdout:1714 write 0x46810f2 thru 0x468e5b4 (0xd4c3 bytes) 2024-09-17T21:30:11.591 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:11.591 INFO:teuthology.orchestra.run.smithi129.stdout:1721 punch from 0x825b0ef to 0x8263287, (0x8198 bytes) 2024-09-17T21:30:11.604 INFO:teuthology.orchestra.run.smithi129.stdout:1722 writesame 0xed6f146 thru 0xed72013 data_size 0x3(0x2ece bytes) 2024-09-17T21:30:11.605 INFO:teuthology.orchestra.run.smithi129.stdout:1723 punch from 0x9f07c53 to 0x9f09c27, (0x1fd4 bytes) 2024-09-17T21:30:11.610 INFO:teuthology.orchestra.run.smithi129.stdout:1725 trunc from 0xed72014 to 0xabfa6a9 2024-09-17T21:30:11.616 INFO:teuthology.orchestra.run.smithi129.stdout:1728 write 0x5714340 thru 0x57183d2 (0x4093 bytes) 2024-09-17T21:30:11.617 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:11.625 INFO:teuthology.orchestra.run.smithi129.stdout:1730 trunc from 0xabfa6a9 to 0x4643699 2024-09-17T21:30:11.631 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:11.629+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:11.641 INFO:teuthology.orchestra.run.smithi129.stdout:1733 write 0x7fdbdfe thru 0x7fe8cca (0xcecd bytes) 2024-09-17T21:30:11.643 INFO:teuthology.orchestra.run.smithi129.stdout:1735 punch from 0x26aafd3 to 0x26af1ea, (0x4217 bytes) 2024-09-17T21:30:11.643 INFO:teuthology.orchestra.run.smithi129.stdout:1737 write 0xa64cd7 thru 0xa73acb (0xedf5 bytes) 2024-09-17T21:30:11.645 INFO:teuthology.orchestra.run.smithi129.stdout:1738 read 0x5f655b1 thru 0x5f708db (0xb32b bytes) 2024-09-17T21:30:11.645 INFO:teuthology.orchestra.run.smithi129.stdout:1740 writesame 0x6cfddb0 thru 0x6d03c03 data_size 0x2(0x5e54 bytes) 2024-09-17T21:30:11.649 INFO:teuthology.orchestra.run.smithi129.stdout:1742 write 0x5617abc thru 0x561bebe (0x4403 bytes) 2024-09-17T21:30:11.650 INFO:teuthology.orchestra.run.smithi129.stdout:1744 read 0x6c4191 thru 0x6cc065 (0x7ed5 bytes) 2024-09-17T21:30:11.658 INFO:teuthology.orchestra.run.smithi129.stdout:1746 writesame 0x9b53eb8 thru 0x9b5c63a data_size 0x1(0x8783 bytes) 2024-09-17T21:30:11.660 INFO:teuthology.orchestra.run.smithi129.stdout:1749 read 0x86a93d6 thru 0x86b9086 (0xfcb1 bytes) 2024-09-17T21:30:11.660 INFO:teuthology.orchestra.run.smithi129.stdout:1750 writesame 0x4b8e9d9 thru 0x4b92447 data_size 0x1(0x3a6f bytes) 2024-09-17T21:30:11.668 INFO:teuthology.orchestra.run.smithi129.stdout:1752 read 0xb9b415 thru 0xba55ac (0xa198 bytes) 2024-09-17T21:30:11.668 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:11.668 INFO:teuthology.orchestra.run.smithi129.stdout:1754 read 0x758af0d thru 0x758d94c (0x2a40 bytes) 2024-09-17T21:30:11.673 INFO:teuthology.orchestra.run.smithi129.stdout:1755 trunc from 0x9b5c63b to 0x5db7bb1 2024-09-17T21:30:11.680 INFO:teuthology.orchestra.run.smithi129.stdout:1756 writesame 0x4f24b50 thru 0x4f33e16 data_size 0x1(0xf2c7 bytes) 2024-09-17T21:30:11.718 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:11.716+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:11.720 INFO:teuthology.orchestra.run.smithi129.stdout:1757 write 0xd2f9f57 thru 0xd2fd41b (0x34c5 bytes) 2024-09-17T21:30:11.729 INFO:teuthology.orchestra.run.smithi129.stdout:1759 trunc from 0xd2fd41c to 0x6cd94d0 2024-09-17T21:30:11.735 INFO:teuthology.orchestra.run.smithi129.stdout:1760 write 0x698ed36 thru 0x6993226 (0x44f1 bytes) 2024-09-17T21:30:11.737 INFO:teuthology.orchestra.run.smithi129.stdout:1761 punch from 0x61da921 to 0x61dc780, (0x1e5f bytes) 2024-09-17T21:30:11.737 INFO:teuthology.orchestra.run.smithi129.stdout:1762 writesame 0x4e83a94 thru 0x4e883b7 data_size 0x3e(0x4924 bytes) 2024-09-17T21:30:11.738 INFO:teuthology.orchestra.run.smithi129.stdout:1763 read 0x4ec38cb thru 0x4ec6789 (0x2ebf bytes) 2024-09-17T21:30:11.743 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:11.743 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:11.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:11.744 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:11.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.744 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:36935 3.a 3:54c8759b:::rbd_data.1120e62eea89.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-17T21:30:11.744 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:11.744 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.781905275, 2024-09-17T21:30:11.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020540151999999999, 2024-09-17T21:30:11.744 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.744 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.744 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:11.744 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.744 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.744 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.744 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 36935 2024-09-17T21:30:11.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.745 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:11.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:11.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958910+0000", 2024-09-17T21:30:11.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:30:11.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958918+0000", 2024-09-17T21:30:11.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6469999999999998e-06 2024-09-17T21:30:11.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958933+0000", 2024-09-17T21:30:11.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4791e-05 2024-09-17T21:30:11.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958940+0000", 2024-09-17T21:30:11.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-17T21:30:11.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958990+0000", 2024-09-17T21:30:11.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9759000000000003e-05 2024-09-17T21:30:11.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.959039+0000", 2024-09-17T21:30:11.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9487000000000002e-05 2024-09-17T21:30:11.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:11.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.962098+0000", 2024-09-17T21:30:11.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003059258 2024-09-17T21:30:11.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:11.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979325+0000", 2024-09-17T21:30:11.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017226841 2024-09-17T21:30:11.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:11.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979343+0000", 2024-09-17T21:30:11.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7486000000000002e-05 2024-09-17T21:30:11.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:11.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979401+0000", 2024-09-17T21:30:11.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7980999999999999e-05 2024-09-17T21:30:11.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979449+0000", 2024-09-17T21:30:11.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8639e-05 2024-09-17T21:30:11.749 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.749 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.749 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.749 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39447 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 909312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.749 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:11.749 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7729673750000003, 2024-09-17T21:30:11.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017419153999999999, 2024-09-17T21:30:11.750 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.750 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.750 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.750 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39447 2024-09-17T21:30:11.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.750 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:11.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:11.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:11.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:30:11.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967849+0000", 2024-09-17T21:30:11.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4339999999999999e-06 2024-09-17T21:30:11.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967853+0000", 2024-09-17T21:30:11.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:30:11.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967857+0000", 2024-09-17T21:30:11.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:30:11.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985193+0000", 2024-09-17T21:30:11.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017335973000000001 2024-09-17T21:30:11.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985228+0000", 2024-09-17T21:30:11.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5154999999999998e-05 2024-09-17T21:30:11.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985266+0000", 2024-09-17T21:30:11.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8448000000000002e-05 2024-09-17T21:30:11.753 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.753 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.753 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.754 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39448 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 974848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.754 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:11.754 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7729545590000004, 2024-09-17T21:30:11.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017517066000000001, 2024-09-17T21:30:11.754 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.754 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.754 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.754 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.754 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.754 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.754 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39448 2024-09-17T21:30:11.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.754 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:11.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:11.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:11.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.72e-07 2024-09-17T21:30:11.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967861+0000", 2024-09-17T21:30:11.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.096e-06 2024-09-17T21:30:11.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967865+0000", 2024-09-17T21:30:11.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9339999999999999e-06 2024-09-17T21:30:11.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967868+0000", 2024-09-17T21:30:11.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.508e-06 2024-09-17T21:30:11.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985285+0000", 2024-09-17T21:30:11.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017417208 2024-09-17T21:30:11.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985339+0000", 2024-09-17T21:30:11.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3931000000000003e-05 2024-09-17T21:30:11.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985377+0000", 2024-09-17T21:30:11.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8016999999999999e-05 2024-09-17T21:30:11.758 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.758 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.758 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.758 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39449 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.758 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:11.758 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7729424490000003, 2024-09-17T21:30:11.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017595137, 2024-09-17T21:30:11.758 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.758 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.758 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.758 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.758 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.759 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.759 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39449 2024-09-17T21:30:11.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.759 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:11.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:11.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:11.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2700000000000001e-07 2024-09-17T21:30:11.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967873+0000", 2024-09-17T21:30:11.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.164e-06 2024-09-17T21:30:11.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967877+0000", 2024-09-17T21:30:11.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.23e-06 2024-09-17T21:30:11.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967880+0000", 2024-09-17T21:30:11.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.286e-06 2024-09-17T21:30:11.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985397+0000", 2024-09-17T21:30:11.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017516982 2024-09-17T21:30:11.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985431+0000", 2024-09-17T21:30:11.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3899999999999997e-05 2024-09-17T21:30:11.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985467+0000", 2024-09-17T21:30:11.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6347999999999998e-05 2024-09-17T21:30:11.762 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.762 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.762 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.762 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39450 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1105920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.762 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:11.763 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7729221919999993, 2024-09-17T21:30:11.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017677824000000002, 2024-09-17T21:30:11.763 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.763 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.763 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.763 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39450 2024-09-17T21:30:11.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.763 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:11.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:11.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967893+0000", 2024-09-17T21:30:11.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9100000000000004e-07 2024-09-17T21:30:11.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967895+0000", 2024-09-17T21:30:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:30:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967901+0000", 2024-09-17T21:30:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.083e-06 2024-09-17T21:30:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967906+0000", 2024-09-17T21:30:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:30:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985493+0000", 2024-09-17T21:30:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017587565999999999 2024-09-17T21:30:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985531+0000", 2024-09-17T21:30:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7490000000000002e-05 2024-09-17T21:30:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985570+0000", 2024-09-17T21:30:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9205000000000002e-05 2024-09-17T21:30:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39451 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1171456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7729018120000006, 2024-09-17T21:30:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017752337999999999, 2024-09-17T21:30:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39451 2024-09-17T21:30:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:30:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967915+0000", 2024-09-17T21:30:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780000000000001e-06 2024-09-17T21:30:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967922+0000", 2024-09-17T21:30:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7020000000000004e-06 2024-09-17T21:30:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967926+0000", 2024-09-17T21:30:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.809e-06 2024-09-17T21:30:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985590+0000", 2024-09-17T21:30:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017663913999999999 2024-09-17T21:30:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985627+0000", 2024-09-17T21:30:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6896e-05 2024-09-17T21:30:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985665+0000", 2024-09-17T21:30:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7793999999999998e-05 2024-09-17T21:30:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39452 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1236992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7728818699999991, 2024-09-17T21:30:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017829602999999999, 2024-09-17T21:30:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39452 2024-09-17T21:30:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:30:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967947+0000", 2024-09-17T21:30:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4280999999999999e-05 2024-09-17T21:30:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967954+0000", 2024-09-17T21:30:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3249999999999996e-06 2024-09-17T21:30:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967959+0000", 2024-09-17T21:30:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2299999999999999e-06 2024-09-17T21:30:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985687+0000", 2024-09-17T21:30:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017727844999999999 2024-09-17T21:30:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985725+0000", 2024-09-17T21:30:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8243e-05 2024-09-17T21:30:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985762+0000", 2024-09-17T21:30:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7023000000000001e-05 2024-09-17T21:30:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39453 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7728453819999999, 2024-09-17T21:30:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017893793000000002, 2024-09-17T21:30:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39453 2024-09-17T21:30:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967970+0000", 2024-09-17T21:30:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:30:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967972+0000", 2024-09-17T21:30:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0229999999999999e-06 2024-09-17T21:30:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967978+0000", 2024-09-17T21:30:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0430000000000004e-06 2024-09-17T21:30:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967982+0000", 2024-09-17T21:30:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2570000000000001e-06 2024-09-17T21:30:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985782+0000", 2024-09-17T21:30:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017799683 2024-09-17T21:30:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985818+0000", 2024-09-17T21:30:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6304999999999999e-05 2024-09-17T21:30:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985863+0000", 2024-09-17T21:30:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4953999999999997e-05 2024-09-17T21:30:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39454 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1368064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7728260539999994, 2024-09-17T21:30:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017968886, 2024-09-17T21:30:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39454 2024-09-17T21:30:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967989+0000", 2024-09-17T21:30:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.15e-07 2024-09-17T21:30:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967991+0000", 2024-09-17T21:30:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.945e-06 2024-09-17T21:30:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967997+0000", 2024-09-17T21:30:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0190000000000003e-06 2024-09-17T21:30:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968001+0000", 2024-09-17T21:30:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6650000000000002e-06 2024-09-17T21:30:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985883+0000", 2024-09-17T21:30:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017881971999999999 2024-09-17T21:30:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985920+0000", 2024-09-17T21:30:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-05 2024-09-17T21:30:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985957+0000", 2024-09-17T21:30:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7370000000000003e-05 2024-09-17T21:30:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39455 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7728068799999992, 2024-09-17T21:30:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018044437999999999, 2024-09-17T21:30:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39455 2024-09-17T21:30:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:30:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968010+0000", 2024-09-17T21:30:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.886e-06 2024-09-17T21:30:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968017+0000", 2024-09-17T21:30:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:30:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968021+0000", 2024-09-17T21:30:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850000000000004e-06 2024-09-17T21:30:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985977+0000", 2024-09-17T21:30:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017956368 2024-09-17T21:30:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986015+0000", 2024-09-17T21:30:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7610000000000001e-05 2024-09-17T21:30:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986052+0000", 2024-09-17T21:30:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7109e-05 2024-09-17T21:30:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39456 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1499136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.77278658, 2024-09-17T21:30:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018114514000000002, 2024-09-17T21:30:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39456 2024-09-17T21:30:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-17T21:30:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968030+0000", 2024-09-17T21:30:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0049999999999999e-06 2024-09-17T21:30:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968036+0000", 2024-09-17T21:30:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0129999999999999e-06 2024-09-17T21:30:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968041+0000", 2024-09-17T21:30:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8300000000000003e-06 2024-09-17T21:30:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986071+0000", 2024-09-17T21:30:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018029620999999999 2024-09-17T21:30:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986106+0000", 2024-09-17T21:30:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4867999999999999e-05 2024-09-17T21:30:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986142+0000", 2024-09-17T21:30:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6566000000000002e-05 2024-09-17T21:30:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39457 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1564672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.772766764, 2024-09-17T21:30:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018188644, 2024-09-17T21:30:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39457 2024-09-17T21:30:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7199999999999999e-07 2024-09-17T21:30:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968050+0000", 2024-09-17T21:30:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1110000000000002e-06 2024-09-17T21:30:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968057+0000", 2024-09-17T21:30:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6170000000000003e-06 2024-09-17T21:30:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968062+0000", 2024-09-17T21:30:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:30:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986163+0000", 2024-09-17T21:30:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018101435999999999 2024-09-17T21:30:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986199+0000", 2024-09-17T21:30:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.591e-05 2024-09-17T21:30:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986236+0000", 2024-09-17T21:30:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7416999999999998e-05 2024-09-17T21:30:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39458 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1630208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7727398240000003, 2024-09-17T21:30:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018255856000000001, 2024-09-17T21:30:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39458 2024-09-17T21:30:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:30:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968077+0000", 2024-09-17T21:30:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.779e-06 2024-09-17T21:30:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968081+0000", 2024-09-17T21:30:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.532e-06 2024-09-17T21:30:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968085+0000", 2024-09-17T21:30:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999999e-06 2024-09-17T21:30:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986256+0000", 2024-09-17T21:30:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018171273000000002 2024-09-17T21:30:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986293+0000", 2024-09-17T21:30:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6534e-05 2024-09-17T21:30:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986330+0000", 2024-09-17T21:30:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7536000000000001e-05 2024-09-17T21:30:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39459 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1695744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7727252460000003, 2024-09-17T21:30:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334493, 2024-09-17T21:30:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39459 2024-09-17T21:30:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968090+0000", 2024-09-17T21:30:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:30:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968091+0000", 2024-09-17T21:30:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3319999999999999e-06 2024-09-17T21:30:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968094+0000", 2024-09-17T21:30:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2590000000000001e-06 2024-09-17T21:30:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968097+0000", 2024-09-17T21:30:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:30:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986350+0000", 2024-09-17T21:30:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018253364000000001 2024-09-17T21:30:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986387+0000", 2024-09-17T21:30:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6350999999999998e-05 2024-09-17T21:30:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986424+0000", 2024-09-17T21:30:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6897999999999998e-05 2024-09-17T21:30:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39460 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1761280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7727141100000008, 2024-09-17T21:30:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018415918999999999, 2024-09-17T21:30:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39460 2024-09-17T21:30:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968101+0000", 2024-09-17T21:30:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T21:30:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968102+0000", 2024-09-17T21:30:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1689999999999999e-06 2024-09-17T21:30:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968105+0000", 2024-09-17T21:30:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:30:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968108+0000", 2024-09-17T21:30:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6630000000000002e-06 2024-09-17T21:30:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986442+0000", 2024-09-17T21:30:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334487 2024-09-17T21:30:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986479+0000", 2024-09-17T21:30:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6557000000000003e-05 2024-09-17T21:30:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986516+0000", 2024-09-17T21:30:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7648999999999998e-05 2024-09-17T21:30:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39461 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1826816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7727033500000005, 2024-09-17T21:30:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018499768, 2024-09-17T21:30:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39461 2024-09-17T21:30:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4699999999999998e-07 2024-09-17T21:30:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968112+0000", 2024-09-17T21:30:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-06 2024-09-17T21:30:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968116+0000", 2024-09-17T21:30:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1080000000000001e-06 2024-09-17T21:30:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968122+0000", 2024-09-17T21:30:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1070000000000001e-06 2024-09-17T21:30:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986537+0000", 2024-09-17T21:30:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018414834000000001 2024-09-17T21:30:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986572+0000", 2024-09-17T21:30:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5726000000000003e-05 2024-09-17T21:30:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986611+0000", 2024-09-17T21:30:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8584000000000002e-05 2024-09-17T21:30:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39462 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1892352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7726889860000004, 2024-09-17T21:30:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018583252000000001, 2024-09-17T21:30:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39462 2024-09-17T21:30:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968126+0000", 2024-09-17T21:30:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968127+0000", 2024-09-17T21:30:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.229e-06 2024-09-17T21:30:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968131+0000", 2024-09-17T21:30:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8010000000000001e-06 2024-09-17T21:30:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968134+0000", 2024-09-17T21:30:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8880000000000001e-06 2024-09-17T21:30:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986634+0000", 2024-09-17T21:30:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018500501999999999 2024-09-17T21:30:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986671+0000", 2024-09-17T21:30:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6881000000000001e-05 2024-09-17T21:30:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986709+0000", 2024-09-17T21:30:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7546999999999998e-05 2024-09-17T21:30:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39463 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1957888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7726773710000003, 2024-09-17T21:30:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018667186999999998, 2024-09-17T21:30:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39463 2024-09-17T21:30:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:30:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968138+0000", 2024-09-17T21:30:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:30:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968142+0000", 2024-09-17T21:30:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.145e-06 2024-09-17T21:30:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968144+0000", 2024-09-17T21:30:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6869999999999999e-06 2024-09-17T21:30:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986728+0000", 2024-09-17T21:30:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018584163000000001 2024-09-17T21:30:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986766+0000", 2024-09-17T21:30:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7987000000000001e-05 2024-09-17T21:30:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986804+0000", 2024-09-17T21:30:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7892000000000002e-05 2024-09-17T21:30:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39464 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.772667062, 2024-09-17T21:30:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018753723999999999, 2024-09-17T21:30:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39464 2024-09-17T21:30:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968148+0000", 2024-09-17T21:30:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:30:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968152+0000", 2024-09-17T21:30:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3749999999999996e-06 2024-09-17T21:30:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968155+0000", 2024-09-17T21:30:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.371e-06 2024-09-17T21:30:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968158+0000", 2024-09-17T21:30:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T21:30:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986824+0000", 2024-09-17T21:30:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018665779 2024-09-17T21:30:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986863+0000", 2024-09-17T21:30:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9273000000000002e-05 2024-09-17T21:30:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986901+0000", 2024-09-17T21:30:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7978000000000001e-05 2024-09-17T21:30:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39465 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7726516629999995, 2024-09-17T21:30:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018825341999999998, 2024-09-17T21:30:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39465 2024-09-17T21:30:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-17T21:30:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968164+0000", 2024-09-17T21:30:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1289999999999999e-06 2024-09-17T21:30:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968167+0000", 2024-09-17T21:30:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1379999999999999e-06 2024-09-17T21:30:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968170+0000", 2024-09-17T21:30:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:30:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986914+0000", 2024-09-17T21:30:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018743829 2024-09-17T21:30:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986950+0000", 2024-09-17T21:30:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5819999999999999e-05 2024-09-17T21:30:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986988+0000", 2024-09-17T21:30:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8362000000000003e-05 2024-09-17T21:30:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:11.832 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:11.834 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:30:11.835 INFO:teuthology.orchestra.run.smithi129.stdout:1764 trunc from 0x6cd94d0 to 0xe7b8bfe 2024-09-17T21:30:11.835 INFO:teuthology.orchestra.run.smithi129.stdout:1765 write 0xd1e285b thru 0xd1ef3bf (0xcb65 bytes) 2024-09-17T21:30:11.835 INFO:teuthology.orchestra.run.smithi129.stdout:1766 read 0x2a13d7d thru 0x2a15397 (0x161b bytes) 2024-09-17T21:30:11.835 INFO:teuthology.orchestra.run.smithi129.stdout:1767 writesame 0x80e3824 thru 0x80e562d data_size 0x5(0x1e0a bytes) 2024-09-17T21:30:11.835 INFO:teuthology.orchestra.run.smithi129.stdout:1769 write 0x1b2bbd0 thru 0x1b34fc6 (0x93f7 bytes) 2024-09-17T21:30:11.835 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:11.835 INFO:teuthology.orchestra.run.smithi129.stdout:1771 write 0x87906e9 thru 0x8797838 (0x7150 bytes) 2024-09-17T21:30:11.836 INFO:teuthology.orchestra.run.smithi129.stdout:1772 trunc from 0xe7b8bfe to 0x4c7d38 2024-09-17T21:30:11.836 INFO:teuthology.orchestra.run.smithi129.stdout:1773 read 0x4547be thru 0x455373 (0xbb6 bytes) 2024-09-17T21:30:11.836 INFO:teuthology.orchestra.run.smithi129.stdout:1774 clone 15 order 21 su 524288 sc 14 2024-09-17T21:30:11.836 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:11.832+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:11.934 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:11.932+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:11.940 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:11.940 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:11.948 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:30:12.035 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:12.033+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:12.092 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:12.092 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:12.092 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:12.092 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:12.093 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:12.101 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:30:12.144 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:12.142+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:12.207 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:12.207 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:12.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:12.207 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:12.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.207 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:30:12.208 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:12.208 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 44.264494489, 2024-09-17T21:30:12.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61572000699999996, 2024-09-17T21:30:12.208 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.208 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.208 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:12.208 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.208 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:30:12.208 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:30:12.208 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T21:30:12.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.208 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:12.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:12.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939745+0000", 2024-09-17T21:30:12.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6899999999999999e-06 2024-09-17T21:30:12.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939753+0000", 2024-09-17T21:30:12.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1950000000000003e-06 2024-09-17T21:30:12.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939767+0000", 2024-09-17T21:30:12.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3429e-05 2024-09-17T21:30:12.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939776+0000", 2024-09-17T21:30:12.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:30:12.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939858+0000", 2024-09-17T21:30:12.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2496000000000004e-05 2024-09-17T21:30:12.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T21:30:12.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939868+0000", 2024-09-17T21:30:12.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1910000000000006e-06 2024-09-17T21:30:12.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940612+0000", 2024-09-17T21:30:12.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074429599999999998 2024-09-17T21:30:12.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:30:12.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940621+0000", 2024-09-17T21:30:12.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9139999999999997e-06 2024-09-17T21:30:12.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554006+0000", 2024-09-17T21:30:12.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61338505099999996 2024-09-17T21:30:12.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554166+0000", 2024-09-17T21:30:12.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015982600000000001 2024-09-17T21:30:12.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:30:12.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554296+0000", 2024-09-17T21:30:12.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013002399999999999 2024-09-17T21:30:12.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:12.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554936+0000", 2024-09-17T21:30:12.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064009400000000001 2024-09-17T21:30:12.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:12.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555389+0000", 2024-09-17T21:30:12.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045355200000000002 2024-09-17T21:30:12.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:12.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555432+0000", 2024-09-17T21:30:12.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2477e-05 2024-09-17T21:30:12.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555463+0000", 2024-09-17T21:30:12.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1612000000000002e-05 2024-09-17T21:30:12.215 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.215 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.215 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.215 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17527 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1005568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.215 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:12.215 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.799266658000001, 2024-09-17T21:30:12.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046452090000000001, 2024-09-17T21:30:12.216 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.216 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.216 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:12.216 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.216 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.216 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.216 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17527 2024-09-17T21:30:12.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.216 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:12.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:12.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405496+0000", 2024-09-17T21:30:12.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052498400000000004 2024-09-17T21:30:12.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405500+0000", 2024-09-17T21:30:12.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8410000000000002e-06 2024-09-17T21:30:12.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405511+0000", 2024-09-17T21:30:12.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0749e-05 2024-09-17T21:30:12.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405517+0000", 2024-09-17T21:30:12.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:30:12.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.219 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451275+0000", 2024-09-17T21:30:12.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045758134999999998 2024-09-17T21:30:12.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.219 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451355+0000", 2024-09-17T21:30:12.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9453e-05 2024-09-17T21:30:12.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451423+0000", 2024-09-17T21:30:12.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8782999999999995e-05 2024-09-17T21:30:12.220 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.220 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.220 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.220 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17530 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1046528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.220 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:12.220 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.798659007000001, 2024-09-17T21:30:12.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046376737000000001, 2024-09-17T21:30:12.220 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.220 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.221 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:12.221 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.221 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.221 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.221 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17530 2024-09-17T21:30:12.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.221 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:12.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:12.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:12.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3300000000000002e-07 2024-09-17T21:30:12.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405581+0000", 2024-09-17T21:30:12.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.074e-06 2024-09-17T21:30:12.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405587+0000", 2024-09-17T21:30:12.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7559999999999996e-06 2024-09-17T21:30:12.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405592+0000", 2024-09-17T21:30:12.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8860000000000003e-06 2024-09-17T21:30:12.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451813+0000", 2024-09-17T21:30:12.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046221119999999997 2024-09-17T21:30:12.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.224 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451891+0000", 2024-09-17T21:30:12.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7449999999999999e-05 2024-09-17T21:30:12.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.224 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451956+0000", 2024-09-17T21:30:12.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4918000000000004e-05 2024-09-17T21:30:12.224 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.224 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.225 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.225 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17615 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.225 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:12.225 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.794026169999999, 2024-09-17T21:30:12.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046741309000000002, 2024-09-17T21:30:12.225 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.225 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.225 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:12.225 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.225 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.225 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17615 2024-09-17T21:30:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:30:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410213+0000", 2024-09-17T21:30:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:30:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410217+0000", 2024-09-17T21:30:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8199999999999998e-06 2024-09-17T21:30:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410220+0000", 2024-09-17T21:30:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6900000000000001e-06 2024-09-17T21:30:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455264+0000", 2024-09-17T21:30:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045044002999999999 2024-09-17T21:30:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455338+0000", 2024-09-17T21:30:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4182000000000002e-05 2024-09-17T21:30:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.456953+0000", 2024-09-17T21:30:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016150089999999999 2024-09-17T21:30:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.230 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.230 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17616 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 872448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.230 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:12.230 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.794014118, 2024-09-17T21:30:12.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946039000000002, 2024-09-17T21:30:12.230 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.230 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.231 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:12.231 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.231 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.231 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.231 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17616 2024-09-17T21:30:12.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.231 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:12.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:12.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:12.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T21:30:12.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410225+0000", 2024-09-17T21:30:12.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3459999999999999e-06 2024-09-17T21:30:12.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410228+0000", 2024-09-17T21:30:12.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.083e-06 2024-09-17T21:30:12.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410232+0000", 2024-09-17T21:30:12.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4419999999999998e-06 2024-09-17T21:30:12.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457004+0000", 2024-09-17T21:30:12.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046771873999999998 2024-09-17T21:30:12.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457084+0000", 2024-09-17T21:30:12.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0506000000000005e-05 2024-09-17T21:30:12.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457170+0000", 2024-09-17T21:30:12.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5494000000000005e-05 2024-09-17T21:30:12.234 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.234 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.234 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.235 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17617 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.235 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:12.235 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.794002716000001, 2024-09-17T21:30:12.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047084168000000003, 2024-09-17T21:30:12.235 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.235 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.235 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:12.235 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.235 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.235 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.235 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17617 2024-09-17T21:30:12.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.236 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:12.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:12.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:12.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-17T21:30:12.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410237+0000", 2024-09-17T21:30:12.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1960000000000001e-06 2024-09-17T21:30:12.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410246+0000", 2024-09-17T21:30:12.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7880000000000001e-06 2024-09-17T21:30:12.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410250+0000", 2024-09-17T21:30:12.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.518e-06 2024-09-17T21:30:12.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457209+0000", 2024-09-17T21:30:12.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046959283999999997 2024-09-17T21:30:12.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457264+0000", 2024-09-17T21:30:12.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673999999999999e-05 2024-09-17T21:30:12.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457319+0000", 2024-09-17T21:30:12.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5497000000000001e-05 2024-09-17T21:30:12.239 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.239 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.239 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.239 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17618 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 899072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.239 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:12.240 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.793983271999998, 2024-09-17T21:30:12.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047204208999999997, 2024-09-17T21:30:12.240 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.240 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.240 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:12.240 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.240 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.240 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.240 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17618 2024-09-17T21:30:12.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.240 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:12.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:12.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:12.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8799999999999998e-07 2024-09-17T21:30:12.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410256+0000", 2024-09-17T21:30:12.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T21:30:12.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410260+0000", 2024-09-17T21:30:12.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6500000000000002e-06 2024-09-17T21:30:12.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410263+0000", 2024-09-17T21:30:12.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7089999999999998e-06 2024-09-17T21:30:12.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457355+0000", 2024-09-17T21:30:12.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047092756999999999 2024-09-17T21:30:12.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457407+0000", 2024-09-17T21:30:12.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1808999999999998e-05 2024-09-17T21:30:12.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457459+0000", 2024-09-17T21:30:12.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1684000000000002e-05 2024-09-17T21:30:12.244 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.244 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.244 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.244 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17619 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 913408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.244 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:12.244 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.793972033999999, 2024-09-17T21:30:12.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047293443999999997, 2024-09-17T21:30:12.245 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.245 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.245 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:12.245 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.245 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.245 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.245 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17619 2024-09-17T21:30:12.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.245 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:12.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:12.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:12.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:30:12.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410267+0000", 2024-09-17T21:30:12.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:30:12.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410270+0000", 2024-09-17T21:30:12.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.01e-06 2024-09-17T21:30:12.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410273+0000", 2024-09-17T21:30:12.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6410000000000002e-06 2024-09-17T21:30:12.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457473+0000", 2024-09-17T21:30:12.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047199829999999998 2024-09-17T21:30:12.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457517+0000", 2024-09-17T21:30:12.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4058999999999999e-05 2024-09-17T21:30:12.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457559+0000", 2024-09-17T21:30:12.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2404999999999998e-05 2024-09-17T21:30:12.249 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.249 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.249 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.249 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17620 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 927744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.249 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:12.249 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.793960861999999, 2024-09-17T21:30:12.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047402409, 2024-09-17T21:30:12.249 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.249 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.249 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:12.249 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.250 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.250 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.250 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17620 2024-09-17T21:30:12.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.250 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:12.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:12.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:12.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:30:12.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410278+0000", 2024-09-17T21:30:12.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1790000000000001e-06 2024-09-17T21:30:12.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410281+0000", 2024-09-17T21:30:12.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.061e-06 2024-09-17T21:30:12.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410284+0000", 2024-09-17T21:30:12.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.65e-06 2024-09-17T21:30:12.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457587+0000", 2024-09-17T21:30:12.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047303206 2024-09-17T21:30:12.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457633+0000", 2024-09-17T21:30:12.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6063999999999998e-05 2024-09-17T21:30:12.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457679+0000", 2024-09-17T21:30:12.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6007000000000002e-05 2024-09-17T21:30:12.253 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.253 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.253 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.254 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17621 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 940032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.254 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:12.254 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.793950446, 2024-09-17T21:30:12.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047486550000000002, 2024-09-17T21:30:12.254 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.254 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.254 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:12.254 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.254 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.254 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.254 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17621 2024-09-17T21:30:12.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.254 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:12.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:12.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410288+0000", 2024-09-17T21:30:12.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:30:12.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410289+0000", 2024-09-17T21:30:12.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1179999999999999e-06 2024-09-17T21:30:12.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410292+0000", 2024-09-17T21:30:12.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:30:12.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410294+0000", 2024-09-17T21:30:12.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T21:30:12.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457694+0000", 2024-09-17T21:30:12.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047399416999999999 2024-09-17T21:30:12.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457734+0000", 2024-09-17T21:30:12.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0528999999999998e-05 2024-09-17T21:30:12.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457774+0000", 2024-09-17T21:30:12.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9529e-05 2024-09-17T21:30:12.258 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.258 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.258 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.258 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17622 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.258 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:12.258 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.793940264, 2024-09-17T21:30:12.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047596619, 2024-09-17T21:30:12.259 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.259 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.259 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:12.259 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.259 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.259 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.259 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17622 2024-09-17T21:30:12.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.259 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:12.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:12.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:12.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:12.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410303+0000", 2024-09-17T21:30:12.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2979999999999999e-06 2024-09-17T21:30:12.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410307+0000", 2024-09-17T21:30:12.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:30:12.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410310+0000", 2024-09-17T21:30:12.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.146e-06 2024-09-17T21:30:12.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457801+0000", 2024-09-17T21:30:12.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047491322000000002 2024-09-17T21:30:12.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457850+0000", 2024-09-17T21:30:12.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8585000000000003e-05 2024-09-17T21:30:12.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457894+0000", 2024-09-17T21:30:12.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4455e-05 2024-09-17T21:30:12.263 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.263 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.263 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.263 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17623 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 968704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.263 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:12.263 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.793923880000001, 2024-09-17T21:30:12.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047689377999999998, 2024-09-17T21:30:12.263 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.264 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.264 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:12.264 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.264 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.264 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.264 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17623 2024-09-17T21:30:12.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.264 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:12.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:12.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:12.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:30:12.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410315+0000", 2024-09-17T21:30:12.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0499999999999999e-06 2024-09-17T21:30:12.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410319+0000", 2024-09-17T21:30:12.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.517e-06 2024-09-17T21:30:12.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410321+0000", 2024-09-17T21:30:12.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:30:12.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457922+0000", 2024-09-17T21:30:12.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047600638000000001 2024-09-17T21:30:12.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457964+0000", 2024-09-17T21:30:12.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2126000000000003e-05 2024-09-17T21:30:12.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458003+0000", 2024-09-17T21:30:12.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9138999999999999e-05 2024-09-17T21:30:12.268 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.268 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.268 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.268 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17624 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.268 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:12.268 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.793913295999999, 2024-09-17T21:30:12.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047798292999999999, 2024-09-17T21:30:12.268 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.268 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.269 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:12.269 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.269 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.269 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.269 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17624 2024-09-17T21:30:12.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.269 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:12.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:12.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:12.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:30:12.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410326+0000", 2024-09-17T21:30:12.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0610000000000001e-06 2024-09-17T21:30:12.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410329+0000", 2024-09-17T21:30:12.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0319999999999999e-06 2024-09-17T21:30:12.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410331+0000", 2024-09-17T21:30:12.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5749999999999999e-06 2024-09-17T21:30:12.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458033+0000", 2024-09-17T21:30:12.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047701371999999999 2024-09-17T21:30:12.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458079+0000", 2024-09-17T21:30:12.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:30:12.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458123+0000", 2024-09-17T21:30:12.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850999999999997e-05 2024-09-17T21:30:12.272 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.272 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.273 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.273 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17625 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.273 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:12.273 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.793903136000001, 2024-09-17T21:30:12.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047883051000000003, 2024-09-17T21:30:12.273 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.273 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.273 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:12.273 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.273 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.273 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.274 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17625 2024-09-17T21:30:12.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.274 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:12.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:12.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:12.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:12.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410336+0000", 2024-09-17T21:30:12.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1310000000000001e-06 2024-09-17T21:30:12.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410339+0000", 2024-09-17T21:30:12.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000001e-06 2024-09-17T21:30:12.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410342+0000", 2024-09-17T21:30:12.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7460000000000001e-06 2024-09-17T21:30:12.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458137+0000", 2024-09-17T21:30:12.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047794931999999998 2024-09-17T21:30:12.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458178+0000", 2024-09-17T21:30:12.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1013999999999997e-05 2024-09-17T21:30:12.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458218+0000", 2024-09-17T21:30:12.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9675999999999998e-05 2024-09-17T21:30:12.277 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.277 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.277 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.278 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17626 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1009664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.278 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:12.278 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.793892515, 2024-09-17T21:30:12.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047991601000000002, 2024-09-17T21:30:12.278 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.278 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.278 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:12.278 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.278 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.278 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.278 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17626 2024-09-17T21:30:12.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.279 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:12.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:12.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410346+0000", 2024-09-17T21:30:12.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:30:12.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410347+0000", 2024-09-17T21:30:12.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.114e-06 2024-09-17T21:30:12.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410350+0000", 2024-09-17T21:30:12.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.101e-06 2024-09-17T21:30:12.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410353+0000", 2024-09-17T21:30:12.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T21:30:12.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458245+0000", 2024-09-17T21:30:12.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0478921 2024-09-17T21:30:12.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458292+0000", 2024-09-17T21:30:12.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397999999999999e-05 2024-09-17T21:30:12.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458337+0000", 2024-09-17T21:30:12.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5221e-05 2024-09-17T21:30:12.282 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.282 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.282 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.282 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17627 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.282 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:12.282 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.793855413999999, 2024-09-17T21:30:12.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048047383999999999, 2024-09-17T21:30:12.283 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.283 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.283 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:12.283 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.283 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.283 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.283 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17627 2024-09-17T21:30:12.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.283 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:12.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:12.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410383+0000", 2024-09-17T21:30:12.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:30:12.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410385+0000", 2024-09-17T21:30:12.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:30:12.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410391+0000", 2024-09-17T21:30:12.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:30:12.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410395+0000", 2024-09-17T21:30:12.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6539999999999998e-06 2024-09-17T21:30:12.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458350+0000", 2024-09-17T21:30:12.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047954855999999997 2024-09-17T21:30:12.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458392+0000", 2024-09-17T21:30:12.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2094000000000001e-05 2024-09-17T21:30:12.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458430+0000", 2024-09-17T21:30:12.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7474000000000001e-05 2024-09-17T21:30:12.287 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.287 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.287 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.287 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17628 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.287 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:12.287 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.793835853000001, 2024-09-17T21:30:12.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048142676000000002, 2024-09-17T21:30:12.287 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.287 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.287 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:12.287 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.288 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.288 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.288 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17628 2024-09-17T21:30:12.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.288 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:12.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:12.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:12.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:30:12.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410404+0000", 2024-09-17T21:30:12.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-06 2024-09-17T21:30:12.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410411+0000", 2024-09-17T21:30:12.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6850000000000004e-06 2024-09-17T21:30:12.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410415+0000", 2024-09-17T21:30:12.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6750000000000001e-06 2024-09-17T21:30:12.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458457+0000", 2024-09-17T21:30:12.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048042045999999998 2024-09-17T21:30:12.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458501+0000", 2024-09-17T21:30:12.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3661000000000001e-05 2024-09-17T21:30:12.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458545+0000", 2024-09-17T21:30:12.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3529000000000003e-05 2024-09-17T21:30:12.291 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.291 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.291 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.292 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17687 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.292 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:12.292 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.7913909, 2024-09-17T21:30:12.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050670448, 2024-09-17T21:30:12.292 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.292 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.292 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:12.292 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.292 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.292 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.292 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17687 2024-09-17T21:30:12.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.292 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:12.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:12.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412848+0000", 2024-09-17T21:30:12.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9100000000000003e-07 2024-09-17T21:30:12.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413102+0000", 2024-09-17T21:30:12.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025415600000000001 2024-09-17T21:30:12.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413111+0000", 2024-09-17T21:30:12.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2010000000000005e-06 2024-09-17T21:30:12.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413116+0000", 2024-09-17T21:30:12.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:30:12.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458559+0000", 2024-09-17T21:30:12.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045442914000000001 2024-09-17T21:30:12.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458632+0000", 2024-09-17T21:30:12.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3207000000000006e-05 2024-09-17T21:30:12.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:12.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458785+0000", 2024-09-17T21:30:12.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015226700000000001 2024-09-17T21:30:12.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:12.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.461648+0000", 2024-09-17T21:30:12.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028633069999999998 2024-09-17T21:30:12.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:12.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463416+0000", 2024-09-17T21:30:12.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017680230000000001 2024-09-17T21:30:12.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:12.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463475+0000", 2024-09-17T21:30:12.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9333000000000003e-05 2024-09-17T21:30:12.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463517+0000", 2024-09-17T21:30:12.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2162000000000001e-05 2024-09-17T21:30:12.297 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.297 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.298 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.298 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17688 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [write 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.298 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:12.298 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.791112032000001, 2024-09-17T21:30:12.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050625098, 2024-09-17T21:30:12.298 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.298 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.298 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:12.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.298 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17688 2024-09-17T21:30:12.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.299 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:12.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:12.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:12.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:30:12.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413128+0000", 2024-09-17T21:30:12.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.813e-06 2024-09-17T21:30:12.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413132+0000", 2024-09-17T21:30:12.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:30:12.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413135+0000", 2024-09-17T21:30:12.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:30:12.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459756+0000", 2024-09-17T21:30:12.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046621293000000001 2024-09-17T21:30:12.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459816+0000", 2024-09-17T21:30:12.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9156000000000001e-05 2024-09-17T21:30:12.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:12.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459912+0000", 2024-09-17T21:30:12.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6302000000000001e-05 2024-09-17T21:30:12.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:12.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463231+0000", 2024-09-17T21:30:12.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033188290000000001 2024-09-17T21:30:12.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:12.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463701+0000", 2024-09-17T21:30:12.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047068199999999999 2024-09-17T21:30:12.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:12.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463728+0000", 2024-09-17T21:30:12.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6726000000000001e-05 2024-09-17T21:30:12.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463751+0000", 2024-09-17T21:30:12.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2824e-05 2024-09-17T21:30:12.303 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.304 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.304 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.304 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17691 3.d 3:b3a055f3:::rbd_data.11207744cc2c.0000000000000083:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:12.304 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:12.304 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.787567128999999, 2024-09-17T21:30:12.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049309370999999998, 2024-09-17T21:30:12.304 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:12.304 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:12.304 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:12.304 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:12.304 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:12.305 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.305 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17691 2024-09-17T21:30:12.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.305 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:12.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:12.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:12.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:12.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:12.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:12.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:12.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416672+0000", 2024-09-17T21:30:12.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.47e-07 2024-09-17T21:30:12.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:12.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416915+0000", 2024-09-17T21:30:12.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024344500000000001 2024-09-17T21:30:12.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:12.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416924+0000", 2024-09-17T21:30:12.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7739999999999993e-06 2024-09-17T21:30:12.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416929+0000", 2024-09-17T21:30:12.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6990000000000002e-06 2024-09-17T21:30:12.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:12.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460414+0000", 2024-09-17T21:30:12.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043485466 2024-09-17T21:30:12.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:12.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460540+0000", 2024-09-17T21:30:12.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012608899999999999 2024-09-17T21:30:12.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:12.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460746+0000", 2024-09-17T21:30:12.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000205577 2024-09-17T21:30:12.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:12.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463800+0000", 2024-09-17T21:30:12.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030542030000000001 2024-09-17T21:30:12.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:12.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465867+0000", 2024-09-17T21:30:12.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020665929999999998 2024-09-17T21:30:12.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:12.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465914+0000", 2024-09-17T21:30:12.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497000000000003e-05 2024-09-17T21:30:12.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:12.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:12.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:12.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465980+0000", 2024-09-17T21:30:12.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6081000000000006e-05 2024-09-17T21:30:12.310 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.310 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.310 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.310 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:12.310 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:12.310 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:12.311 INFO:teuthology.orchestra.run.smithi129.stdout: marking collection for removal 2024-09-17T21:30:12.311 INFO:teuthology.orchestra.run.smithi129.stdout:setting '_remove' omap key 2024-09-17T21:30:12.311 INFO:teuthology.orchestra.run.smithi129.stdout:finish_remove_pgs 3.1a_head removing 3.1a 2024-09-17T21:30:12.312 INFO:teuthology.orchestra.run.smithi129.stdout:Remove successful 2024-09-17T21:30:12.312 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:12.309+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:12.312 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:30:12.412 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:12.410+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:12.417 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:12.417 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:12.417 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:12.417 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:12.425 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:30:12.513 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:12.511+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:12.568 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:12.569 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:12.569 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:12.569 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:12.569 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:12.577 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:30:12.623 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:12.620+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:12.635 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:30:12.692 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:12.692 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:12.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:12.692 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:12.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.692 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:37817 3.17 3:eab85a8d:::rbd_header.1120e62eea89:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:30:12.692 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:12.692 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.993407051, 2024-09-17T21:30:12.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86635970399999995, 2024-09-17T21:30:12.693 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.693 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.693 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:12.693 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.693 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.693 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.693 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37817 2024-09-17T21:30:12.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.693 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:12.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:12.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695457+0000", 2024-09-17T21:30:12.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5799999999999998e-07 2024-09-17T21:30:12.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695461+0000", 2024-09-17T21:30:12.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1110000000000001e-06 2024-09-17T21:30:12.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695471+0000", 2024-09-17T21:30:12.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0441e-05 2024-09-17T21:30:12.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695477+0000", 2024-09-17T21:30:12.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7599999999999999e-06 2024-09-17T21:30:12.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695533+0000", 2024-09-17T21:30:12.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6570999999999998e-05 2024-09-17T21:30:12.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:30:12.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695543+0000", 2024-09-17T21:30:12.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9979999999999998e-06 2024-09-17T21:30:12.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561472+0000", 2024-09-17T21:30:12.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86592841899999995 2024-09-17T21:30:12.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561610+0000", 2024-09-17T21:30:12.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013854699999999999 2024-09-17T21:30:12.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561816+0000", 2024-09-17T21:30:12.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020509900000000001 2024-09-17T21:30:12.698 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.698 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.698 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.698 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50202 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.698 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:12.698 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.5812362609999999, 2024-09-17T21:30:12.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.076620261999999995, 2024-09-17T21:30:12.698 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.698 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.698 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:12.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.699 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50202 2024-09-17T21:30:12.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.699 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:12.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:12.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107629+0000", 2024-09-17T21:30:12.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3709999999999998e-06 2024-09-17T21:30:12.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119765+0000", 2024-09-17T21:30:12.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012135683 2024-09-17T21:30:12.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119787+0000", 2024-09-17T21:30:12.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2748999999999999e-05 2024-09-17T21:30:12.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119797+0000", 2024-09-17T21:30:12.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8169999999999994e-06 2024-09-17T21:30:12.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119872+0000", 2024-09-17T21:30:12.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270999999999994e-05 2024-09-17T21:30:12.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119987+0000", 2024-09-17T21:30:12.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011557699999999999 2024-09-17T21:30:12.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:12.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.121766+0000", 2024-09-17T21:30:12.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00177853 2024-09-17T21:30:12.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:12.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178307+0000", 2024-09-17T21:30:12.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056541559999999998 2024-09-17T21:30:12.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:12.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184097+0000", 2024-09-17T21:30:12.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057899029999999999 2024-09-17T21:30:12.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:12.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184172+0000", 2024-09-17T21:30:12.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5226000000000001e-05 2024-09-17T21:30:12.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184247+0000", 2024-09-17T21:30:12.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4574999999999997e-05 2024-09-17T21:30:12.704 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.704 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.704 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.704 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50203 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [truncate 20971520] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.705 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:12.705 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.569036356, 2024-09-17T21:30:12.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065009779000000004, 2024-09-17T21:30:12.705 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.705 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.705 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:12.705 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.705 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.705 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.705 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50203 2024-09-17T21:30:12.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.705 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:12.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:12.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119828+0000", 2024-09-17T21:30:12.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6390000000000001e-06 2024-09-17T21:30:12.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119832+0000", 2024-09-17T21:30:12.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7699999999999999e-06 2024-09-17T21:30:12.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119845+0000", 2024-09-17T21:30:12.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3093e-05 2024-09-17T21:30:12.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119852+0000", 2024-09-17T21:30:12.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6950000000000003e-06 2024-09-17T21:30:12.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.160945+0000", 2024-09-17T21:30:12.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041093076999999999 2024-09-17T21:30:12.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161071+0000", 2024-09-17T21:30:12.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012569799999999999 2024-09-17T21:30:12.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:12.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161283+0000", 2024-09-17T21:30:12.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021201999999999999 2024-09-17T21:30:12.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:12.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.179360+0000", 2024-09-17T21:30:12.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018077784999999999 2024-09-17T21:30:12.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:12.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184726+0000", 2024-09-17T21:30:12.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053652960000000003 2024-09-17T21:30:12.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:12.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184787+0000", 2024-09-17T21:30:12.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1402000000000006e-05 2024-09-17T21:30:12.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184836+0000", 2024-09-17T21:30:12.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9304000000000001e-05 2024-09-17T21:30:12.711 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.711 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.711 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.711 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50206 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.711 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:12.711 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.5319548160000001, 2024-09-17T21:30:12.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.090361968000000001, 2024-09-17T21:30:12.711 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.712 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.712 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:12.712 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.712 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.712 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.712 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50206 2024-09-17T21:30:12.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.712 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:12.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:12.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156911+0000", 2024-09-17T21:30:12.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-17T21:30:12.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173670+0000", 2024-09-17T21:30:12.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016758907 2024-09-17T21:30:12.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173696+0000", 2024-09-17T21:30:12.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6329000000000002e-05 2024-09-17T21:30:12.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173708+0000", 2024-09-17T21:30:12.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2512e-05 2024-09-17T21:30:12.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173788+0000", 2024-09-17T21:30:12.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9332999999999994e-05 2024-09-17T21:30:12.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173918+0000", 2024-09-17T21:30:12.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013063300000000001 2024-09-17T21:30:12.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:12.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.176228+0000", 2024-09-17T21:30:12.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002309757 2024-09-17T21:30:12.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:12.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.245164+0000", 2024-09-17T21:30:12.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068936301000000005 2024-09-17T21:30:12.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:12.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247176+0000", 2024-09-17T21:30:12.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020117590000000001 2024-09-17T21:30:12.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:12.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247215+0000", 2024-09-17T21:30:12.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9094000000000002e-05 2024-09-17T21:30:12.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247270+0000", 2024-09-17T21:30:12.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4793999999999997e-05 2024-09-17T21:30:12.718 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.718 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.718 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.718 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50207 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [truncate 21556327] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.718 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:12.718 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.515105836, 2024-09-17T21:30:12.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.074055363999999999, 2024-09-17T21:30:12.718 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.719 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.719 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:12.719 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.719 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.719 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.719 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50207 2024-09-17T21:30:12.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.719 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:12.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:12.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173759+0000", 2024-09-17T21:30:12.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8160000000000001e-06 2024-09-17T21:30:12.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173763+0000", 2024-09-17T21:30:12.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5360000000000003e-06 2024-09-17T21:30:12.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173771+0000", 2024-09-17T21:30:12.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8849999999999999e-06 2024-09-17T21:30:12.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173777+0000", 2024-09-17T21:30:12.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7799999999999997e-06 2024-09-17T21:30:12.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221363+0000", 2024-09-17T21:30:12.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047585493 2024-09-17T21:30:12.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221430+0000", 2024-09-17T21:30:12.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7129000000000001e-05 2024-09-17T21:30:12.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:12.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221550+0000", 2024-09-17T21:30:12.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120658 2024-09-17T21:30:12.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:12.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246113+0000", 2024-09-17T21:30:12.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02456262 2024-09-17T21:30:12.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:12.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247744+0000", 2024-09-17T21:30:12.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001630692 2024-09-17T21:30:12.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:12.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247777+0000", 2024-09-17T21:30:12.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3015000000000002e-05 2024-09-17T21:30:12.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247812+0000", 2024-09-17T21:30:12.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.574e-05 2024-09-17T21:30:12.725 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.725 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.725 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.725 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50828 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.725 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:12.725 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.206993904, 2024-09-17T21:30:12.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047324185999999997, 2024-09-17T21:30:12.725 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.726 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.726 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:12.726 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.726 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.726 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.726 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50828 2024-09-17T21:30:12.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.726 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:12.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:12.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:12.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.46e-07 2024-09-17T21:30:12.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481871+0000", 2024-09-17T21:30:12.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T21:30:12.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481878+0000", 2024-09-17T21:30:12.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:30:12.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481882+0000", 2024-09-17T21:30:12.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:30:12.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529038+0000", 2024-09-17T21:30:12.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047156034999999999 2024-09-17T21:30:12.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529099+0000", 2024-09-17T21:30:12.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0316000000000003e-05 2024-09-17T21:30:12.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529193+0000", 2024-09-17T21:30:12.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4369000000000004e-05 2024-09-17T21:30:12.730 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.730 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.730 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.731 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50829 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 12845056~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.731 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:12.731 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.2069751099999999, 2024-09-17T21:30:12.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047887596999999997, 2024-09-17T21:30:12.731 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.731 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.731 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:12.731 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.731 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.731 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.731 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50829 2024-09-17T21:30:12.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.732 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:12.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:12.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:12.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:30:12.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481890+0000", 2024-09-17T21:30:12.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.877e-06 2024-09-17T21:30:12.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481895+0000", 2024-09-17T21:30:12.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:30:12.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481900+0000", 2024-09-17T21:30:12.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4440000000000001e-06 2024-09-17T21:30:12.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529225+0000", 2024-09-17T21:30:12.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047325117999999999 2024-09-17T21:30:12.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529292+0000", 2024-09-17T21:30:12.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7100000000000005e-05 2024-09-17T21:30:12.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529775+0000", 2024-09-17T21:30:12.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048322799999999999 2024-09-17T21:30:12.736 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.736 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.736 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.736 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50830 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.736 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:12.736 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.2069567079999999, 2024-09-17T21:30:12.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048062123999999998, 2024-09-17T21:30:12.736 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.736 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.736 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:12.737 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.737 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.737 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.737 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50830 2024-09-17T21:30:12.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.737 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:12.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:12.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481907+0000", 2024-09-17T21:30:12.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:30:12.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481909+0000", 2024-09-17T21:30:12.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T21:30:12.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481915+0000", 2024-09-17T21:30:12.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.02e-06 2024-09-17T21:30:12.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481920+0000", 2024-09-17T21:30:12.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6799999999999998e-06 2024-09-17T21:30:12.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529793+0000", 2024-09-17T21:30:12.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047873042999999997 2024-09-17T21:30:12.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529870+0000", 2024-09-17T21:30:12.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6967999999999999e-05 2024-09-17T21:30:12.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529968+0000", 2024-09-17T21:30:12.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8085999999999996e-05 2024-09-17T21:30:12.741 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.741 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.741 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.741 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50831 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 18087936~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.741 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:12.741 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.206936362, 2024-09-17T21:30:12.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048663304999999997, 2024-09-17T21:30:12.741 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.741 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.741 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:12.742 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.742 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.742 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.742 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50831 2024-09-17T21:30:12.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.742 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:12.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:12.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:12.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:30:12.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481929+0000", 2024-09-17T21:30:12.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2189999999999998e-06 2024-09-17T21:30:12.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481935+0000", 2024-09-17T21:30:12.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.383e-06 2024-09-17T21:30:12.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481939+0000", 2024-09-17T21:30:12.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2479999999999998e-06 2024-09-17T21:30:12.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530095+0000", 2024-09-17T21:30:12.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048156617999999998 2024-09-17T21:30:12.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530159+0000", 2024-09-17T21:30:12.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3394999999999999e-05 2024-09-17T21:30:12.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530590+0000", 2024-09-17T21:30:12.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043101199999999999 2024-09-17T21:30:12.745 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.746 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.746 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.746 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50832 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 19660800~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.746 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:12.746 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.2069177149999999, 2024-09-17T21:30:12.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049339466999999998, 2024-09-17T21:30:12.746 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.747 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.747 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:12.747 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.747 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.747 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.747 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50832 2024-09-17T21:30:12.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.747 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:12.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:12.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481946+0000", 2024-09-17T21:30:12.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:12.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481951+0000", 2024-09-17T21:30:12.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1479999999999997e-06 2024-09-17T21:30:12.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481956+0000", 2024-09-17T21:30:12.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7339999999999996e-06 2024-09-17T21:30:12.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481961+0000", 2024-09-17T21:30:12.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4889999999999997e-06 2024-09-17T21:30:12.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530630+0000", 2024-09-17T21:30:12.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048668596000000001 2024-09-17T21:30:12.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530720+0000", 2024-09-17T21:30:12.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0163999999999999e-05 2024-09-17T21:30:12.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531285+0000", 2024-09-17T21:30:12.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056493199999999998 2024-09-17T21:30:12.751 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.751 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.751 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.751 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50833 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.751 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:12.752 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.206895571, 2024-09-17T21:30:12.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049908069999999999, 2024-09-17T21:30:12.752 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.752 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.752 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:12.752 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.752 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.752 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.752 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50833 2024-09-17T21:30:12.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.752 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:12.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:12.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481968+0000", 2024-09-17T21:30:12.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5600000000000001e-07 2024-09-17T21:30:12.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481970+0000", 2024-09-17T21:30:12.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:30:12.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481975+0000", 2024-09-17T21:30:12.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8499999999999999e-06 2024-09-17T21:30:12.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481980+0000", 2024-09-17T21:30:12.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-17T21:30:12.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531337+0000", 2024-09-17T21:30:12.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049356874000000002 2024-09-17T21:30:12.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531392+0000", 2024-09-17T21:30:12.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4826e-05 2024-09-17T21:30:12.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531875+0000", 2024-09-17T21:30:12.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048345099999999998 2024-09-17T21:30:12.757 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.757 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.757 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.758 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50834 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 24903680~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.758 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:12.758 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.2068708029999999, 2024-09-17T21:30:12.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050534015000000002, 2024-09-17T21:30:12.758 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.758 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.758 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:12.758 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.758 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.758 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.759 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50834 2024-09-17T21:30:12.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.759 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:12.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:12.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481993+0000", 2024-09-17T21:30:12.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:30:12.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481995+0000", 2024-09-17T21:30:12.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.199e-06 2024-09-17T21:30:12.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482001+0000", 2024-09-17T21:30:12.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.772e-06 2024-09-17T21:30:12.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482006+0000", 2024-09-17T21:30:12.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0660000000000003e-06 2024-09-17T21:30:12.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531923+0000", 2024-09-17T21:30:12.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049917200000000002 2024-09-17T21:30:12.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532004+0000", 2024-09-17T21:30:12.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0758000000000002e-05 2024-09-17T21:30:12.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532526+0000", 2024-09-17T21:30:12.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00052258899999999995 2024-09-17T21:30:12.762 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.762 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.762 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.763 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50835 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 28573696~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.763 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:12.763 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.2068508200000001, 2024-09-17T21:30:12.763 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051204701999999998, 2024-09-17T21:30:12.763 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.763 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.763 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:12.763 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.763 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.763 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.764 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50835 2024-09-17T21:30:12.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.764 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:12.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:12.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482013+0000", 2024-09-17T21:30:12.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:30:12.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482015+0000", 2024-09-17T21:30:12.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.272e-06 2024-09-17T21:30:12.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482020+0000", 2024-09-17T21:30:12.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3040000000000004e-06 2024-09-17T21:30:12.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482025+0000", 2024-09-17T21:30:12.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5650000000000003e-06 2024-09-17T21:30:12.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532569+0000", 2024-09-17T21:30:12.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050544661999999997 2024-09-17T21:30:12.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532649+0000", 2024-09-17T21:30:12.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9640000000000003e-05 2024-09-17T21:30:12.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533217+0000", 2024-09-17T21:30:12.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056784100000000003 2024-09-17T21:30:12.767 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.767 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.768 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.768 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50836 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 30146560~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.768 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:12.768 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.2068312029999999, 2024-09-17T21:30:12.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051875589, 2024-09-17T21:30:12.769 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.769 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.769 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:12.769 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.769 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.769 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.769 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50836 2024-09-17T21:30:12.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.769 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:12.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:12.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:12.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3700000000000001e-07 2024-09-17T21:30:12.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482034+0000", 2024-09-17T21:30:12.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0059999999999999e-06 2024-09-17T21:30:12.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482040+0000", 2024-09-17T21:30:12.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:30:12.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482044+0000", 2024-09-17T21:30:12.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:30:12.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533260+0000", 2024-09-17T21:30:12.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051215416 2024-09-17T21:30:12.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533348+0000", 2024-09-17T21:30:12.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8695000000000004e-05 2024-09-17T21:30:12.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533907+0000", 2024-09-17T21:30:12.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055902199999999995 2024-09-17T21:30:12.773 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.774 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.774 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.774 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50844 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.774 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:12.774 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.2066677770000001, 2024-09-17T21:30:12.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0475644, 2024-09-17T21:30:12.774 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.774 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.774 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:12.775 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.775 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.775 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.775 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50844 2024-09-17T21:30:12.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.775 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:12.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:12.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482196+0000", 2024-09-17T21:30:12.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.39e-07 2024-09-17T21:30:12.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482198+0000", 2024-09-17T21:30:12.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0099999999999998e-06 2024-09-17T21:30:12.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482204+0000", 2024-09-17T21:30:12.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.93e-06 2024-09-17T21:30:12.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482208+0000", 2024-09-17T21:30:12.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7439999999999998e-06 2024-09-17T21:30:12.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529318+0000", 2024-09-17T21:30:12.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047110051999999999 2024-09-17T21:30:12.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529396+0000", 2024-09-17T21:30:12.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7712000000000004e-05 2024-09-17T21:30:12.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529760+0000", 2024-09-17T21:30:12.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00036351299999999999 2024-09-17T21:30:12.779 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.779 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.779 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.779 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50845 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 21757952~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.779 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:12.779 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.206648674, 2024-09-17T21:30:12.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048084625999999998, 2024-09-17T21:30:12.779 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.779 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.779 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:12.780 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.780 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.780 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.780 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50845 2024-09-17T21:30:12.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.780 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:12.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:12.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482215+0000", 2024-09-17T21:30:12.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:30:12.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482216+0000", 2024-09-17T21:30:12.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.823e-06 2024-09-17T21:30:12.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482222+0000", 2024-09-17T21:30:12.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.507e-06 2024-09-17T21:30:12.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482227+0000", 2024-09-17T21:30:12.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6609999999999999e-06 2024-09-17T21:30:12.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529792+0000", 2024-09-17T21:30:12.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047565075999999998 2024-09-17T21:30:12.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529871+0000", 2024-09-17T21:30:12.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8855000000000003e-05 2024-09-17T21:30:12.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530299+0000", 2024-09-17T21:30:12.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000428307 2024-09-17T21:30:12.784 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.784 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.784 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.784 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50846 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 25427968~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.784 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:12.784 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.206626287, 2024-09-17T21:30:12.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048735142000000002, 2024-09-17T21:30:12.784 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.784 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.785 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:12.785 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.785 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.785 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.785 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50846 2024-09-17T21:30:12.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.786 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:12.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:12.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:12.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:30:12.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482239+0000", 2024-09-17T21:30:12.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.871e-06 2024-09-17T21:30:12.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482244+0000", 2024-09-17T21:30:12.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.49e-06 2024-09-17T21:30:12.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482249+0000", 2024-09-17T21:30:12.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6269999999999999e-06 2024-09-17T21:30:12.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530347+0000", 2024-09-17T21:30:12.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048097721000000003 2024-09-17T21:30:12.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530437+0000", 2024-09-17T21:30:12.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0604000000000002e-05 2024-09-17T21:30:12.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530972+0000", 2024-09-17T21:30:12.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053445000000000001 2024-09-17T21:30:12.794 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.794 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.794 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.794 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50847 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 27000832~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.795 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:12.795 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.2066083910000001, 2024-09-17T21:30:12.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049308668, 2024-09-17T21:30:12.795 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.795 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.796 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:12.796 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.796 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.796 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.796 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50847 2024-09-17T21:30:12.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.797 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:12.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:12.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:12.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:30:12.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482257+0000", 2024-09-17T21:30:12.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.846e-06 2024-09-17T21:30:12.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482264+0000", 2024-09-17T21:30:12.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8170000000000001e-06 2024-09-17T21:30:12.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482283+0000", 2024-09-17T21:30:12.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9718e-05 2024-09-17T21:30:12.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531018+0000", 2024-09-17T21:30:12.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048734497000000002 2024-09-17T21:30:12.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531080+0000", 2024-09-17T21:30:12.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1795e-05 2024-09-17T21:30:12.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531563+0000", 2024-09-17T21:30:12.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048366499999999998 2024-09-17T21:30:12.805 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.805 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.805 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.806 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50848 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 30670848~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.806 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:12.806 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.2065718489999999, 2024-09-17T21:30:12.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049893337000000003, 2024-09-17T21:30:12.807 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.807 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.807 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:12.807 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.807 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.808 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.808 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50848 2024-09-17T21:30:12.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.808 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:12.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:12.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482292+0000", 2024-09-17T21:30:12.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T21:30:12.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482294+0000", 2024-09-17T21:30:12.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3209999999999999e-06 2024-09-17T21:30:12.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482300+0000", 2024-09-17T21:30:12.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7189999999999998e-06 2024-09-17T21:30:12.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482305+0000", 2024-09-17T21:30:12.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T21:30:12.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531606+0000", 2024-09-17T21:30:12.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049301273999999999 2024-09-17T21:30:12.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531696+0000", 2024-09-17T21:30:12.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0161999999999995e-05 2024-09-17T21:30:12.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532184+0000", 2024-09-17T21:30:12.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048806400000000002 2024-09-17T21:30:12.817 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.817 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.818 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.818 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50849 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 32243712~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:12.818 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:12.819 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.2065524080000001, 2024-09-17T21:30:12.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050444607000000002, 2024-09-17T21:30:12.819 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:12.819 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:12.819 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:12.819 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:12.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:12.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:12.820 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50849 2024-09-17T21:30:12.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.820 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:12.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:12.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:12.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:12.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:12.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:12.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:12.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482311+0000", 2024-09-17T21:30:12.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:30:12.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:12.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482316+0000", 2024-09-17T21:30:12.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1209999999999998e-06 2024-09-17T21:30:12.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:12.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482321+0000", 2024-09-17T21:30:12.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4779999999999999e-06 2024-09-17T21:30:12.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:12.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482326+0000", 2024-09-17T21:30:12.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7099999999999998e-06 2024-09-17T21:30:12.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:12.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532221+0000", 2024-09-17T21:30:12.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049894936000000001 2024-09-17T21:30:12.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:12.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532287+0000", 2024-09-17T21:30:12.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5914000000000006e-05 2024-09-17T21:30:12.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:12.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:12.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:12.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532755+0000", 2024-09-17T21:30:12.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000468107 2024-09-17T21:30:12.826 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.826 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.826 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.826 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:12.826 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:12.826 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:12.826 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:30:12.828 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:30:12.829 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone14 from 0x4c7d38 (overlap 0x4c7d38) to 0x7a61f 2024-09-17T21:30:12.830 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.1a.1 2024-09-17T21:30:12.830 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:12.826+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:12.854 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-17T21:30:12.928 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:12.926+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:12.952 INFO:teuthology.orchestra.run.smithi129.stdout:1776 read 0x1d9742 thru 0x1e32a3 (0x9b62 bytes) 2024-09-17T21:30:12.970 INFO:teuthology.orchestra.run.smithi129.stdout:1778 writesame 0x4aa7ec0 thru 0x4ab6420 data_size 0x1(0xe561 bytes) 2024-09-17T21:30:12.972 INFO:teuthology.orchestra.run.smithi129.stdout:1779 trunc from 0x4ab6421 to 0x5a13dcc 2024-09-17T21:30:12.978 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:12.978 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:12.978 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:12.978 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:12.979 INFO:teuthology.orchestra.run.smithi129.stdout:1780 punch from 0x25f8ac2 to 0x260057e, (0x7abc bytes) 2024-09-17T21:30:12.979 INFO:teuthology.orchestra.run.smithi129.stdout:1781 punch from 0x45c4caf to 0x45ca386, (0x56d7 bytes) 2024-09-17T21:30:12.979 INFO:teuthology.orchestra.run.smithi129.stdout:1782 trunc from 0x5a13dcc to 0x52b7257 2024-09-17T21:30:12.985 INFO:teuthology.orchestra.run.smithi129.stdout:1783 punch from 0x2bcbae1 to 0x2bd72b7, (0xb7d6 bytes) 2024-09-17T21:30:12.986 INFO:teuthology.orchestra.run.smithi129.stdout:1784 punch from 0x2cf5a89 to 0x2d03050, (0xd5c7 bytes) 2024-09-17T21:30:12.986 INFO:teuthology.orchestra.run.smithi129.stdout:1786 writesame 0x36f6131 thru 0x36fbfaa data_size 0x57(0x5e7a bytes) 2024-09-17T21:30:12.989 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:30:12.990 INFO:teuthology.orchestra.run.smithi129.stdout:1787 read 0x317c980 thru 0x3182d75 (0x63f6 bytes) 2024-09-17T21:30:12.990 INFO:teuthology.orchestra.run.smithi129.stdout:1789 trunc from 0x52b7257 to 0x47ed460 2024-09-17T21:30:12.997 INFO:teuthology.orchestra.run.smithi129.stdout:1791 read 0x4262ed6 thru 0x42649e5 (0x1b10 bytes) 2024-09-17T21:30:13.004 INFO:teuthology.orchestra.run.smithi129.stdout:1792 trunc from 0x47ed460 to 0xa56d8b6 2024-09-17T21:30:13.011 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:13.011 INFO:teuthology.orchestra.run.smithi129.stdout:1795 punch from 0x5faedcf to 0x5fb0aef, (0x1d20 bytes) 2024-09-17T21:30:13.011 INFO:teuthology.orchestra.run.smithi129.stdout:1797 read 0x9b1b3c3 thru 0x9b2182b (0x6469 bytes) 2024-09-17T21:30:13.011 INFO:teuthology.orchestra.run.smithi129.stdout:1798 read 0x8634c2d thru 0x86362ff (0x16d3 bytes) 2024-09-17T21:30:13.012 INFO:teuthology.orchestra.run.smithi129.stdout:1799 read 0x586e924 thru 0x587e143 (0xf820 bytes) 2024-09-17T21:30:13.012 INFO:teuthology.orchestra.run.smithi129.stdout:1800 writesame 0x9c755de thru 0x9c77cd0 data_size 0xd(0x26f3 bytes) 2024-09-17T21:30:13.014 INFO:teuthology.orchestra.run.smithi129.stdout:1801 punch from 0x1d92039 to 0x1d9590d, (0x38d4 bytes) 2024-09-17T21:30:13.015 INFO:teuthology.orchestra.run.smithi129.stdout:1802 read 0x411a69f thru 0x4121325 (0x6c87 bytes) 2024-09-17T21:30:13.017 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:13.017 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:13.018 INFO:teuthology.orchestra.run.smithi129.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-17T21:30:13.024 INFO:teuthology.orchestra.run.smithi129.stdout:1803 write 0xccbe1fa thru 0xcccded2 (0xfcd9 bytes) 2024-09-17T21:30:13.029 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:13.027+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:13.032 INFO:teuthology.orchestra.run.smithi129.stdout:1804 write 0xdc7da44 thru 0xdc7e0ac (0x669 bytes) 2024-09-17T21:30:13.035 INFO:teuthology.orchestra.run.smithi129.stdout:1805 writesame 0x4d7bd57 thru 0x4d81c94 data_size 0x49(0x5f3e bytes) 2024-09-17T21:30:13.040 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:13.040 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:13.041 INFO:teuthology.orchestra.run.smithi129.stdout:1806 write 0xb54eeca thru 0xb55b57d (0xc6b4 bytes) 2024-09-17T21:30:13.043 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:13.043 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:13.043 INFO:teuthology.orchestra.run.smithi129.stdout:1809 writesame 0x241b05e thru 0x241da07 data_size 0x1(0x29aa bytes) 2024-09-17T21:30:13.050 INFO:teuthology.orchestra.run.smithi129.stdout:1810 write 0xd2dac49 thru 0xd2e3ffc (0x93b4 bytes) 2024-09-17T21:30:13.053 INFO:teuthology.orchestra.run.smithi129.stdout:1811 read 0x232a6a thru 0x23b9d6 (0x8f6d bytes) 2024-09-17T21:30:13.054 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:13.054 INFO:teuthology.orchestra.run.smithi129.stdout:1813 read 0x7fdb08c thru 0x7fe1b27 (0x6a9c bytes) 2024-09-17T21:30:13.057 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:13.057 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:13.064 INFO:teuthology.orchestra.run.smithi129.stdout:1814 trunc from 0xdc7e0ad to 0x551c27c 2024-09-17T21:30:13.076 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:13.076 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:13.076 INFO:teuthology.orchestra.run.smithi129.stdout:1817 read 0x18a56eb thru 0x18a8175 (0x2a8b bytes) 2024-09-17T21:30:13.077 INFO:teuthology.orchestra.run.smithi129.stdout:1818 punch from 0x1f08170 to 0x1f0af9a, (0x2e2a bytes) 2024-09-17T21:30:13.081 INFO:teuthology.orchestra.run.smithi129.stdout:1819 write 0x5e3d8b9 thru 0x5e46801 (0x8f49 bytes) 2024-09-17T21:30:13.084 INFO:teuthology.orchestra.run.smithi129.stdout:1820 punch from 0x45c0f11 to 0x45cf4dd, (0xe5cc bytes) 2024-09-17T21:30:13.094 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:13.094 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:13.099 INFO:teuthology.orchestra.run.smithi129.stdout:1821 writesame 0xe26cc38 thru 0xe272aed data_size 0x1b(0x5eb6 bytes) 2024-09-17T21:30:13.099 INFO:teuthology.orchestra.run.smithi129.stdout:1822 write 0xbaf46b6 thru 0xbb03b5c (0xf4a7 bytes) 2024-09-17T21:30:13.103 INFO:teuthology.orchestra.run.smithi129.stdout:1823 read 0x1796165 thru 0x17a1b5e (0xb9fa bytes) 2024-09-17T21:30:13.103 INFO:teuthology.orchestra.run.smithi129.stdout:1824 read 0x44a920f thru 0x44ab703 (0x24f5 bytes) 2024-09-17T21:30:13.112 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:13.112 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:13.117 INFO:teuthology.orchestra.run.smithi129.stdout:1825 trunc from 0xe272aee to 0x3876537 2024-09-17T21:30:13.120 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:13.121 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:13.125 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:30:13.130 INFO:teuthology.orchestra.run.smithi129.stdout:1826 punch from 0x3073063 to 0x30772f8, (0x4295 bytes) 2024-09-17T21:30:13.130 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:13.130 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:13.128+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:13.140 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:13.140 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:13.144 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:30:13.145 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:30:13.148 INFO:teuthology.orchestra.run.smithi129.stdout:1828 write 0xe75bf53 thru 0xe76bbbb (0xfc69 bytes) 2024-09-17T21:30:13.151 INFO:teuthology.orchestra.run.smithi129.stdout:1830 write 0x54b9e6a thru 0x54c2f11 (0x90a8 bytes) 2024-09-17T21:30:13.155 INFO:teuthology.orchestra.run.smithi129.stdout:1832 punch from 0x1608378 to 0x1611008, (0x8c90 bytes) 2024-09-17T21:30:13.155 INFO:teuthology.orchestra.run.smithi129.stdout:1833 write 0xa09ca78 thru 0xa0a8877 (0xbe00 bytes) 2024-09-17T21:30:13.160 INFO:teuthology.orchestra.run.smithi129.stdout:1834 write 0x324093b thru 0x32499b1 (0x9077 bytes) 2024-09-17T21:30:13.162 INFO:teuthology.orchestra.run.smithi129.stdout:1835 read 0x650cf18 thru 0x6515387 (0x8470 bytes) 2024-09-17T21:30:13.162 INFO:teuthology.orchestra.run.smithi129.stdout:1836 read 0x6145722 thru 0x614fdd5 (0xa6b4 bytes) 2024-09-17T21:30:13.162 INFO:teuthology.orchestra.run.smithi129.stdout:1837 read 0xab02a7b thru 0xab0c259 (0x97df bytes) 2024-09-17T21:30:13.162 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:13.162 INFO:teuthology.orchestra.run.smithi129.stdout:1840 writesame 0x4d32cb2 thru 0x4d332bf data_size 0x19(0x60e bytes) 2024-09-17T21:30:13.175 INFO:teuthology.orchestra.run.smithi129.stdout:1841 trunc from 0xe76bbbc to 0x4d8069e 2024-09-17T21:30:13.197 INFO:teuthology.orchestra.run.smithi129.stdout:1842 write 0xa38a2ad thru 0xa395e61 (0xbbb5 bytes) 2024-09-17T21:30:13.211 INFO:teuthology.orchestra.run.smithi129.stdout:1843 trunc from 0xa395e62 to 0x2c3b04 2024-09-17T21:30:13.231 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:13.229+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:13.251 INFO:teuthology.orchestra.run.smithi129.stdout:1845 writesame 0xce9dc5e thru 0xceaa454 data_size 0x7(0xc7f7 bytes) 2024-09-17T21:30:13.251 INFO:teuthology.orchestra.run.smithi129.stdout:1846 read 0x8741469 thru 0x874fc60 (0xe7f8 bytes) 2024-09-17T21:30:13.251 INFO:teuthology.orchestra.run.smithi129.stdout:1848 writesame 0xb4c4d9d thru 0xb4caec1 data_size 0x1(0x6125 bytes) 2024-09-17T21:30:13.255 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:13.255 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:13.265 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:30:13.265 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:13.265 INFO:teuthology.orchestra.run.smithi129.stdout:1850 write 0x797fe27 thru 0x7980406 (0x5e0 bytes) 2024-09-17T21:30:13.269 INFO:teuthology.orchestra.run.smithi129.stdout:1851 punch from 0x8a8f35 to 0x8b1525, (0x85f0 bytes) 2024-09-17T21:30:13.269 INFO:teuthology.orchestra.run.smithi129.stdout:1852 write 0xbde23d7 thru 0xbdeba07 (0x9631 bytes) 2024-09-17T21:30:13.271 INFO:teuthology.orchestra.run.smithi129.stdout:1854 write 0xa89c51b thru 0xa8a142b (0x4f11 bytes) 2024-09-17T21:30:13.272 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:13.273 INFO:teuthology.orchestra.run.smithi129.stdout:1856 trunc from 0xceaa455 to 0xcd88a09 2024-09-17T21:30:13.278 INFO:teuthology.orchestra.run.smithi129.stdout:1859 write 0x3070cb7 thru 0x3075c99 (0x4fe3 bytes) 2024-09-17T21:30:13.281 INFO:teuthology.orchestra.run.smithi129.stdout:1860 read 0x8e20ce2 thru 0x8e217c5 (0xae4 bytes) 2024-09-17T21:30:13.282 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:13.282 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:13.282 INFO:teuthology.orchestra.run.smithi129.stdout:1863 read 0xa9cb057 thru 0xa9cd03f (0x1fe9 bytes) 2024-09-17T21:30:13.282 INFO:teuthology.orchestra.run.smithi129.stdout:1864 write 0x82a68bd thru 0x82b6732 (0xfe76 bytes) 2024-09-17T21:30:13.292 INFO:teuthology.orchestra.run.smithi129.stdout:1866 write 0xe19b974 thru 0xe19cd5d (0x13ea bytes) 2024-09-17T21:30:13.295 INFO:teuthology.orchestra.run.smithi129.stdout:1867 write 0x61a2c22 thru 0x61af776 (0xcb55 bytes) 2024-09-17T21:30:13.299 INFO:teuthology.orchestra.run.smithi129.stdout:1868 read 0xb471d0 thru 0xb4fbda (0x8a0b bytes) 2024-09-17T21:30:13.301 INFO:teuthology.orchestra.run.smithi129.stdout:1869 trunc from 0xe19cd5e to 0xc3ff4ca 2024-09-17T21:30:13.320 INFO:teuthology.orchestra.run.smithi129.stdout:1870 write 0xeb361ef thru 0xeb3a031 (0x3e43 bytes) 2024-09-17T21:30:13.332 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:13.330+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:13.338 INFO:teuthology.orchestra.run.smithi129.stdout:1871 trunc from 0xeb3a032 to 0x2777329 2024-09-17T21:30:13.359 INFO:teuthology.orchestra.run.smithi129.stdout:1872 writesame 0x68c3da3 thru 0x68c601c data_size 0x3(0x227a bytes) 2024-09-17T21:30:13.360 INFO:teuthology.orchestra.run.smithi129.stdout:1873 flatten 2024-09-17T21:30:13.368 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:13.368 INFO:teuthology.orchestra.run.smithi129.stdout:1876 write 0x2565955 thru 0x256841d (0x2ac9 bytes) 2024-09-17T21:30:13.372 INFO:teuthology.orchestra.run.smithi129.stdout:1877 read 0x1e40114 thru 0x1e4cb14 (0xca01 bytes) 2024-09-17T21:30:13.373 INFO:teuthology.orchestra.run.smithi129.stdout:1878 clone 16 order 22 su 65536 sc 4 2024-09-17T21:30:13.386 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:13.386 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:13.395 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:30:13.433 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:13.431+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:13.534 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:13.532+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:13.539 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:13.540 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:13.549 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:30:13.636 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:13.633+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:13.648 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is unset 2024-09-17T21:30:13.663 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:13.663 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:13.673 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:30:13.736 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:13.734+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:13.778 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:13.779 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:13.788 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:30:13.831 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-17T21:30:13.837 INFO:teuthology.orchestra.run.smithi129.stdout:1879 read 0x3e459c7 thru 0x3e465ea (0xc24 bytes) 2024-09-17T21:30:13.837 INFO:teuthology.orchestra.run.smithi129.stdout:1883 read 0x562026a thru 0x5621f7e (0x1d15 bytes) 2024-09-17T21:30:13.838 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:13.850 INFO:teuthology.orchestra.run.smithi129.stdout:1885 write 0x9675c6a thru 0x9683c26 (0xdfbd bytes) 2024-09-17T21:30:13.854 INFO:teuthology.orchestra.run.smithi129.stdout:1887 writesame 0x1f57b78 thru 0x1f5a501 data_size 0xd(0x298a bytes) 2024-09-17T21:30:13.858 INFO:teuthology.orchestra.run.smithi129.stdout:1889 read 0x1bbae98 thru 0x1bc5686 (0xa7ef bytes) 2024-09-17T21:30:13.858 INFO:teuthology.orchestra.run.smithi129.stdout:1890 writesame 0x1a44338 thru 0x1a4af70 data_size 0x5(0x6c39 bytes) 2024-09-17T21:30:13.871 INFO:teuthology.orchestra.run.smithi129.stdout:1891 write 0xb378c7f thru 0xb384384 (0xb706 bytes) 2024-09-17T21:30:13.872 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:13.870+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:13.887 INFO:teuthology.orchestra.run.smithi129.stdout:1892 trunc from 0xb384385 to 0x2354c88 2024-09-17T21:30:13.903 INFO:teuthology.orchestra.run.smithi129.stdout:1893 read 0x476329 thru 0x47eb17 (0x87ef bytes) 2024-09-17T21:30:13.903 INFO:teuthology.orchestra.run.smithi129.stdout:1895 write 0x87a62a thru 0x87e5de (0x3fb5 bytes) 2024-09-17T21:30:13.914 INFO:teuthology.orchestra.run.smithi129.stdout:1896 write 0x30ef200 thru 0x30f4de8 (0x5be9 bytes) 2024-09-17T21:30:13.925 INFO:teuthology.orchestra.run.smithi129.stdout:1898 writesame 0x54d8730 thru 0x54de120 data_size 0x19(0x59f1 bytes) 2024-09-17T21:30:13.926 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:13.926 INFO:teuthology.orchestra.run.smithi129.stdout:1900 read 0x435fd30 thru 0x4361dd0 (0x20a1 bytes) 2024-09-17T21:30:13.926 INFO:teuthology.orchestra.run.smithi129.stdout:1901 read 0x247e922 thru 0x24852bb (0x699a bytes) 2024-09-17T21:30:13.926 INFO:teuthology.orchestra.run.smithi129.stdout:1902 write 0x298790f thru 0x298a04c (0x273e bytes) 2024-09-17T21:30:13.927 INFO:teuthology.orchestra.run.smithi129.stdout:1903 write 0x42cb197 thru 0x42d388d (0x86f7 bytes) 2024-09-17T21:30:13.931 INFO:teuthology.orchestra.run.smithi129.stdout:1905 punch from 0x41b6232 to 0x41c1280, (0xb04e bytes) 2024-09-17T21:30:13.931 INFO:teuthology.orchestra.run.smithi129.stdout:1908 read 0x28daed9 thru 0x28e9dce (0xeef6 bytes) 2024-09-17T21:30:13.938 INFO:teuthology.orchestra.run.smithi129.stdout:1909 write 0x6af0853 thru 0x6af9928 (0x90d6 bytes) 2024-09-17T21:30:13.938 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:13.936+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:13.940 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:13.943 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:13.943 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:13.948 INFO:teuthology.orchestra.run.smithi129.stdout:1911 trunc from 0x6af9929 to 0x155c78b 2024-09-17T21:30:13.954 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:30:13.972 INFO:teuthology.orchestra.run.smithi129.stdout:1912 write 0x73cc948 thru 0x73ce0e0 (0x1799 bytes) 2024-09-17T21:30:13.983 INFO:teuthology.orchestra.run.smithi129.stdout:1913 write 0x8d64c6c thru 0x8d686e2 (0x3a77 bytes) 2024-09-17T21:30:13.991 INFO:teuthology.orchestra.run.smithi129.stdout:1914 writesame 0x978eb95 thru 0x9797b01 data_size 0x1(0x8f6d bytes) 2024-09-17T21:30:13.991 INFO:teuthology.orchestra.run.smithi129.stdout:1915 read 0x2bc25b4 thru 0x2bceab3 (0xc500 bytes) 2024-09-17T21:30:13.991 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:13.996 INFO:teuthology.orchestra.run.smithi129.stdout:1917 write 0x9fd558f thru 0x9fe5249 (0xfcbb bytes) 2024-09-17T21:30:13.999 INFO:teuthology.orchestra.run.smithi129.stdout:1919 write 0x90f6c7 thru 0x91ca41 (0xd37b bytes) 2024-09-17T21:30:14.015 INFO:teuthology.orchestra.run.smithi129.stdout:1920 write 0xcd08e5a thru 0xcd17592 (0xe739 bytes) 2024-09-17T21:30:14.022 INFO:teuthology.orchestra.run.smithi129.stdout:1922 read 0x3589376 thru 0x358e373 (0x4ffe bytes) 2024-09-17T21:30:14.022 INFO:teuthology.orchestra.run.smithi129.stdout:1923 read 0x602e19 thru 0x60786c (0x4a54 bytes) 2024-09-17T21:30:14.023 INFO:teuthology.orchestra.run.smithi129.stdout:1924 read 0x81b0231 thru 0x81b0555 (0x325 bytes) 2024-09-17T21:30:14.023 INFO:teuthology.orchestra.run.smithi129.stdout:1927 write 0x1f57892 thru 0x1f61531 (0x9ca0 bytes) 2024-09-17T21:30:14.024 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:14.024 INFO:teuthology.orchestra.run.smithi129.stdout:1930 read 0x71dbedc thru 0x71ddee1 (0x2006 bytes) 2024-09-17T21:30:14.024 INFO:teuthology.orchestra.run.smithi129.stdout:1931 read 0x77b61db thru 0x77c0286 (0xa0ac bytes) 2024-09-17T21:30:14.025 INFO:teuthology.orchestra.run.smithi129.stdout:1932 read 0x911256d thru 0x9113754 (0x11e8 bytes) 2024-09-17T21:30:14.025 INFO:teuthology.orchestra.run.smithi129.stdout:1933 read 0x553006a thru 0x5538a2d (0x89c4 bytes) 2024-09-17T21:30:14.032 INFO:teuthology.orchestra.run.smithi129.stdout:1936 trunc from 0xcd17593 to 0x66ddc83 2024-09-17T21:30:14.038 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:14.036+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:14.044 INFO:teuthology.orchestra.run.smithi129.stdout:1937 punch from 0x323982a to 0x3245065, (0xb83b bytes) 2024-09-17T21:30:14.058 INFO:teuthology.orchestra.run.smithi129.stdout:1938 write 0xe42c4ff thru 0xe42e018 (0x1b1a bytes) 2024-09-17T21:30:14.061 INFO:teuthology.orchestra.run.smithi129.stdout:1939 read 0x59979fb thru 0x59a10c7 (0x96cd bytes) 2024-09-17T21:30:14.061 INFO:teuthology.orchestra.run.smithi129.stdout:1941 write 0xe0ec4e5 thru 0xe0f821a (0xbd36 bytes) 2024-09-17T21:30:14.064 INFO:teuthology.orchestra.run.smithi129.stdout:1942 write 0x335b595 thru 0x33605e8 (0x5054 bytes) 2024-09-17T21:30:14.080 INFO:teuthology.orchestra.run.smithi129.stdout:1943 trunc from 0xe42e019 to 0x3477ad7 2024-09-17T21:30:14.090 INFO:teuthology.orchestra.run.smithi129.stdout:1946 read 0x2def58d thru 0x2dfc413 (0xce87 bytes) 2024-09-17T21:30:14.091 INFO:teuthology.orchestra.run.smithi129.stdout:1947 punch from 0x1e30e99 to 0x1e372b9, (0x6420 bytes) 2024-09-17T21:30:14.091 INFO:teuthology.orchestra.run.smithi129.stdout:1948 punch from 0x97f19d to 0x988414, (0x9277 bytes) 2024-09-17T21:30:14.091 INFO:teuthology.orchestra.run.smithi129.stdout:1949 punch from 0x4167a1 to 0x424f54, (0xe7b3 bytes) 2024-09-17T21:30:14.102 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:14.102 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:14.103 INFO:teuthology.orchestra.run.smithi129.stdout:1952 write 0x9c3b281 thru 0x9c3cc25 (0x19a5 bytes) 2024-09-17T21:30:14.106 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:14.106 INFO:teuthology.orchestra.run.smithi129.stdout:1957 read 0x4440080 thru 0x444b746 (0xb6c7 bytes) 2024-09-17T21:30:14.111 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:30:14.116 INFO:teuthology.orchestra.run.smithi129.stdout:1959 write 0xe31fd66 thru 0xe322f7b (0x3216 bytes) 2024-09-17T21:30:14.119 INFO:teuthology.orchestra.run.smithi129.stdout:1960 read 0x1b9b362 thru 0x1ba81f1 (0xce90 bytes) 2024-09-17T21:30:14.120 INFO:teuthology.orchestra.run.smithi129.stdout:1961 punch from 0x384e768 to 0x3854c73, (0x650b bytes) 2024-09-17T21:30:14.120 INFO:teuthology.orchestra.run.smithi129.stdout:1962 write 0xdcae3fd thru 0xdcb010e (0x1d12 bytes) 2024-09-17T21:30:14.123 INFO:teuthology.orchestra.run.smithi129.stdout:1964 read 0xc6cd5f4 thru 0xc6d3bd9 (0x65e6 bytes) 2024-09-17T21:30:14.123 INFO:teuthology.orchestra.run.smithi129.stdout:1965 write 0xd37caa thru 0xd4078a (0x8ae1 bytes) 2024-09-17T21:30:14.125 INFO:teuthology.orchestra.run.smithi129.stdout:1966 punch from 0x98f2ee2 to 0x9902975, (0xfa93 bytes) 2024-09-17T21:30:14.125 INFO:teuthology.orchestra.run.smithi129.stdout:1967 read 0x6f43b1c thru 0x6f50238 (0xc71d bytes) 2024-09-17T21:30:14.125 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:14.126 INFO:teuthology.orchestra.run.smithi129.stdout:1969 punch from 0x1c0af3a to 0x1c1602b, (0xb0f1 bytes) 2024-09-17T21:30:14.126 INFO:teuthology.orchestra.run.smithi129.stdout:1970 read 0x4a3628c thru 0x4a4494d (0xe6c2 bytes) 2024-09-17T21:30:14.126 INFO:teuthology.orchestra.run.smithi129.stdout:1972 read 0xc743a46 thru 0xc74526f (0x182a bytes) 2024-09-17T21:30:14.126 INFO:teuthology.orchestra.run.smithi129.stdout:1974 read 0xca92ea5 thru 0xcaa0d3f (0xde9b bytes) 2024-09-17T21:30:14.126 INFO:teuthology.orchestra.run.smithi129.stdout:1976 write 0x401dd69 thru 0x402ba52 (0xdcea bytes) 2024-09-17T21:30:14.131 INFO:teuthology.orchestra.run.smithi129.stdout:1977 writesame 0xde7cdc2 thru 0xde8ab3e data_size 0x1(0xdd7d bytes) 2024-09-17T21:30:14.139 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:14.137+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:14.148 INFO:teuthology.orchestra.run.smithi129.stdout:1979 writesame 0xc65f0a6 thru 0xc661d2f data_size 0x1(0x2c8a bytes) 2024-09-17T21:30:14.154 INFO:teuthology.orchestra.run.smithi129.stdout:1980 punch from 0x2135155 to 0x21402f9, (0xb1a4 bytes) 2024-09-17T21:30:14.167 INFO:teuthology.orchestra.run.smithi129.stdout:1981 trunc from 0xe322f7c to 0x3e30ea2 2024-09-17T21:30:14.177 INFO:teuthology.orchestra.run.smithi129.stdout:1982 flatten 2024-09-17T21:30:14.185 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:14.185 INFO:teuthology.orchestra.run.smithi129.stdout:1984 writesame 0xa07207 thru 0xa11952 data_size 0x56(0xa74c bytes) 2024-09-17T21:30:14.186 INFO:teuthology.orchestra.run.smithi129.stdout:1986 write 0x3743f68 thru 0x375076a (0xc803 bytes) 2024-09-17T21:30:14.189 INFO:teuthology.orchestra.run.smithi129.stdout:1987 punch from 0xab7f67 to 0xac5217, (0xd2b0 bytes) 2024-09-17T21:30:14.189 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:14.190 INFO:teuthology.orchestra.run.smithi129.stdout:1989 trunc from 0x3e30ea2 to 0x365ab56 2024-09-17T21:30:14.196 INFO:teuthology.orchestra.run.smithi129.stdout:1990 read 0x2c36d08 thru 0x2c37b1e (0xe17 bytes) 2024-09-17T21:30:14.209 INFO:teuthology.orchestra.run.smithi129.stdout:1991 write 0xa4f5876 thru 0xa4f6718 (0xea3 bytes) 2024-09-17T21:30:14.212 INFO:teuthology.orchestra.run.smithi129.stdout:1993 writesame 0x60d8ffa thru 0x60e7da9 data_size 0x8(0xedb0 bytes) 2024-09-17T21:30:14.217 INFO:teuthology.orchestra.run.smithi129.stdout:1994 punch from 0x23d83c9 to 0x23dcc2a, (0x4861 bytes) 2024-09-17T21:30:14.261 INFO:teuthology.orchestra.run.smithi129.stdout:1995 clone 17 order 25 su 8388608 sc 6 2024-09-17T21:30:14.262 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:14.262 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:14.262 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:14.260+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:14.267 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:30:14.363 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:14.361+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:14.412 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:14.412 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:14.420 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:30:14.464 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:14.462+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:14.524 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:14.524 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:14.532 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:30:14.565 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:14.563+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:14.667 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:14.664+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:14.680 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:14.680 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:14.689 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:30:14.768 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:14.765+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:14.773 INFO:teuthology.orchestra.run.smithi129.stdout:get_pg_num_history pg_num_history pg_num_history(e47 pg_nums {1={17=1},2={20=8},3={25=32}} deleted_pools ) 2024-09-17T21:30:14.774 INFO:teuthology.orchestra.run.smithi129.stdout:Importing pgid 3.1a 2024-09-17T21:30:14.774 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:582fa413:::rbd_data.11207744cc2c.0000000000000149:head# 2024-09-17T21:30:14.774 INFO:teuthology.orchestra.run.smithi129.stdout:snapset 0=[]:{} 2024-09-17T21:30:14.775 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:58668c6d:::rbd_data.1120b21c3309.000000000000006e:head# 2024-09-17T21:30:14.775 INFO:teuthology.orchestra.run.smithi129.stdout:snapset f=[]:{} 2024-09-17T21:30:14.776 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:587ac12c:::rbd_data.1120b21c3309.00000000000000f5:head# 2024-09-17T21:30:14.776 INFO:teuthology.orchestra.run.smithi129.stdout:snapset f=[]:{} 2024-09-17T21:30:14.777 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:592066d8:::rbd_data.11207744cc2c.0000000000000087:5# 2024-09-17T21:30:14.777 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:592066d8:::rbd_data.11207744cc2c.0000000000000087:head# 2024-09-17T21:30:14.778 INFO:teuthology.orchestra.run.smithi129.stdout:snapset 5=[]:{5=[5]} 2024-09-17T21:30:14.778 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4# 2024-09-17T21:30:14.780 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:head# 2024-09-17T21:30:14.780 INFO:teuthology.orchestra.run.smithi129.stdout:snapset 4=[]:{4=[4]} 2024-09-17T21:30:14.781 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5a0f90aa:::rbd_data.11207744cc2c.0000000000000145:head# 2024-09-17T21:30:14.781 INFO:teuthology.orchestra.run.smithi129.stdout:snapset 0=[]:{} 2024-09-17T21:30:14.781 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5a30f936:::rbd_data.1120b21c3309.00000000000000b8:head# 2024-09-17T21:30:14.781 INFO:teuthology.orchestra.run.smithi129.stdout:snapset f=[]:{} 2024-09-17T21:30:14.782 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5a4e2dc1:::rbd_data.1120b21c3309.00000000000000d4:head# 2024-09-17T21:30:14.782 INFO:teuthology.orchestra.run.smithi129.stdout:snapset f=[]:{} 2024-09-17T21:30:14.782 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5a6712ae:::rbd_id.image_client.0-clone11:head# 2024-09-17T21:30:14.783 INFO:teuthology.orchestra.run.smithi129.stdout:snapset 0=[]:{} 2024-09-17T21:30:14.783 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5a9e26c2:::rbd_object_map.1120555f8258:head# 2024-09-17T21:30:14.783 INFO:teuthology.orchestra.run.smithi129.stdout:snapset 0=[]:{} 2024-09-17T21:30:14.783 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5ac70768:::rbd_data.1120b21c3309.0000000000000163:head# 2024-09-17T21:30:14.783 INFO:teuthology.orchestra.run.smithi129.stdout:snapset f=[]:{} 2024-09-17T21:30:14.783 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5b6d4ec2:::rbd_data.1120d7424d8b.000000000000000a:e# 2024-09-17T21:30:14.784 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5b6d4ec2:::rbd_data.1120d7424d8b.000000000000000a:head# 2024-09-17T21:30:14.785 INFO:teuthology.orchestra.run.smithi129.stdout:snapset e=[]:{e=[e]} 2024-09-17T21:30:14.785 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5c2e077a:::rbd_data.1120b21c3309.0000000000000155:head# 2024-09-17T21:30:14.785 INFO:teuthology.orchestra.run.smithi129.stdout:snapset f=[]:{} 2024-09-17T21:30:14.785 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5c360ae6:::rbd_data.1120b21c3309.000000000000015d:head# 2024-09-17T21:30:14.785 INFO:teuthology.orchestra.run.smithi129.stdout:snapset f=[]:{} 2024-09-17T21:30:14.785 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5ca68579:::rbd_data.1120d7424d8b.000000000000007c:e# 2024-09-17T21:30:14.786 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5ca68579:::rbd_data.1120d7424d8b.000000000000007c:head# 2024-09-17T21:30:14.786 INFO:teuthology.orchestra.run.smithi129.stdout:snapset e=[]:{e=[e]} 2024-09-17T21:30:14.786 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5cea7035:::rbd_info:head# 2024-09-17T21:30:14.786 INFO:teuthology.orchestra.run.smithi129.stdout:snapset 0=[]:{} 2024-09-17T21:30:14.786 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5cfb4ea8:::rbd_data.1120b21c3309.0000000000000020:f# 2024-09-17T21:30:14.787 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5cfb4ea8:::rbd_data.1120b21c3309.0000000000000020:head# 2024-09-17T21:30:14.787 INFO:teuthology.orchestra.run.smithi129.stdout:snapset f=[]:{f=[f]} 2024-09-17T21:30:14.788 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5d18af01:::rbd_data.1120b21c3309.000000000000009c:head# 2024-09-17T21:30:14.788 INFO:teuthology.orchestra.run.smithi129.stdout:snapset f=[]:{} 2024-09-17T21:30:14.788 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5d1e6b55:::rbd_data.1120b21c3309.0000000000000029:head# 2024-09-17T21:30:14.788 INFO:teuthology.orchestra.run.smithi129.stdout:snapset f=[]:{} 2024-09-17T21:30:14.788 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5d3d0a43:::rbd_data.1120d7424d8b.000000000000001e:e# 2024-09-17T21:30:14.789 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5d3d0a43:::rbd_data.1120d7424d8b.000000000000001e:head# 2024-09-17T21:30:14.789 INFO:teuthology.orchestra.run.smithi129.stdout:snapset e=[]:{e=[e]} 2024-09-17T21:30:14.789 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5da731dc:::rbd_data.1120818f9bfd.000000000000000b:head# 2024-09-17T21:30:14.792 INFO:teuthology.orchestra.run.smithi129.stdout:snapset 0=[]:{} 2024-09-17T21:30:14.794 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:14.796 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:14.796 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5dde59b0:::rbd_data.1120b21c3309.000000000000009e:head# 2024-09-17T21:30:14.797 INFO:teuthology.orchestra.run.smithi129.stdout:snapset f=[]:{} 2024-09-17T21:30:14.797 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5e28433d:::rbd_data.1120e62eea89.0000000000000005:head# 2024-09-17T21:30:14.797 INFO:teuthology.orchestra.run.smithi129.stdout:snapset 0=[]:{} 2024-09-17T21:30:14.797 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5e7dbd4c:::rbd_data.11207744cc2c.0000000000000022:5# 2024-09-17T21:30:14.798 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5e7dbd4c:::rbd_data.11207744cc2c.0000000000000022:head# 2024-09-17T21:30:14.798 INFO:teuthology.orchestra.run.smithi129.stdout:snapset 5=[]:{5=[5]} 2024-09-17T21:30:14.799 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5f070e49:::rbd_data.1120b21c3309.0000000000000125:head# 2024-09-17T21:30:14.799 INFO:teuthology.orchestra.run.smithi129.stdout:snapset f=[]:{} 2024-09-17T21:30:14.799 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5f6403b3:::rbd_data.1120b21c3309.0000000000000051:head# 2024-09-17T21:30:14.799 INFO:teuthology.orchestra.run.smithi129.stdout:snapset f=[]:{} 2024-09-17T21:30:14.799 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5f730104:::rbd_data.1120b21c3309.0000000000000179:head# 2024-09-17T21:30:14.800 INFO:teuthology.orchestra.run.smithi129.stdout:snapset f=[]:{} 2024-09-17T21:30:14.800 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5f7349db:::rbd_data.1120b21c3309.00000000000000ff:head# 2024-09-17T21:30:14.800 INFO:teuthology.orchestra.run.smithi129.stdout:snapset f=[]:{} 2024-09-17T21:30:14.800 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:5fdea744:::rbd_data.1120b21c3309.0000000000000096:head# 2024-09-17T21:30:14.801 INFO:teuthology.orchestra.run.smithi129.stdout:snapset f=[]:{} 2024-09-17T21:30:14.801 INFO:teuthology.orchestra.run.smithi129.stdout:write_pg epoch 45 info 3.1a( v 45'239 (0'0,45'239] local-lis/les=25/26 n=36 ec=25/25 lis/c=25/25 les/c/f=26/26/0 sis=25) 2024-09-17T21:30:14.802 INFO:teuthology.orchestra.run.smithi129.stdout:Import successful 2024-09-17T21:30:14.803 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:30:14.813 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-17T21:30:14.869 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:14.867+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:14.908 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:14.908 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:14.917 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:30:14.933 INFO:teuthology.orchestra.run.smithi129.stdout:1996 trunc from 0xa4f6719 to 0x756f569 2024-09-17T21:30:14.958 INFO:teuthology.orchestra.run.smithi129.stdout:1997 trunc from 0x756f569 to 0xbe2b3ee 2024-09-17T21:30:14.964 INFO:teuthology.orchestra.run.smithi129.stdout:1998 writesame 0x7e0e87d thru 0x7e188b2 data_size 0x1(0xa036 bytes) 2024-09-17T21:30:14.992 INFO:teuthology.orchestra.run.smithi129.stdout:2000 write 0xe492cc9 thru 0xe49c5f8 (0x9930 bytes) 2024-09-17T21:30:14.995 INFO:teuthology.orchestra.run.smithi129.stdout:2002 punch from 0x7a1e579 to 0x7a2bea6, (0xd92d bytes) 2024-09-17T21:30:14.995 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:14.995 INFO:teuthology.orchestra.run.smithi129.stdout:2004 punch from 0x2c152e8 to 0x2c1f8f2, (0xa60a bytes) 2024-09-17T21:30:15.001 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:14.999+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:15.011 INFO:teuthology.orchestra.run.smithi129.stdout:2005 trunc from 0xe49c5f9 to 0x6788e6 2024-09-17T21:30:15.034 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.034 INFO:teuthology.orchestra.run.smithi129.stdout:2007 read 0x67c10 thru 0x7396f (0xbd60 bytes) 2024-09-17T21:30:15.052 INFO:teuthology.orchestra.run.smithi129.stdout:2008 write 0xa1e998a thru 0xa1f93ba (0xfa31 bytes) 2024-09-17T21:30:15.055 INFO:teuthology.orchestra.run.smithi129.stdout:2009 read 0x4b75d46 thru 0x4b83c10 (0xdecb bytes) 2024-09-17T21:30:15.055 INFO:teuthology.orchestra.run.smithi129.stdout:2010 writesame 0x148700a thru 0x1491ff9 data_size 0x28(0xaff0 bytes) 2024-09-17T21:30:15.057 INFO:teuthology.orchestra.run.smithi129.stdout:2011 read 0x2d30a52 thru 0x2d3c891 (0xbe40 bytes) 2024-09-17T21:30:15.057 INFO:teuthology.orchestra.run.smithi129.stdout:2012 write 0x35e704d thru 0x35eb066 (0x401a bytes) 2024-09-17T21:30:15.059 INFO:teuthology.orchestra.run.smithi129.stdout:2013 write 0x59f4f53 thru 0x5a01a25 (0xcad3 bytes) 2024-09-17T21:30:15.062 INFO:teuthology.orchestra.run.smithi129.stdout:2014 write 0x78e092e thru 0x78e2343 (0x1a16 bytes) 2024-09-17T21:30:15.064 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.064 INFO:teuthology.orchestra.run.smithi129.stdout:2017 read 0x5ca715 thru 0x5d4e3e (0xa72a bytes) 2024-09-17T21:30:15.065 INFO:teuthology.orchestra.run.smithi129.stdout:2018 write 0x73ea613 thru 0x73eee6e (0x485c bytes) 2024-09-17T21:30:15.066 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.066 INFO:teuthology.orchestra.run.smithi129.stdout:2023 punch from 0x1a6df42 to 0x1a7609a, (0x8158 bytes) 2024-09-17T21:30:15.066 INFO:teuthology.orchestra.run.smithi129.stdout:2024 write 0x2ffe73 thru 0x30988b (0x9a19 bytes) 2024-09-17T21:30:15.067 INFO:teuthology.orchestra.run.smithi129.stdout:2025 read 0x60157f5 thru 0x601d44c (0x7c58 bytes) 2024-09-17T21:30:15.070 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:15.070 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:15.070 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:15.068+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:15.078 INFO:teuthology.orchestra.run.smithi129.stdout:2027 write 0xe156ad1 thru 0xe156c57 (0x187 bytes) 2024-09-17T21:30:15.080 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:30:15.080 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.080 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.081 INFO:teuthology.orchestra.run.smithi129.stdout:2031 read 0xa4d42af thru 0xa4d79c8 (0x371a bytes) 2024-09-17T21:30:15.093 INFO:teuthology.orchestra.run.smithi129.stdout:2032 trunc from 0xe156c58 to 0x44fbcfc 2024-09-17T21:30:15.104 INFO:teuthology.orchestra.run.smithi129.stdout:2033 punch from 0x10dffab to 0x10eea2a, (0xea7f bytes) 2024-09-17T21:30:15.104 INFO:teuthology.orchestra.run.smithi129.stdout:2034 write 0x387fd18 thru 0x388718f (0x7478 bytes) 2024-09-17T21:30:15.106 INFO:teuthology.orchestra.run.smithi129.stdout:2036 read 0x39b5d69 thru 0x39b8dc1 (0x3059 bytes) 2024-09-17T21:30:15.107 INFO:teuthology.orchestra.run.smithi129.stdout:2037 write 0x3bc9a34 thru 0x3bcfc36 (0x6203 bytes) 2024-09-17T21:30:15.108 INFO:teuthology.orchestra.run.smithi129.stdout:2038 read 0x1e963bb thru 0x1e98b9e (0x27e4 bytes) 2024-09-17T21:30:15.118 INFO:teuthology.orchestra.run.smithi129.stdout:2039 writesame 0x73f7c46 thru 0x73f9d70 data_size 0x1(0x212b bytes) 2024-09-17T21:30:15.130 INFO:teuthology.orchestra.run.smithi129.stdout:2040 write 0xbb7f82f thru 0xbb8a4fc (0xacce bytes) 2024-09-17T21:30:15.139 INFO:teuthology.orchestra.run.smithi129.stdout:2041 trunc from 0xbb8a4fd to 0x56888d5 2024-09-17T21:30:15.145 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:30:15.155 INFO:teuthology.orchestra.run.smithi129.stdout:2042 write 0x6ddc643 thru 0x6dde3c1 (0x1d7f bytes) 2024-09-17T21:30:15.156 INFO:teuthology.orchestra.run.smithi129.stdout:2044 writesame 0x214b1ae thru 0x214e09e data_size 0x1(0x2ef1 bytes) 2024-09-17T21:30:15.162 INFO:teuthology.orchestra.run.smithi129.stdout:2046 read 0x28c7175 thru 0x28cedab (0x7c37 bytes) 2024-09-17T21:30:15.170 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:15.168+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:15.173 INFO:teuthology.orchestra.run.smithi129.stdout:2047 write 0xadc701d thru 0xadc79d5 (0x9b9 bytes) 2024-09-17T21:30:15.175 INFO:teuthology.orchestra.run.smithi129.stdout:2049 read 0x7a34 thru 0x12037 (0xa604 bytes) 2024-09-17T21:30:15.175 INFO:teuthology.orchestra.run.smithi129.stdout:2050 flatten 2024-09-17T21:30:15.191 INFO:teuthology.orchestra.run.smithi129.stdout:2051 write 0xe9d8160 thru 0xe9dd028 (0x4ec9 bytes) 2024-09-17T21:30:15.194 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.194 INFO:teuthology.orchestra.run.smithi129.stdout:2053 punch from 0x694ac20 to 0x69554a6, (0xa886 bytes) 2024-09-17T21:30:15.194 INFO:teuthology.orchestra.run.smithi129.stdout:2055 read 0xfd4bf7 thru 0xfdfcfd (0xb107 bytes) 2024-09-17T21:30:15.195 INFO:teuthology.orchestra.run.smithi129.stdout:2056 writesame 0x2c54adb thru 0x2c554ef data_size 0x1(0xa15 bytes) 2024-09-17T21:30:15.196 INFO:teuthology.orchestra.run.smithi129.stdout:2058 write 0xa680f5a thru 0xa68cfc2 (0xc069 bytes) 2024-09-17T21:30:15.198 INFO:teuthology.orchestra.run.smithi129.stdout:2059 punch from 0xb8e3d2c to 0xb8eb8ea, (0x7bbe bytes) 2024-09-17T21:30:15.198 INFO:teuthology.orchestra.run.smithi129.stdout:2060 write 0xce7b56b thru 0xce839aa (0x8440 bytes) 2024-09-17T21:30:15.201 INFO:teuthology.orchestra.run.smithi129.stdout:2061 write 0xc4053e6 thru 0xc415186 (0xfda1 bytes) 2024-09-17T21:30:15.206 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.206 INFO:teuthology.orchestra.run.smithi129.stdout:2063 read 0x3c5eacd thru 0x3c63b0b (0x503f bytes) 2024-09-17T21:30:15.206 INFO:teuthology.orchestra.run.smithi129.stdout:2064 punch from 0x71bfa84 to 0x71ce69a, (0xec16 bytes) 2024-09-17T21:30:15.206 INFO:teuthology.orchestra.run.smithi129.stdout:2065 read 0xe2cdb67 thru 0xe2d5e86 (0x8320 bytes) 2024-09-17T21:30:15.206 INFO:teuthology.orchestra.run.smithi129.stdout:2068 punch from 0x86d5789 to 0x86de489, (0x8d00 bytes) 2024-09-17T21:30:15.206 INFO:teuthology.orchestra.run.smithi129.stdout:2070 read 0xc9f326b thru 0xc9fa229 (0x6fbf bytes) 2024-09-17T21:30:15.206 INFO:teuthology.orchestra.run.smithi129.stdout:2071 read 0xd6d1492 thru 0xd6dfeff (0xea6e bytes) 2024-09-17T21:30:15.206 INFO:teuthology.orchestra.run.smithi129.stdout:2072 punch from 0x67c9ece to 0x67cddc6, (0x3ef8 bytes) 2024-09-17T21:30:15.206 INFO:teuthology.orchestra.run.smithi129.stdout:2073 read 0xd1e363b thru 0xd1e43fa (0xdc0 bytes) 2024-09-17T21:30:15.206 INFO:teuthology.orchestra.run.smithi129.stdout:2074 writesame 0x47e98dd thru 0x47eb6bc data_size 0x10(0x1de0 bytes) 2024-09-17T21:30:15.207 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.207 INFO:teuthology.orchestra.run.smithi129.stdout:2076 read 0x6b8f16d thru 0x6b9e93c (0xf7d0 bytes) 2024-09-17T21:30:15.207 INFO:teuthology.orchestra.run.smithi129.stdout:2077 writesame 0xd313144 thru 0xd31fafe data_size 0x1(0xc9bb bytes) 2024-09-17T21:30:15.229 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:15.229 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:15.233 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.238 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:30:15.239 INFO:teuthology.orchestra.run.smithi129.stdout:2079 trunc from 0xe9dd029 to 0x9f3856a 2024-09-17T21:30:15.257 INFO:teuthology.orchestra.run.smithi129.stdout:2080 trunc from 0x9f3856a to 0x63bd0cc 2024-09-17T21:30:15.263 INFO:teuthology.orchestra.run.smithi129.stdout:2081 read 0x60eda7c thru 0x60fc017 (0xe59c bytes) 2024-09-17T21:30:15.264 INFO:teuthology.orchestra.run.smithi129.stdout:2082 write 0x2880b7 thru 0x28c360 (0x42aa bytes) 2024-09-17T21:30:15.265 INFO:teuthology.orchestra.run.smithi129.stdout:2083 writesame 0x393f73 thru 0x395084 data_size 0x26(0x1112 bytes) 2024-09-17T21:30:15.268 INFO:teuthology.orchestra.run.smithi129.stdout:2085 read 0x275e9c thru 0x283864 (0xd9c9 bytes) 2024-09-17T21:30:15.269 INFO:teuthology.orchestra.run.smithi129.stdout:2087 read 0x342c1d0 thru 0x3437763 (0xb594 bytes) 2024-09-17T21:30:15.269 INFO:teuthology.orchestra.run.smithi129.stdout:2088 write 0x9d3569 thru 0x9de24a (0xace2 bytes) 2024-09-17T21:30:15.277 INFO:teuthology.orchestra.run.smithi129.stdout:2090 trunc from 0x63bd0cc to 0xb2704e6 2024-09-17T21:30:15.279 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:15.277+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:15.280 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:15.280 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:15.280 INFO:teuthology.orchestra.run.smithi129.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-17T21:30:15.283 INFO:teuthology.orchestra.run.smithi129.stdout:2091 write 0x1d7f9b6 thru 0x1d7fc4a (0x295 bytes) 2024-09-17T21:30:15.284 INFO:teuthology.orchestra.run.smithi129.stdout:2092 read 0x760d6bf thru 0x761d36a (0xfcac bytes) 2024-09-17T21:30:15.284 INFO:teuthology.orchestra.run.smithi129.stdout:2093 writesame 0x3ebcc3c thru 0x3ec018b data_size 0x8(0x3550 bytes) 2024-09-17T21:30:15.286 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.286 INFO:teuthology.orchestra.run.smithi129.stdout:2095 punch from 0x715ccc7 to 0x7165962, (0x8c9b bytes) 2024-09-17T21:30:15.286 INFO:teuthology.orchestra.run.smithi129.stdout:2096 writesame 0x1b7eb78 thru 0x1b85562 data_size 0x55(0x69eb bytes) 2024-09-17T21:30:15.290 INFO:teuthology.orchestra.run.smithi129.stdout:2097 trunc from 0xb2704e6 to 0xcffe269 2024-09-17T21:30:15.295 INFO:teuthology.orchestra.run.smithi129.stdout:2099 read 0x11995cd thru 0x119fc5a (0x668e bytes) 2024-09-17T21:30:15.295 INFO:teuthology.orchestra.run.smithi129.stdout:2100 read 0xccb1306 thru 0xccbe7a1 (0xd49c bytes) 2024-09-17T21:30:15.296 INFO:teuthology.orchestra.run.smithi129.stdout:2101 read 0xaef3a06 thru 0xaefdf28 (0xa523 bytes) 2024-09-17T21:30:15.296 INFO:teuthology.orchestra.run.smithi129.stdout:2102 write 0x3043c21 thru 0x3044595 (0x975 bytes) 2024-09-17T21:30:15.297 INFO:teuthology.orchestra.run.smithi129.stdout:2104 punch from 0x79dbaf4 to 0x79e0e66, (0x5372 bytes) 2024-09-17T21:30:15.297 INFO:teuthology.orchestra.run.smithi129.stdout:2105 writesame 0x7e6b49c thru 0x7e7a3c6 data_size 0x3(0xef2b bytes) 2024-09-17T21:30:15.299 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:15.299 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:15.307 INFO:teuthology.orchestra.run.smithi129.stdout:2109 punch from 0x82fbc9b to 0x83090c1, (0xd426 bytes) 2024-09-17T21:30:15.307 INFO:teuthology.orchestra.run.smithi129.stdout:2111 punch from 0x2bef3bc to 0x2bf30da, (0x3d1e bytes) 2024-09-17T21:30:15.313 INFO:teuthology.orchestra.run.smithi129.stdout:2112 trunc from 0xcffe269 to 0x7cae6d5 2024-09-17T21:30:15.316 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:15.316 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:15.323 INFO:teuthology.orchestra.run.smithi129.stdout:2113 read 0x771a722 thru 0x7729d0a (0xf5e9 bytes) 2024-09-17T21:30:15.324 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.324 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.324 INFO:teuthology.orchestra.run.smithi129.stdout:2117 writesame 0x600f7eb thru 0x6013405 data_size 0x17(0x3c1b bytes) 2024-09-17T21:30:15.325 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f /var/log/ceph/exp.3.1a.1 2024-09-17T21:30:15.334 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:15.335 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:15.337 INFO:teuthology.orchestra.run.smithi129.stdout:2118 write 0xbab0508 thru 0xbabae55 (0xa94e bytes) 2024-09-17T21:30:15.344 INFO:teuthology.orchestra.run.smithi129.stdout:2120 write 0xd083e8c thru 0xd086ba7 (0x2d1c bytes) 2024-09-17T21:30:15.353 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:15.354 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:15.354 INFO:teuthology.orchestra.run.smithi129.stdout:2121 trunc from 0xd086ba8 to 0x704a84a 2024-09-17T21:30:15.368 INFO:teuthology.orchestra.run.smithi129.stdout:2122 read 0x12dd476 thru 0x12df7d0 (0x235b bytes) 2024-09-17T21:30:15.368 INFO:teuthology.orchestra.run.smithi129.stdout:2123 write 0x46da896 thru 0x46e0d27 (0x6492 bytes) 2024-09-17T21:30:15.372 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:15.370+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:15.373 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:15.373 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:15.377 INFO:teuthology.orchestra.run.smithi129.stdout:2124 trunc from 0x704a84a to 0x120b26d 2024-09-17T21:30:15.390 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:15.390 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:15.393 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:15.393 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:15.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:15.393 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:15.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.393 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:30:15.393 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:15.393 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.450342376000002, 2024-09-17T21:30:15.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61572000699999996, 2024-09-17T21:30:15.393 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.393 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.393 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:15.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:30:15.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:30:15.394 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T21:30:15.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.394 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:15.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:15.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939745+0000", 2024-09-17T21:30:15.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6899999999999999e-06 2024-09-17T21:30:15.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939753+0000", 2024-09-17T21:30:15.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1950000000000003e-06 2024-09-17T21:30:15.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939767+0000", 2024-09-17T21:30:15.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3429e-05 2024-09-17T21:30:15.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939776+0000", 2024-09-17T21:30:15.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:30:15.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939858+0000", 2024-09-17T21:30:15.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2496000000000004e-05 2024-09-17T21:30:15.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T21:30:15.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939868+0000", 2024-09-17T21:30:15.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1910000000000006e-06 2024-09-17T21:30:15.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940612+0000", 2024-09-17T21:30:15.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074429599999999998 2024-09-17T21:30:15.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:30:15.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940621+0000", 2024-09-17T21:30:15.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9139999999999997e-06 2024-09-17T21:30:15.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554006+0000", 2024-09-17T21:30:15.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61338505099999996 2024-09-17T21:30:15.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554166+0000", 2024-09-17T21:30:15.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015982600000000001 2024-09-17T21:30:15.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:30:15.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554296+0000", 2024-09-17T21:30:15.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013002399999999999 2024-09-17T21:30:15.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:15.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554936+0000", 2024-09-17T21:30:15.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064009400000000001 2024-09-17T21:30:15.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:15.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555389+0000", 2024-09-17T21:30:15.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045355200000000002 2024-09-17T21:30:15.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:15.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555432+0000", 2024-09-17T21:30:15.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2477e-05 2024-09-17T21:30:15.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555463+0000", 2024-09-17T21:30:15.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1612000000000002e-05 2024-09-17T21:30:15.401 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.401 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.401 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.401 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17527 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1005568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.401 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:15.401 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.985114544999998, 2024-09-17T21:30:15.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046452090000000001, 2024-09-17T21:30:15.401 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.401 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.401 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:15.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.402 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.402 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17527 2024-09-17T21:30:15.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.402 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:15.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:15.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405496+0000", 2024-09-17T21:30:15.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052498400000000004 2024-09-17T21:30:15.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405500+0000", 2024-09-17T21:30:15.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8410000000000002e-06 2024-09-17T21:30:15.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405511+0000", 2024-09-17T21:30:15.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0749e-05 2024-09-17T21:30:15.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405517+0000", 2024-09-17T21:30:15.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:30:15.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451275+0000", 2024-09-17T21:30:15.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045758134999999998 2024-09-17T21:30:15.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451355+0000", 2024-09-17T21:30:15.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9453e-05 2024-09-17T21:30:15.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451423+0000", 2024-09-17T21:30:15.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8782999999999995e-05 2024-09-17T21:30:15.405 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.405 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.405 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.405 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17530 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1046528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.406 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:15.406 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.984506893999999, 2024-09-17T21:30:15.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046376737000000001, 2024-09-17T21:30:15.406 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.406 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.406 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:15.406 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.406 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.406 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.406 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17530 2024-09-17T21:30:15.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.406 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:15.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:15.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:15.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3300000000000002e-07 2024-09-17T21:30:15.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405581+0000", 2024-09-17T21:30:15.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.074e-06 2024-09-17T21:30:15.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405587+0000", 2024-09-17T21:30:15.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7559999999999996e-06 2024-09-17T21:30:15.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405592+0000", 2024-09-17T21:30:15.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8860000000000003e-06 2024-09-17T21:30:15.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451813+0000", 2024-09-17T21:30:15.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046221119999999997 2024-09-17T21:30:15.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451891+0000", 2024-09-17T21:30:15.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7449999999999999e-05 2024-09-17T21:30:15.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451956+0000", 2024-09-17T21:30:15.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4918000000000004e-05 2024-09-17T21:30:15.410 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.410 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.410 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.410 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17615 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.410 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:15.410 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.979874057, 2024-09-17T21:30:15.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046741309000000002, 2024-09-17T21:30:15.410 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.410 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.411 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:15.411 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.411 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.411 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.411 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17615 2024-09-17T21:30:15.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.411 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:15.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:15.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:15.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:30:15.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410213+0000", 2024-09-17T21:30:15.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:30:15.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410217+0000", 2024-09-17T21:30:15.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8199999999999998e-06 2024-09-17T21:30:15.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410220+0000", 2024-09-17T21:30:15.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6900000000000001e-06 2024-09-17T21:30:15.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455264+0000", 2024-09-17T21:30:15.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045044002999999999 2024-09-17T21:30:15.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455338+0000", 2024-09-17T21:30:15.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4182000000000002e-05 2024-09-17T21:30:15.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.456953+0000", 2024-09-17T21:30:15.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016150089999999999 2024-09-17T21:30:15.414 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.415 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.415 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.415 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17616 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 872448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.415 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:15.415 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.979862005000001, 2024-09-17T21:30:15.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946039000000002, 2024-09-17T21:30:15.415 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.415 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.415 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:15.415 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.415 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.415 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.416 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17616 2024-09-17T21:30:15.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.416 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:15.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:15.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:15.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T21:30:15.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410225+0000", 2024-09-17T21:30:15.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3459999999999999e-06 2024-09-17T21:30:15.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410228+0000", 2024-09-17T21:30:15.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.083e-06 2024-09-17T21:30:15.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410232+0000", 2024-09-17T21:30:15.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4419999999999998e-06 2024-09-17T21:30:15.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457004+0000", 2024-09-17T21:30:15.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046771873999999998 2024-09-17T21:30:15.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457084+0000", 2024-09-17T21:30:15.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0506000000000005e-05 2024-09-17T21:30:15.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457170+0000", 2024-09-17T21:30:15.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5494000000000005e-05 2024-09-17T21:30:15.419 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.419 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.419 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.420 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17617 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.420 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:15.420 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.979850602999999, 2024-09-17T21:30:15.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047084168000000003, 2024-09-17T21:30:15.420 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.420 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.420 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:15.420 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.420 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.420 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.420 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17617 2024-09-17T21:30:15.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.421 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:15.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:15.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:15.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-17T21:30:15.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410237+0000", 2024-09-17T21:30:15.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1960000000000001e-06 2024-09-17T21:30:15.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410246+0000", 2024-09-17T21:30:15.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7880000000000001e-06 2024-09-17T21:30:15.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410250+0000", 2024-09-17T21:30:15.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.518e-06 2024-09-17T21:30:15.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457209+0000", 2024-09-17T21:30:15.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046959283999999997 2024-09-17T21:30:15.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457264+0000", 2024-09-17T21:30:15.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673999999999999e-05 2024-09-17T21:30:15.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457319+0000", 2024-09-17T21:30:15.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5497000000000001e-05 2024-09-17T21:30:15.424 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.424 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.424 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.424 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17618 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 899072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.424 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:15.424 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.979831159, 2024-09-17T21:30:15.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047204208999999997, 2024-09-17T21:30:15.425 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.425 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.425 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:15.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.425 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17618 2024-09-17T21:30:15.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.425 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:15.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:15.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:15.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8799999999999998e-07 2024-09-17T21:30:15.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410256+0000", 2024-09-17T21:30:15.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T21:30:15.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410260+0000", 2024-09-17T21:30:15.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6500000000000002e-06 2024-09-17T21:30:15.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410263+0000", 2024-09-17T21:30:15.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7089999999999998e-06 2024-09-17T21:30:15.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457355+0000", 2024-09-17T21:30:15.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047092756999999999 2024-09-17T21:30:15.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457407+0000", 2024-09-17T21:30:15.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1808999999999998e-05 2024-09-17T21:30:15.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457459+0000", 2024-09-17T21:30:15.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1684000000000002e-05 2024-09-17T21:30:15.429 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.429 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.429 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.429 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17619 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 913408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.429 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:15.429 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.979819921000001, 2024-09-17T21:30:15.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047293443999999997, 2024-09-17T21:30:15.429 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.429 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.429 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:15.429 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.430 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17619 2024-09-17T21:30:15.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.430 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:15.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:15.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:15.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:30:15.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410267+0000", 2024-09-17T21:30:15.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:30:15.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410270+0000", 2024-09-17T21:30:15.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.01e-06 2024-09-17T21:30:15.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410273+0000", 2024-09-17T21:30:15.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6410000000000002e-06 2024-09-17T21:30:15.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457473+0000", 2024-09-17T21:30:15.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047199829999999998 2024-09-17T21:30:15.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457517+0000", 2024-09-17T21:30:15.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4058999999999999e-05 2024-09-17T21:30:15.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457559+0000", 2024-09-17T21:30:15.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2404999999999998e-05 2024-09-17T21:30:15.433 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.433 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.433 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.434 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17620 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 927744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.434 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:15.434 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.979808749, 2024-09-17T21:30:15.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047402409, 2024-09-17T21:30:15.434 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.434 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.434 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:15.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.434 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17620 2024-09-17T21:30:15.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.434 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:15.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:15.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:15.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:30:15.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410278+0000", 2024-09-17T21:30:15.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1790000000000001e-06 2024-09-17T21:30:15.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410281+0000", 2024-09-17T21:30:15.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.061e-06 2024-09-17T21:30:15.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410284+0000", 2024-09-17T21:30:15.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.65e-06 2024-09-17T21:30:15.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457587+0000", 2024-09-17T21:30:15.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047303206 2024-09-17T21:30:15.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457633+0000", 2024-09-17T21:30:15.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6063999999999998e-05 2024-09-17T21:30:15.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457679+0000", 2024-09-17T21:30:15.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6007000000000002e-05 2024-09-17T21:30:15.438 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.438 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.438 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.438 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17621 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 940032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.438 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:15.438 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.979798333000002, 2024-09-17T21:30:15.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047486550000000002, 2024-09-17T21:30:15.439 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.439 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.439 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:15.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.439 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17621 2024-09-17T21:30:15.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.439 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:15.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:15.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410288+0000", 2024-09-17T21:30:15.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:30:15.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410289+0000", 2024-09-17T21:30:15.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1179999999999999e-06 2024-09-17T21:30:15.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410292+0000", 2024-09-17T21:30:15.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:30:15.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410294+0000", 2024-09-17T21:30:15.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T21:30:15.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457694+0000", 2024-09-17T21:30:15.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047399416999999999 2024-09-17T21:30:15.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457734+0000", 2024-09-17T21:30:15.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0528999999999998e-05 2024-09-17T21:30:15.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457774+0000", 2024-09-17T21:30:15.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9529e-05 2024-09-17T21:30:15.443 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.443 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.443 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.443 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17622 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.443 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:15.443 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.979788151000001, 2024-09-17T21:30:15.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047596619, 2024-09-17T21:30:15.443 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.443 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.443 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:15.443 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.443 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.444 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.444 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17622 2024-09-17T21:30:15.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.444 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:15.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:15.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:15.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:15.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410303+0000", 2024-09-17T21:30:15.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2979999999999999e-06 2024-09-17T21:30:15.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410307+0000", 2024-09-17T21:30:15.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:30:15.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410310+0000", 2024-09-17T21:30:15.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.146e-06 2024-09-17T21:30:15.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457801+0000", 2024-09-17T21:30:15.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047491322000000002 2024-09-17T21:30:15.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457850+0000", 2024-09-17T21:30:15.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8585000000000003e-05 2024-09-17T21:30:15.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457894+0000", 2024-09-17T21:30:15.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4455e-05 2024-09-17T21:30:15.447 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.447 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.447 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.447 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17623 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 968704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.447 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:15.447 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.979771766999999, 2024-09-17T21:30:15.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047689377999999998, 2024-09-17T21:30:15.447 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.447 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.448 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:15.448 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.448 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.448 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.448 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17623 2024-09-17T21:30:15.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.448 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:15.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:15.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:15.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:30:15.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410315+0000", 2024-09-17T21:30:15.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0499999999999999e-06 2024-09-17T21:30:15.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410319+0000", 2024-09-17T21:30:15.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.517e-06 2024-09-17T21:30:15.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410321+0000", 2024-09-17T21:30:15.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:30:15.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457922+0000", 2024-09-17T21:30:15.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047600638000000001 2024-09-17T21:30:15.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457964+0000", 2024-09-17T21:30:15.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2126000000000003e-05 2024-09-17T21:30:15.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458003+0000", 2024-09-17T21:30:15.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9138999999999999e-05 2024-09-17T21:30:15.451 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.451 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.451 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.452 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17624 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.452 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:15.452 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.979761183000001, 2024-09-17T21:30:15.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047798292999999999, 2024-09-17T21:30:15.452 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.452 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.452 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:15.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.452 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17624 2024-09-17T21:30:15.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.453 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:15.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:15.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:15.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:30:15.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410326+0000", 2024-09-17T21:30:15.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0610000000000001e-06 2024-09-17T21:30:15.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410329+0000", 2024-09-17T21:30:15.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0319999999999999e-06 2024-09-17T21:30:15.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410331+0000", 2024-09-17T21:30:15.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5749999999999999e-06 2024-09-17T21:30:15.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458033+0000", 2024-09-17T21:30:15.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047701371999999999 2024-09-17T21:30:15.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458079+0000", 2024-09-17T21:30:15.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:30:15.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458123+0000", 2024-09-17T21:30:15.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850999999999997e-05 2024-09-17T21:30:15.456 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.456 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.456 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.456 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17625 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.456 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:15.456 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.979751022999999, 2024-09-17T21:30:15.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047883051000000003, 2024-09-17T21:30:15.457 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.457 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.457 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:15.457 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.457 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.457 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.457 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17625 2024-09-17T21:30:15.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.457 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:15.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:15.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:15.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:15.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410336+0000", 2024-09-17T21:30:15.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1310000000000001e-06 2024-09-17T21:30:15.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410339+0000", 2024-09-17T21:30:15.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000001e-06 2024-09-17T21:30:15.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410342+0000", 2024-09-17T21:30:15.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7460000000000001e-06 2024-09-17T21:30:15.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458137+0000", 2024-09-17T21:30:15.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047794931999999998 2024-09-17T21:30:15.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458178+0000", 2024-09-17T21:30:15.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1013999999999997e-05 2024-09-17T21:30:15.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458218+0000", 2024-09-17T21:30:15.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9675999999999998e-05 2024-09-17T21:30:15.460 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.461 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.461 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.461 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17626 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1009664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.461 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:15.461 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.979740402000001, 2024-09-17T21:30:15.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047991601000000002, 2024-09-17T21:30:15.461 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.461 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.461 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:15.461 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.461 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.462 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.462 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17626 2024-09-17T21:30:15.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.462 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:15.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:15.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410346+0000", 2024-09-17T21:30:15.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:30:15.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410347+0000", 2024-09-17T21:30:15.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.114e-06 2024-09-17T21:30:15.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410350+0000", 2024-09-17T21:30:15.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.101e-06 2024-09-17T21:30:15.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410353+0000", 2024-09-17T21:30:15.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T21:30:15.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458245+0000", 2024-09-17T21:30:15.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0478921 2024-09-17T21:30:15.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458292+0000", 2024-09-17T21:30:15.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397999999999999e-05 2024-09-17T21:30:15.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458337+0000", 2024-09-17T21:30:15.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5221e-05 2024-09-17T21:30:15.465 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.465 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.465 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.465 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17627 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.466 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:15.466 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.979703301000001, 2024-09-17T21:30:15.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048047383999999999, 2024-09-17T21:30:15.466 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.466 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.466 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:15.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.466 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17627 2024-09-17T21:30:15.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.466 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:15.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:15.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410383+0000", 2024-09-17T21:30:15.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:30:15.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410385+0000", 2024-09-17T21:30:15.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:30:15.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410391+0000", 2024-09-17T21:30:15.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:30:15.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410395+0000", 2024-09-17T21:30:15.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6539999999999998e-06 2024-09-17T21:30:15.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458350+0000", 2024-09-17T21:30:15.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047954855999999997 2024-09-17T21:30:15.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458392+0000", 2024-09-17T21:30:15.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2094000000000001e-05 2024-09-17T21:30:15.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458430+0000", 2024-09-17T21:30:15.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7474000000000001e-05 2024-09-17T21:30:15.470 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.470 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.470 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.470 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17628 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.470 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:15.470 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.979683739999999, 2024-09-17T21:30:15.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048142676000000002, 2024-09-17T21:30:15.471 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.471 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.471 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:15.471 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.471 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.471 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.471 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17628 2024-09-17T21:30:15.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.471 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:15.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:15.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:15.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:30:15.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410404+0000", 2024-09-17T21:30:15.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-06 2024-09-17T21:30:15.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410411+0000", 2024-09-17T21:30:15.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6850000000000004e-06 2024-09-17T21:30:15.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410415+0000", 2024-09-17T21:30:15.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6750000000000001e-06 2024-09-17T21:30:15.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458457+0000", 2024-09-17T21:30:15.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048042045999999998 2024-09-17T21:30:15.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458501+0000", 2024-09-17T21:30:15.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3661000000000001e-05 2024-09-17T21:30:15.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458545+0000", 2024-09-17T21:30:15.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3529000000000003e-05 2024-09-17T21:30:15.474 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.475 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.475 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.475 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17687 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.475 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:15.475 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.977238787000001, 2024-09-17T21:30:15.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050670448, 2024-09-17T21:30:15.475 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.475 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.475 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:15.475 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.475 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.475 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.476 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17687 2024-09-17T21:30:15.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.476 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:15.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:15.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412848+0000", 2024-09-17T21:30:15.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9100000000000003e-07 2024-09-17T21:30:15.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413102+0000", 2024-09-17T21:30:15.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025415600000000001 2024-09-17T21:30:15.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413111+0000", 2024-09-17T21:30:15.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2010000000000005e-06 2024-09-17T21:30:15.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413116+0000", 2024-09-17T21:30:15.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:30:15.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458559+0000", 2024-09-17T21:30:15.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045442914000000001 2024-09-17T21:30:15.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458632+0000", 2024-09-17T21:30:15.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3207000000000006e-05 2024-09-17T21:30:15.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:15.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458785+0000", 2024-09-17T21:30:15.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015226700000000001 2024-09-17T21:30:15.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:15.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.461648+0000", 2024-09-17T21:30:15.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028633069999999998 2024-09-17T21:30:15.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:15.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463416+0000", 2024-09-17T21:30:15.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017680230000000001 2024-09-17T21:30:15.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:15.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463475+0000", 2024-09-17T21:30:15.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9333000000000003e-05 2024-09-17T21:30:15.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463517+0000", 2024-09-17T21:30:15.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2162000000000001e-05 2024-09-17T21:30:15.481 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.481 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.481 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.481 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17688 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [write 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.481 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:15.481 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.976959918999999, 2024-09-17T21:30:15.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050625098, 2024-09-17T21:30:15.481 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.481 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.481 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:15.482 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.482 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.482 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.482 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17688 2024-09-17T21:30:15.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.482 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:15.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:15.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:15.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:30:15.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413128+0000", 2024-09-17T21:30:15.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.813e-06 2024-09-17T21:30:15.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413132+0000", 2024-09-17T21:30:15.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:30:15.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413135+0000", 2024-09-17T21:30:15.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:30:15.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459756+0000", 2024-09-17T21:30:15.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046621293000000001 2024-09-17T21:30:15.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459816+0000", 2024-09-17T21:30:15.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9156000000000001e-05 2024-09-17T21:30:15.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:15.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459912+0000", 2024-09-17T21:30:15.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6302000000000001e-05 2024-09-17T21:30:15.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:15.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463231+0000", 2024-09-17T21:30:15.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033188290000000001 2024-09-17T21:30:15.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:15.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463701+0000", 2024-09-17T21:30:15.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047068199999999999 2024-09-17T21:30:15.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:15.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463728+0000", 2024-09-17T21:30:15.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6726000000000001e-05 2024-09-17T21:30:15.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463751+0000", 2024-09-17T21:30:15.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2824e-05 2024-09-17T21:30:15.487 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.487 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.487 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.487 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17691 3.d 3:b3a055f3:::rbd_data.11207744cc2c.0000000000000083:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:15.487 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:15.487 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.973415016000001, 2024-09-17T21:30:15.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049309370999999998, 2024-09-17T21:30:15.488 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:15.488 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:15.488 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:15.488 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:15.488 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:15.488 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.488 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17691 2024-09-17T21:30:15.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.488 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:15.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:15.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:15.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:15.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:15.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:15.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:15.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416672+0000", 2024-09-17T21:30:15.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.47e-07 2024-09-17T21:30:15.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:15.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416915+0000", 2024-09-17T21:30:15.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024344500000000001 2024-09-17T21:30:15.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:15.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416924+0000", 2024-09-17T21:30:15.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7739999999999993e-06 2024-09-17T21:30:15.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416929+0000", 2024-09-17T21:30:15.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6990000000000002e-06 2024-09-17T21:30:15.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:15.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460414+0000", 2024-09-17T21:30:15.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043485466 2024-09-17T21:30:15.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:15.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460540+0000", 2024-09-17T21:30:15.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012608899999999999 2024-09-17T21:30:15.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:15.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460746+0000", 2024-09-17T21:30:15.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000205577 2024-09-17T21:30:15.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:15.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463800+0000", 2024-09-17T21:30:15.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030542030000000001 2024-09-17T21:30:15.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:15.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465867+0000", 2024-09-17T21:30:15.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020665929999999998 2024-09-17T21:30:15.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:15.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465914+0000", 2024-09-17T21:30:15.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497000000000003e-05 2024-09-17T21:30:15.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:15.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:15.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:15.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465980+0000", 2024-09-17T21:30:15.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6081000000000006e-05 2024-09-17T21:30:15.493 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.493 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.493 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.493 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:15.493 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:15.493 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:15.495 INFO:teuthology.orchestra.run.smithi129.stdout:2125 punch from 0xcc5957 to 0xcc60a1, (0x74a bytes) 2024-09-17T21:30:15.495 INFO:teuthology.orchestra.run.smithi129.stdout:2126 read 0xa21fc4 thru 0xa26bd7 (0x4c14 bytes) 2024-09-17T21:30:15.495 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.495 INFO:teuthology.orchestra.run.smithi129.stdout:2129 read 0x3915bf thru 0x39ef28 (0xd96a bytes) 2024-09-17T21:30:15.495 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:30:15.496 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:15.492+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:15.496 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:30:15.497 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:30:15.500 INFO:teuthology.orchestra.run.smithi129.stdout:2130 write 0x2653ac5 thru 0x2654455 (0x991 bytes) 2024-09-17T21:30:15.500 INFO:teuthology.orchestra.run.smithi129.stdout:2131 write 0x2339094 thru 0x2344ddf (0xbd4c bytes) 2024-09-17T21:30:15.500 INFO:teuthology.orchestra.run.smithi129.stdout:2132 write 0x1979abe thru 0x1985582 (0xbac5 bytes) 2024-09-17T21:30:15.500 INFO:teuthology.orchestra.run.smithi129.stdout:2133 read 0x14814c thru 0x155326 (0xd1db bytes) 2024-09-17T21:30:15.500 INFO:teuthology.orchestra.run.smithi129.stdout:2134 trunc from 0x2654456 to 0xc905ed 2024-09-17T21:30:15.500 INFO:teuthology.orchestra.run.smithi129.stdout:2135 read 0xaeafa2 thru 0xaf076d (0x57cc bytes) 2024-09-17T21:30:15.500 INFO:teuthology.orchestra.run.smithi129.stdout:2138 writesame 0x472f421 thru 0x4730e39 data_size 0x11(0x1a19 bytes) 2024-09-17T21:30:15.500 INFO:teuthology.orchestra.run.smithi129.stdout:2140 read 0x2e73c96 thru 0x2e816a3 (0xda0e bytes) 2024-09-17T21:30:15.500 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.501 INFO:teuthology.orchestra.run.smithi129.stdout:2143 read 0x2088aa7 thru 0x208e7d7 (0x5d31 bytes) 2024-09-17T21:30:15.501 INFO:teuthology.orchestra.run.smithi129.stdout:2144 read 0x25921d3 thru 0x259fdc5 (0xdbf3 bytes) 2024-09-17T21:30:15.501 INFO:teuthology.orchestra.run.smithi129.stdout:2145 write 0x6707e31 thru 0x671355f (0xb72f bytes) 2024-09-17T21:30:15.501 INFO:teuthology.orchestra.run.smithi129.stdout:2146 read 0x6611163 thru 0x66167c1 (0x565f bytes) 2024-09-17T21:30:15.501 INFO:teuthology.orchestra.run.smithi129.stdout:2147 read 0x2d0f97f thru 0x2d1681c (0x6e9e bytes) 2024-09-17T21:30:15.501 INFO:teuthology.orchestra.run.smithi129.stdout:2148 write 0x7b14e52 thru 0x7b1a404 (0x55b3 bytes) 2024-09-17T21:30:15.501 INFO:teuthology.orchestra.run.smithi129.stdout:2150 punch from 0x3958b81 to 0x3963df7, (0xb276 bytes) 2024-09-17T21:30:15.501 INFO:teuthology.orchestra.run.smithi129.stdout:2151 read 0x4d4b819 thru 0x4d4d990 (0x2178 bytes) 2024-09-17T21:30:15.501 INFO:teuthology.orchestra.run.smithi129.stdout:2152 write 0xa6ca2f2 thru 0xa6d88d7 (0xe5e6 bytes) 2024-09-17T21:30:15.501 INFO:teuthology.orchestra.run.smithi129.stdout:2153 read 0x94a0f2 thru 0x9593bc (0xf2cb bytes) 2024-09-17T21:30:15.501 INFO:teuthology.orchestra.run.smithi129.stdout:2155 trunc from 0xa6d88d8 to 0xada82e4 2024-09-17T21:30:15.501 INFO:teuthology.orchestra.run.smithi129.stdout:2156 write 0x3791bb1 thru 0x37a13c7 (0xf817 bytes) 2024-09-17T21:30:15.501 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.501 INFO:teuthology.orchestra.run.smithi129.stdout:2158 read 0x3df22fa thru 0x3df44dd (0x21e4 bytes) 2024-09-17T21:30:15.502 INFO:teuthology.orchestra.run.smithi129.stdout:2159 writesame 0x80ad5e2 thru 0x80bccfd data_size 0xa(0xf71c bytes) 2024-09-17T21:30:15.502 INFO:teuthology.orchestra.run.smithi129.stdout:2160 trunc from 0xada82e4 to 0x619dbff 2024-09-17T21:30:15.502 INFO:teuthology.orchestra.run.smithi129.stdout:2161 write 0x551c982 thru 0x552b0a6 (0xe725 bytes) 2024-09-17T21:30:15.502 INFO:teuthology.orchestra.run.smithi129.stdout:2162 punch from 0xbaea64 to 0xbb4131, (0x56cd bytes) 2024-09-17T21:30:15.502 INFO:teuthology.orchestra.run.smithi129.stdout:2163 writesame 0x89aeb7d thru 0x89b3458 data_size 0x2(0x48dc bytes) 2024-09-17T21:30:15.502 INFO:teuthology.orchestra.run.smithi129.stdout:2164 write 0x474853c thru 0x474e25c (0x5d21 bytes) 2024-09-17T21:30:15.502 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.502 INFO:teuthology.orchestra.run.smithi129.stdout:2171 punch from 0x34a78cf to 0x34aa7ee, (0x2f1f bytes) 2024-09-17T21:30:15.502 INFO:teuthology.orchestra.run.smithi129.stdout:2172 write 0x9c7c69 thru 0x9d79ba (0xfd52 bytes) 2024-09-17T21:30:15.503 INFO:teuthology.orchestra.run.smithi129.stdout:2173 read 0x7e9280 thru 0x7eb39f (0x2120 bytes) 2024-09-17T21:30:15.513 INFO:teuthology.orchestra.run.smithi129.stdout:2174 writesame 0xc9374ab thru 0xc944f16 data_size 0xe(0xda6c bytes) 2024-09-17T21:30:15.513 INFO:teuthology.orchestra.run.smithi129.stdout:2177 read 0x3392849 thru 0x3396942 (0x40fa bytes) 2024-09-17T21:30:15.513 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.513 INFO:teuthology.orchestra.run.smithi129.stdout:2179 read 0x1a9f4e7 thru 0x1aa5d1a (0x6834 bytes) 2024-09-17T21:30:15.513 INFO:teuthology.orchestra.run.smithi129.stdout:2180 read 0x9fc9b77 thru 0x9fcd4de (0x3968 bytes) 2024-09-17T21:30:15.521 INFO:teuthology.orchestra.run.smithi129.stdout:2181 trunc from 0xc944f17 to 0x62c28d2 2024-09-17T21:30:15.545 INFO:teuthology.orchestra.run.smithi129.stdout:2182 write 0xedce2ad thru 0xeddcc1e (0xe972 bytes) 2024-09-17T21:30:15.548 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.566 INFO:teuthology.orchestra.run.smithi129.stdout:2184 trunc from 0xeddcc1f to 0x1b9bb88 2024-09-17T21:30:15.579 INFO:teuthology.orchestra.run.smithi129.stdout:2185 read 0x1864756 thru 0x186e76d (0xa018 bytes) 2024-09-17T21:30:15.580 INFO:teuthology.orchestra.run.smithi129.stdout:2186 read 0xa81846 thru 0xa84956 (0x3111 bytes) 2024-09-17T21:30:15.595 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:15.593+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:15.597 INFO:teuthology.orchestra.run.smithi129.stdout:2187 write 0x9a6af3f thru 0x9a6fe3c (0x4efe bytes) 2024-09-17T21:30:15.599 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.599 INFO:teuthology.orchestra.run.smithi129.stdout:2189 punch from 0x7a9ee19 to 0x7aa56cd, (0x68b4 bytes) 2024-09-17T21:30:15.599 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.606 INFO:teuthology.orchestra.run.smithi129.stdout:2191 trunc from 0x9a6fe3d to 0x4245bbd 2024-09-17T21:30:15.613 INFO:teuthology.orchestra.run.smithi129.stdout:2192 punch from 0x2d84657 to 0x2d94177, (0xfb20 bytes) 2024-09-17T21:30:15.618 INFO:teuthology.orchestra.run.smithi129.stdout:2193 trunc from 0x4245bbd to 0xc94b50 2024-09-17T21:30:15.628 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.628 INFO:teuthology.orchestra.run.smithi129.stdout:2198 punch from 0x35ca4 to 0x4210e, (0xc46a bytes) 2024-09-17T21:30:15.628 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.628 INFO:teuthology.orchestra.run.smithi129.stdout:2201 read 0x9ea851 thru 0x9f69a1 (0xc151 bytes) 2024-09-17T21:30:15.641 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:15.641 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:15.641 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:15.641 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:15.641 INFO:teuthology.orchestra.run.smithi129.stdout:2202 writesame 0x5500bf4 thru 0x5504936 data_size 0x1(0x3d43 bytes) 2024-09-17T21:30:15.659 INFO:teuthology.orchestra.run.smithi129.stdout:2203 writesame 0xd51aff8 thru 0xd5219e5 data_size 0xe(0x69ee bytes) 2024-09-17T21:30:15.659 INFO:teuthology.orchestra.run.smithi129.stdout:2204 write 0x6b1c7ea thru 0x6b2af20 (0xe737 bytes) 2024-09-17T21:30:15.661 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:30:15.661 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.664 INFO:teuthology.orchestra.run.smithi129.stdout:2208 trunc from 0xd5219e6 to 0xa299b3f 2024-09-17T21:30:15.667 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:30:15.674 INFO:teuthology.orchestra.run.smithi129.stdout:2209 trunc from 0xa299b3f to 0x6fa0d9e 2024-09-17T21:30:15.680 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.689 INFO:teuthology.orchestra.run.smithi129.stdout:2211 trunc from 0x6fa0d9e to 0xd99e019 2024-09-17T21:30:15.695 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:15.693+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:15.696 INFO:teuthology.orchestra.run.smithi129.stdout:2212 write 0x2958902 thru 0x295a67c (0x1d7b bytes) 2024-09-17T21:30:15.714 INFO:teuthology.orchestra.run.smithi129.stdout:2216 trunc from 0xd99e019 to 0x16a46d8 2024-09-17T21:30:15.740 INFO:teuthology.orchestra.run.smithi129.stdout:2217 write 0xab929eb thru 0xab93b07 (0x111d bytes) 2024-09-17T21:30:15.744 INFO:teuthology.orchestra.run.smithi129.stdout:2219 read 0x17beba7 thru 0x17c3d4b (0x51a5 bytes) 2024-09-17T21:30:15.744 INFO:teuthology.orchestra.run.smithi129.stdout:2220 writesame 0x870261a thru 0x870b95f data_size 0x7(0x9346 bytes) 2024-09-17T21:30:15.749 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.749 INFO:teuthology.orchestra.run.smithi129.stdout:2223 write 0x2674024 thru 0x26785cc (0x45a9 bytes) 2024-09-17T21:30:15.750 INFO:teuthology.orchestra.run.smithi129.stdout:2224 read 0x7007d96 thru 0x7014768 (0xc9d3 bytes) 2024-09-17T21:30:15.755 INFO:teuthology.orchestra.run.smithi129.stdout:2225 trunc from 0xab93b08 to 0x7350cb1 2024-09-17T21:30:15.761 INFO:teuthology.orchestra.run.smithi129.stdout:2226 write 0x4a17ab5 thru 0x4a1d58e (0x5ada bytes) 2024-09-17T21:30:15.767 INFO:teuthology.orchestra.run.smithi129.stdout:2227 trunc from 0x7350cb1 to 0xaa5fe9d 2024-09-17T21:30:15.778 INFO:teuthology.orchestra.run.smithi129.stdout:2228 write 0xb785be8 thru 0xb78aa7f (0x4e98 bytes) 2024-09-17T21:30:15.779 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.784 INFO:teuthology.orchestra.run.smithi129.stdout:2232 trunc from 0xb78aa80 to 0x757f243 2024-09-17T21:30:15.788 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:15.788 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:15.789 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:15.789 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:15.789 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:15.791 INFO:teuthology.orchestra.run.smithi129.stdout:2236 read 0x3d48302 thru 0x3d4a6c9 (0x23c8 bytes) 2024-09-17T21:30:15.791 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.792 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:15.792 INFO:teuthology.orchestra.run.smithi129.stdout:2239 read 0x4968d1d thru 0x4971ca4 (0x8f88 bytes) 2024-09-17T21:30:15.792 INFO:teuthology.orchestra.run.smithi129.stdout:2241 clone 18 order 23 su 524288 sc 5 2024-09-17T21:30:15.796 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:15.794+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:15.800 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:30:15.897 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:15.895+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:15.947 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:15.947 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:15.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:15.947 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:15.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.947 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:37817 3.17 3:eab85a8d:::rbd_header.1120e62eea89:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:30:15.947 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:15.948 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.248907554000001, 2024-09-17T21:30:15.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86635970399999995, 2024-09-17T21:30:15.948 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:15.948 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:15.948 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:15.948 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:15.948 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:15.948 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.948 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37817 2024-09-17T21:30:15.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.948 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:15.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:15.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:15.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:15.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:15.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:15.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695457+0000", 2024-09-17T21:30:15.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5799999999999998e-07 2024-09-17T21:30:15.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:15.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695461+0000", 2024-09-17T21:30:15.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1110000000000001e-06 2024-09-17T21:30:15.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:15.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695471+0000", 2024-09-17T21:30:15.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0441e-05 2024-09-17T21:30:15.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695477+0000", 2024-09-17T21:30:15.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7599999999999999e-06 2024-09-17T21:30:15.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:15.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695533+0000", 2024-09-17T21:30:15.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6570999999999998e-05 2024-09-17T21:30:15.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:30:15.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695543+0000", 2024-09-17T21:30:15.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9979999999999998e-06 2024-09-17T21:30:15.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:15.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561472+0000", 2024-09-17T21:30:15.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86592841899999995 2024-09-17T21:30:15.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:15.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561610+0000", 2024-09-17T21:30:15.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013854699999999999 2024-09-17T21:30:15.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:15.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561816+0000", 2024-09-17T21:30:15.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020509900000000001 2024-09-17T21:30:15.952 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:15.952 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:15.953 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:15.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.953 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50202 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:15.953 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:15.953 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8367367640000003, 2024-09-17T21:30:15.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.076620261999999995, 2024-09-17T21:30:15.953 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:15.953 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:15.953 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:15.953 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:15.953 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:15.953 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.953 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50202 2024-09-17T21:30:15.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.954 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:15.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:15.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:15.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:15.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:15.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:15.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107629+0000", 2024-09-17T21:30:15.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3709999999999998e-06 2024-09-17T21:30:15.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:15.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119765+0000", 2024-09-17T21:30:15.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012135683 2024-09-17T21:30:15.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:15.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119787+0000", 2024-09-17T21:30:15.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2748999999999999e-05 2024-09-17T21:30:15.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119797+0000", 2024-09-17T21:30:15.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8169999999999994e-06 2024-09-17T21:30:15.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:15.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119872+0000", 2024-09-17T21:30:15.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270999999999994e-05 2024-09-17T21:30:15.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:15.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119987+0000", 2024-09-17T21:30:15.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011557699999999999 2024-09-17T21:30:15.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:15.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.121766+0000", 2024-09-17T21:30:15.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00177853 2024-09-17T21:30:15.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:15.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178307+0000", 2024-09-17T21:30:15.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056541559999999998 2024-09-17T21:30:15.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:15.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184097+0000", 2024-09-17T21:30:15.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057899029999999999 2024-09-17T21:30:15.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:15.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184172+0000", 2024-09-17T21:30:15.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5226000000000001e-05 2024-09-17T21:30:15.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:15.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184247+0000", 2024-09-17T21:30:15.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4574999999999997e-05 2024-09-17T21:30:15.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:15.958 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:15.959 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:15.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.959 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50203 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [truncate 20971520] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:15.959 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:15.959 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8245368590000002, 2024-09-17T21:30:15.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065009779000000004, 2024-09-17T21:30:15.959 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:15.959 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:15.959 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:15.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:15.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:15.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.959 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50203 2024-09-17T21:30:15.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.960 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:15.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:15.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:15.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:15.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:15.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:15.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119828+0000", 2024-09-17T21:30:15.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6390000000000001e-06 2024-09-17T21:30:15.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:15.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119832+0000", 2024-09-17T21:30:15.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7699999999999999e-06 2024-09-17T21:30:15.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:15.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119845+0000", 2024-09-17T21:30:15.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3093e-05 2024-09-17T21:30:15.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119852+0000", 2024-09-17T21:30:15.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6950000000000003e-06 2024-09-17T21:30:15.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:15.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.160945+0000", 2024-09-17T21:30:15.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041093076999999999 2024-09-17T21:30:15.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:15.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161071+0000", 2024-09-17T21:30:15.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012569799999999999 2024-09-17T21:30:15.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:15.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161283+0000", 2024-09-17T21:30:15.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021201999999999999 2024-09-17T21:30:15.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:15.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.179360+0000", 2024-09-17T21:30:15.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018077784999999999 2024-09-17T21:30:15.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:15.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184726+0000", 2024-09-17T21:30:15.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053652960000000003 2024-09-17T21:30:15.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:15.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184787+0000", 2024-09-17T21:30:15.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1402000000000006e-05 2024-09-17T21:30:15.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:15.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184836+0000", 2024-09-17T21:30:15.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9304000000000001e-05 2024-09-17T21:30:15.964 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:15.964 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:15.964 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:15.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.964 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50206 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:15.965 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:15.965 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7874553190000002, 2024-09-17T21:30:15.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.090361968000000001, 2024-09-17T21:30:15.965 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:15.965 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:15.965 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:15.965 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:15.965 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:15.965 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.965 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50206 2024-09-17T21:30:15.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.965 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:15.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:15.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:15.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:15.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:15.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:15.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156911+0000", 2024-09-17T21:30:15.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-17T21:30:15.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:15.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173670+0000", 2024-09-17T21:30:15.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016758907 2024-09-17T21:30:15.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:15.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173696+0000", 2024-09-17T21:30:15.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6329000000000002e-05 2024-09-17T21:30:15.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173708+0000", 2024-09-17T21:30:15.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2512e-05 2024-09-17T21:30:15.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:15.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173788+0000", 2024-09-17T21:30:15.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9332999999999994e-05 2024-09-17T21:30:15.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:15.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173918+0000", 2024-09-17T21:30:15.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013063300000000001 2024-09-17T21:30:15.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:15.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.176228+0000", 2024-09-17T21:30:15.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002309757 2024-09-17T21:30:15.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:15.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.245164+0000", 2024-09-17T21:30:15.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068936301000000005 2024-09-17T21:30:15.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:15.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247176+0000", 2024-09-17T21:30:15.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020117590000000001 2024-09-17T21:30:15.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:15.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247215+0000", 2024-09-17T21:30:15.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9094000000000002e-05 2024-09-17T21:30:15.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:15.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247270+0000", 2024-09-17T21:30:15.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4793999999999997e-05 2024-09-17T21:30:15.970 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:15.970 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:15.970 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:15.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.970 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50207 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [truncate 21556327] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:15.971 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:15.971 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7706063390000004, 2024-09-17T21:30:15.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.074055363999999999, 2024-09-17T21:30:15.971 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:15.971 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:15.971 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:15.971 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:15.971 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:15.971 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.971 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50207 2024-09-17T21:30:15.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.971 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:15.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:15.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:15.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:15.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:15.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:15.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173759+0000", 2024-09-17T21:30:15.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8160000000000001e-06 2024-09-17T21:30:15.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:15.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173763+0000", 2024-09-17T21:30:15.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5360000000000003e-06 2024-09-17T21:30:15.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:15.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173771+0000", 2024-09-17T21:30:15.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8849999999999999e-06 2024-09-17T21:30:15.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173777+0000", 2024-09-17T21:30:15.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7799999999999997e-06 2024-09-17T21:30:15.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:15.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221363+0000", 2024-09-17T21:30:15.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047585493 2024-09-17T21:30:15.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:15.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221430+0000", 2024-09-17T21:30:15.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7129000000000001e-05 2024-09-17T21:30:15.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:15.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221550+0000", 2024-09-17T21:30:15.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120658 2024-09-17T21:30:15.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:15.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246113+0000", 2024-09-17T21:30:15.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02456262 2024-09-17T21:30:15.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:15.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247744+0000", 2024-09-17T21:30:15.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001630692 2024-09-17T21:30:15.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:15.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247777+0000", 2024-09-17T21:30:15.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3015000000000002e-05 2024-09-17T21:30:15.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:15.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247812+0000", 2024-09-17T21:30:15.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.574e-05 2024-09-17T21:30:15.976 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:15.976 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:15.976 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:15.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.977 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50828 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:15.977 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:15.977 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.4624944070000003, 2024-09-17T21:30:15.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047324185999999997, 2024-09-17T21:30:15.977 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:15.977 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:15.977 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:15.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:15.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:15.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.977 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50828 2024-09-17T21:30:15.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.978 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:15.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:15.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:15.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:15.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:15.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:15.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:15.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.46e-07 2024-09-17T21:30:15.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:15.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481871+0000", 2024-09-17T21:30:15.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T21:30:15.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:15.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481878+0000", 2024-09-17T21:30:15.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:30:15.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481882+0000", 2024-09-17T21:30:15.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:30:15.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:15.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529038+0000", 2024-09-17T21:30:15.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047156034999999999 2024-09-17T21:30:15.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:15.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529099+0000", 2024-09-17T21:30:15.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0316000000000003e-05 2024-09-17T21:30:15.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:15.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529193+0000", 2024-09-17T21:30:15.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4369000000000004e-05 2024-09-17T21:30:15.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:15.981 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:15.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:15.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.981 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50829 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 12845056~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:15.981 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:15.981 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.4624756129999996, 2024-09-17T21:30:15.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047887596999999997, 2024-09-17T21:30:15.982 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:15.982 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:15.982 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:15.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:15.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:15.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.982 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50829 2024-09-17T21:30:15.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.982 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:15.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:15.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:15.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:15.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:15.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:15.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:15.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:30:15.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:15.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481890+0000", 2024-09-17T21:30:15.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.877e-06 2024-09-17T21:30:15.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:15.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481895+0000", 2024-09-17T21:30:15.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:30:15.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481900+0000", 2024-09-17T21:30:15.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4440000000000001e-06 2024-09-17T21:30:15.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:15.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529225+0000", 2024-09-17T21:30:15.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047325117999999999 2024-09-17T21:30:15.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:15.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529292+0000", 2024-09-17T21:30:15.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7100000000000005e-05 2024-09-17T21:30:15.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:15.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529775+0000", 2024-09-17T21:30:15.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048322799999999999 2024-09-17T21:30:15.986 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:15.986 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:15.986 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:15.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.986 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50830 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:15.986 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:15.986 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.4624572110000003, 2024-09-17T21:30:15.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048062123999999998, 2024-09-17T21:30:15.986 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:15.986 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:15.986 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:15.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:15.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:15.987 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.987 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50830 2024-09-17T21:30:15.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.987 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:15.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:15.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:15.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:15.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:15.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:15.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481907+0000", 2024-09-17T21:30:15.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:30:15.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:15.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481909+0000", 2024-09-17T21:30:15.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T21:30:15.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:15.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481915+0000", 2024-09-17T21:30:15.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.02e-06 2024-09-17T21:30:15.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481920+0000", 2024-09-17T21:30:15.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6799999999999998e-06 2024-09-17T21:30:15.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:15.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529793+0000", 2024-09-17T21:30:15.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047873042999999997 2024-09-17T21:30:15.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:15.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529870+0000", 2024-09-17T21:30:15.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6967999999999999e-05 2024-09-17T21:30:15.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:15.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529968+0000", 2024-09-17T21:30:15.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8085999999999996e-05 2024-09-17T21:30:15.990 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:15.990 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:15.990 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:15.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.991 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50831 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 18087936~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:15.991 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:15.991 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.4624368649999999, 2024-09-17T21:30:15.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048663304999999997, 2024-09-17T21:30:15.991 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:15.991 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:15.991 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:15.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:15.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:15.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.991 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50831 2024-09-17T21:30:15.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.992 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:15.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:15.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:15.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:15.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:15.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:15.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:15.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:30:15.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:15.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481929+0000", 2024-09-17T21:30:15.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2189999999999998e-06 2024-09-17T21:30:15.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:15.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481935+0000", 2024-09-17T21:30:15.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.383e-06 2024-09-17T21:30:15.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481939+0000", 2024-09-17T21:30:15.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2479999999999998e-06 2024-09-17T21:30:15.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:15.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530095+0000", 2024-09-17T21:30:15.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048156617999999998 2024-09-17T21:30:15.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:15.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530159+0000", 2024-09-17T21:30:15.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3394999999999999e-05 2024-09-17T21:30:15.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:15.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530590+0000", 2024-09-17T21:30:15.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043101199999999999 2024-09-17T21:30:15.995 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:15.995 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:15.995 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:15.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.995 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50832 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 19660800~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:15.995 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:15.995 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.4624182179999998, 2024-09-17T21:30:15.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049339466999999998, 2024-09-17T21:30:15.996 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:15.996 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:15.996 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:15.996 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:15.996 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:15.996 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:15.996 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50832 2024-09-17T21:30:15.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.996 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:15.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:15.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:15.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:15.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:15.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:15.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:15.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481946+0000", 2024-09-17T21:30:15.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:15.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:15.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481951+0000", 2024-09-17T21:30:15.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1479999999999997e-06 2024-09-17T21:30:15.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:15.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481956+0000", 2024-09-17T21:30:15.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7339999999999996e-06 2024-09-17T21:30:15.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:15.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481961+0000", 2024-09-17T21:30:15.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4889999999999997e-06 2024-09-17T21:30:15.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:15.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530630+0000", 2024-09-17T21:30:15.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048668596000000001 2024-09-17T21:30:15.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:15.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:15.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530720+0000", 2024-09-17T21:30:15.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0163999999999999e-05 2024-09-17T21:30:15.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:15.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531285+0000", 2024-09-17T21:30:16.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056493199999999998 2024-09-17T21:30:16.000 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.000 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.000 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.000 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50833 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:16.000 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:16.000 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.4623960739999999, 2024-09-17T21:30:16.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049908069999999999, 2024-09-17T21:30:16.001 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.001 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.001 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:16.001 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:16.001 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:16.001 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:16.001 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50833 2024-09-17T21:30:16.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.001 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:16.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:16.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481968+0000", 2024-09-17T21:30:16.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5600000000000001e-07 2024-09-17T21:30:16.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481970+0000", 2024-09-17T21:30:16.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:30:16.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481975+0000", 2024-09-17T21:30:16.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8499999999999999e-06 2024-09-17T21:30:16.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481980+0000", 2024-09-17T21:30:16.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-17T21:30:16.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531337+0000", 2024-09-17T21:30:16.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049356874000000002 2024-09-17T21:30:16.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531392+0000", 2024-09-17T21:30:16.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4826e-05 2024-09-17T21:30:16.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531875+0000", 2024-09-17T21:30:16.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048345099999999998 2024-09-17T21:30:16.004 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.005 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.005 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.005 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50834 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 24903680~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:16.005 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:16.005 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.4623713059999996, 2024-09-17T21:30:16.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050534015000000002, 2024-09-17T21:30:16.005 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.005 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.005 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:16.005 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:16.005 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:16.005 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:16.006 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50834 2024-09-17T21:30:16.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.006 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:16.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:16.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481993+0000", 2024-09-17T21:30:16.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:30:16.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481995+0000", 2024-09-17T21:30:16.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.199e-06 2024-09-17T21:30:16.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482001+0000", 2024-09-17T21:30:16.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.772e-06 2024-09-17T21:30:16.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482006+0000", 2024-09-17T21:30:16.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0660000000000003e-06 2024-09-17T21:30:16.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531923+0000", 2024-09-17T21:30:16.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049917200000000002 2024-09-17T21:30:16.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532004+0000", 2024-09-17T21:30:16.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0758000000000002e-05 2024-09-17T21:30:16.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532526+0000", 2024-09-17T21:30:16.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00052258899999999995 2024-09-17T21:30:16.009 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.009 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.009 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.009 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50835 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 28573696~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:16.009 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:16.010 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.462351323, 2024-09-17T21:30:16.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051204701999999998, 2024-09-17T21:30:16.010 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.010 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.010 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:16.010 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:16.010 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:16.010 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:16.010 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50835 2024-09-17T21:30:16.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.010 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:16.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:16.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482013+0000", 2024-09-17T21:30:16.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:30:16.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482015+0000", 2024-09-17T21:30:16.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.272e-06 2024-09-17T21:30:16.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482020+0000", 2024-09-17T21:30:16.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3040000000000004e-06 2024-09-17T21:30:16.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482025+0000", 2024-09-17T21:30:16.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5650000000000003e-06 2024-09-17T21:30:16.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532569+0000", 2024-09-17T21:30:16.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050544661999999997 2024-09-17T21:30:16.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532649+0000", 2024-09-17T21:30:16.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9640000000000003e-05 2024-09-17T21:30:16.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533217+0000", 2024-09-17T21:30:16.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056784100000000003 2024-09-17T21:30:16.014 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.014 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.014 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.014 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50836 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 30146560~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:16.014 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:16.014 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.4623317059999996, 2024-09-17T21:30:16.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051875589, 2024-09-17T21:30:16.014 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.014 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.014 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:16.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:16.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:16.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:16.015 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50836 2024-09-17T21:30:16.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.015 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:16.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:16.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:16.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3700000000000001e-07 2024-09-17T21:30:16.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482034+0000", 2024-09-17T21:30:16.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0059999999999999e-06 2024-09-17T21:30:16.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482040+0000", 2024-09-17T21:30:16.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:30:16.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482044+0000", 2024-09-17T21:30:16.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:30:16.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533260+0000", 2024-09-17T21:30:16.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051215416 2024-09-17T21:30:16.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533348+0000", 2024-09-17T21:30:16.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8695000000000004e-05 2024-09-17T21:30:16.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533907+0000", 2024-09-17T21:30:16.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055902199999999995 2024-09-17T21:30:16.018 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.018 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.018 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.019 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50844 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:16.019 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:16.019 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.4621682800000002, 2024-09-17T21:30:16.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0475644, 2024-09-17T21:30:16.019 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.019 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.019 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:16.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:16.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:16.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:16.019 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50844 2024-09-17T21:30:16.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.019 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:16.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:16.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482196+0000", 2024-09-17T21:30:16.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.39e-07 2024-09-17T21:30:16.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482198+0000", 2024-09-17T21:30:16.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0099999999999998e-06 2024-09-17T21:30:16.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482204+0000", 2024-09-17T21:30:16.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.93e-06 2024-09-17T21:30:16.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482208+0000", 2024-09-17T21:30:16.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7439999999999998e-06 2024-09-17T21:30:16.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529318+0000", 2024-09-17T21:30:16.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047110051999999999 2024-09-17T21:30:16.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529396+0000", 2024-09-17T21:30:16.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7712000000000004e-05 2024-09-17T21:30:16.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529760+0000", 2024-09-17T21:30:16.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00036351299999999999 2024-09-17T21:30:16.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.023 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.023 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50845 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 21757952~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:16.023 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:16.024 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.4621491769999997, 2024-09-17T21:30:16.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048084625999999998, 2024-09-17T21:30:16.024 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.024 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.024 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:16.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:16.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:16.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:16.024 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50845 2024-09-17T21:30:16.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.024 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:16.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:16.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482215+0000", 2024-09-17T21:30:16.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:30:16.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482216+0000", 2024-09-17T21:30:16.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.823e-06 2024-09-17T21:30:16.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482222+0000", 2024-09-17T21:30:16.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.507e-06 2024-09-17T21:30:16.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482227+0000", 2024-09-17T21:30:16.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6609999999999999e-06 2024-09-17T21:30:16.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529792+0000", 2024-09-17T21:30:16.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047565075999999998 2024-09-17T21:30:16.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529871+0000", 2024-09-17T21:30:16.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8855000000000003e-05 2024-09-17T21:30:16.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530299+0000", 2024-09-17T21:30:16.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000428307 2024-09-17T21:30:16.028 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.028 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.028 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.028 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50846 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 25427968~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:16.028 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:16.028 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.4621267900000001, 2024-09-17T21:30:16.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048735142000000002, 2024-09-17T21:30:16.028 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.029 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.029 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:16.029 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:16.029 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:16.029 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:16.029 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50846 2024-09-17T21:30:16.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.029 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:16.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:16.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:16.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:30:16.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482239+0000", 2024-09-17T21:30:16.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.871e-06 2024-09-17T21:30:16.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482244+0000", 2024-09-17T21:30:16.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.49e-06 2024-09-17T21:30:16.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482249+0000", 2024-09-17T21:30:16.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6269999999999999e-06 2024-09-17T21:30:16.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530347+0000", 2024-09-17T21:30:16.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048097721000000003 2024-09-17T21:30:16.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530437+0000", 2024-09-17T21:30:16.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0604000000000002e-05 2024-09-17T21:30:16.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530972+0000", 2024-09-17T21:30:16.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053445000000000001 2024-09-17T21:30:16.032 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.033 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.033 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.033 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50847 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 27000832~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:16.033 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:16.033 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.462108894, 2024-09-17T21:30:16.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049308668, 2024-09-17T21:30:16.033 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.033 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.033 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:16.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:16.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:16.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:16.034 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50847 2024-09-17T21:30:16.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.034 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:16.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:16.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:16.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:30:16.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482257+0000", 2024-09-17T21:30:16.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.846e-06 2024-09-17T21:30:16.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482264+0000", 2024-09-17T21:30:16.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8170000000000001e-06 2024-09-17T21:30:16.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482283+0000", 2024-09-17T21:30:16.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9718e-05 2024-09-17T21:30:16.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531018+0000", 2024-09-17T21:30:16.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048734497000000002 2024-09-17T21:30:16.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531080+0000", 2024-09-17T21:30:16.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1795e-05 2024-09-17T21:30:16.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531563+0000", 2024-09-17T21:30:16.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048366499999999998 2024-09-17T21:30:16.037 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.037 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.037 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.038 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50848 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 30670848~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:16.038 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:16.038 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.4620723519999999, 2024-09-17T21:30:16.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049893337000000003, 2024-09-17T21:30:16.038 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.038 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.038 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:16.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:16.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:16.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:16.038 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50848 2024-09-17T21:30:16.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.038 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:16.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:16.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482292+0000", 2024-09-17T21:30:16.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T21:30:16.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482294+0000", 2024-09-17T21:30:16.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3209999999999999e-06 2024-09-17T21:30:16.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482300+0000", 2024-09-17T21:30:16.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7189999999999998e-06 2024-09-17T21:30:16.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482305+0000", 2024-09-17T21:30:16.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T21:30:16.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531606+0000", 2024-09-17T21:30:16.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049301273999999999 2024-09-17T21:30:16.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531696+0000", 2024-09-17T21:30:16.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0161999999999995e-05 2024-09-17T21:30:16.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532184+0000", 2024-09-17T21:30:16.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048806400000000002 2024-09-17T21:30:16.042 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.042 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.042 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.042 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50849 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 32243712~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:16.042 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:16.042 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.4620529109999998, 2024-09-17T21:30:16.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050444607000000002, 2024-09-17T21:30:16.042 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.042 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.043 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:16.043 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:16.043 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:16.043 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:16.043 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50849 2024-09-17T21:30:16.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.043 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:16.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:16.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482311+0000", 2024-09-17T21:30:16.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:30:16.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482316+0000", 2024-09-17T21:30:16.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1209999999999998e-06 2024-09-17T21:30:16.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482321+0000", 2024-09-17T21:30:16.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4779999999999999e-06 2024-09-17T21:30:16.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482326+0000", 2024-09-17T21:30:16.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7099999999999998e-06 2024-09-17T21:30:16.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532221+0000", 2024-09-17T21:30:16.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049894936000000001 2024-09-17T21:30:16.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532287+0000", 2024-09-17T21:30:16.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5914000000000006e-05 2024-09-17T21:30:16.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532755+0000", 2024-09-17T21:30:16.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000468107 2024-09-17T21:30:16.046 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.046 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.046 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.046 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.047 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.047 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:16.048 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:30:16.049 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:16.046+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:16.149 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:16.146+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:16.151 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:16.151 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:16.151 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:16.151 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:16.159 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:30:16.250 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:16.248+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:16.301 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:16.301 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:16.301 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:16.302 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:16.302 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:16.310 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:30:16.351 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:16.349+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:16.452 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:16.449+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:16.455 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:16.455 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:16.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:16.455 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:16.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.455 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:39715 3.0 e42/25 3:050aa3f8:::rbd_data.1120ab27bfd0.0000000000000001:head v 42'21, mlcod=41'18)", 2024-09-17T21:30:16.455 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:03.634569+0000", 2024-09-17T21:30:16.455 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 12.817667096999999, 2024-09-17T21:30:16.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0044088290000000004, 2024-09-17T21:30:16.455 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.456 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.456 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:16.456 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.634569+0000", 2024-09-17T21:30:16.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.634569+0000", 2024-09-17T21:30:16.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.634571+0000", 2024-09-17T21:30:16.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5319999999999999e-06 2024-09-17T21:30:16.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.634583+0000", 2024-09-17T21:30:16.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.261e-05 2024-09-17T21:30:16.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.634603+0000", 2024-09-17T21:30:16.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9828999999999999e-05 2024-09-17T21:30:16.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.634613+0000", 2024-09-17T21:30:16.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5389999999999996e-06 2024-09-17T21:30:16.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.636599+0000", 2024-09-17T21:30:16.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0019867890000000001 2024-09-17T21:30:16.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.636621+0000", 2024-09-17T21:30:16.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2110999999999999e-05 2024-09-17T21:30:16.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:16.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.638927+0000", 2024-09-17T21:30:16.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00230532 2024-09-17T21:30:16.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:03.638978+0000", 2024-09-17T21:30:16.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1099e-05 2024-09-17T21:30:16.460 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.460 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.460 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.460 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:45801 3.4 e45/25 3:2431819f:::rbd_data.1120b21c3309.0000000000000000:head v 45'118, mlcod=44'116)", 2024-09-17T21:30:16.460 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.735946+0000", 2024-09-17T21:30:16.460 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.7162904399999999, 2024-09-17T21:30:16.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0054180599999999997, 2024-09-17T21:30:16.460 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.460 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.460 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:16.460 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.735946+0000", 2024-09-17T21:30:16.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.735946+0000", 2024-09-17T21:30:16.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.735947+0000", 2024-09-17T21:30:16.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.477e-06 2024-09-17T21:30:16.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.736212+0000", 2024-09-17T21:30:16.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026511299999999998 2024-09-17T21:30:16.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.736235+0000", 2024-09-17T21:30:16.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2742e-05 2024-09-17T21:30:16.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.736245+0000", 2024-09-17T21:30:16.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0172e-05 2024-09-17T21:30:16.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.736328+0000", 2024-09-17T21:30:16.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2898000000000005e-05 2024-09-17T21:30:16.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.736349+0000", 2024-09-17T21:30:16.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1046000000000001e-05 2024-09-17T21:30:16.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:16.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.741303+0000", 2024-09-17T21:30:16.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0049536980000000003 2024-09-17T21:30:16.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.741364+0000", 2024-09-17T21:30:16.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0914e-05 2024-09-17T21:30:16.464 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.464 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.465 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:45807 3.9 e45/25 3:9731cab8:::rbd_data.1120b21c3309.0000000000000005:head v 45'23, mlcod=44'21)", 2024-09-17T21:30:16.465 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.742709+0000", 2024-09-17T21:30:16.465 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.709526791, 2024-09-17T21:30:16.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039918530000000001, 2024-09-17T21:30:16.465 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.465 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.465 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:16.465 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.742709+0000", 2024-09-17T21:30:16.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.742709+0000", 2024-09-17T21:30:16.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.742710+0000", 2024-09-17T21:30:16.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0809999999999999e-06 2024-09-17T21:30:16.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.743492+0000", 2024-09-17T21:30:16.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00078200199999999996 2024-09-17T21:30:16.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.743512+0000", 2024-09-17T21:30:16.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9602999999999998e-05 2024-09-17T21:30:16.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.743521+0000", 2024-09-17T21:30:16.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2250000000000006e-06 2024-09-17T21:30:16.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.743596+0000", 2024-09-17T21:30:16.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4238000000000003e-05 2024-09-17T21:30:16.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.743616+0000", 2024-09-17T21:30:16.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0446e-05 2024-09-17T21:30:16.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:16.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.746660+0000", 2024-09-17T21:30:16.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0030443219999999999 2024-09-17T21:30:16.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.746701+0000", 2024-09-17T21:30:16.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0936000000000002e-05 2024-09-17T21:30:16.469 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.469 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.469 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.469 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:47424 3.4 e45/25 3:23621007:::rbd_data.1120b21c3309.000000000000001f:head v 45'126, mlcod=45'123)", 2024-09-17T21:30:16.469 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.835534+0000", 2024-09-17T21:30:16.469 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6167019499999995, 2024-09-17T21:30:16.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038193630000000001, 2024-09-17T21:30:16.470 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.470 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.470 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:16.470 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.835534+0000", 2024-09-17T21:30:16.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.835534+0000", 2024-09-17T21:30:16.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.835536+0000", 2024-09-17T21:30:16.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3009999999999999e-06 2024-09-17T21:30:16.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.835567+0000", 2024-09-17T21:30:16.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1130999999999997e-05 2024-09-17T21:30:16.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.835582+0000", 2024-09-17T21:30:16.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5566e-05 2024-09-17T21:30:16.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.835590+0000", 2024-09-17T21:30:16.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0150000000000003e-06 2024-09-17T21:30:16.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.835675+0000", 2024-09-17T21:30:16.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4868000000000001e-05 2024-09-17T21:30:16.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.835694+0000", 2024-09-17T21:30:16.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9154999999999999e-05 2024-09-17T21:30:16.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:16.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.839306+0000", 2024-09-17T21:30:16.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003611923 2024-09-17T21:30:16.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.839354+0000", 2024-09-17T21:30:16.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7404000000000002e-05 2024-09-17T21:30:16.473 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.474 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.474 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.474 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48021 3.5 e45/37 3:a6441874:::rbd_data.1120b21c3309.0000000000000039:head v 45'62, mlcod=44'60)", 2024-09-17T21:30:16.474 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.876760+0000", 2024-09-17T21:30:16.474 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5754760020000003, 2024-09-17T21:30:16.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045026060000000001, 2024-09-17T21:30:16.474 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.474 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.474 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:16.474 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.876760+0000", 2024-09-17T21:30:16.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.876760+0000", 2024-09-17T21:30:16.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.876761+0000", 2024-09-17T21:30:16.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.85e-07 2024-09-17T21:30:16.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.876798+0000", 2024-09-17T21:30:16.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7366000000000001e-05 2024-09-17T21:30:16.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.876812+0000", 2024-09-17T21:30:16.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.366e-05 2024-09-17T21:30:16.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.876819+0000", 2024-09-17T21:30:16.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3669999999999999e-06 2024-09-17T21:30:16.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.876873+0000", 2024-09-17T21:30:16.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3650000000000003e-05 2024-09-17T21:30:16.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.876893+0000", 2024-09-17T21:30:16.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0349000000000001e-05 2024-09-17T21:30:16.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:16.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.881218+0000", 2024-09-17T21:30:16.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043242339999999997 2024-09-17T21:30:16.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.881263+0000", 2024-09-17T21:30:16.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5095000000000001e-05 2024-09-17T21:30:16.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.478 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.478 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48022 3.5 e45/37 3:a6441874:::rbd_data.1120b21c3309.0000000000000039:head v 45'64, mlcod=44'60)", 2024-09-17T21:30:16.478 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.877803+0000", 2024-09-17T21:30:16.478 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.574432882, 2024-09-17T21:30:16.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063935559999999999, 2024-09-17T21:30:16.479 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.479 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.479 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:16.479 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.877803+0000", 2024-09-17T21:30:16.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.877803+0000", 2024-09-17T21:30:16.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.877804+0000", 2024-09-17T21:30:16.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3399999999999998e-07 2024-09-17T21:30:16.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.877810+0000", 2024-09-17T21:30:16.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6790000000000002e-06 2024-09-17T21:30:16.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.877821+0000", 2024-09-17T21:30:16.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.113e-05 2024-09-17T21:30:16.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.877827+0000", 2024-09-17T21:30:16.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3589999999999996e-06 2024-09-17T21:30:16.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.877877+0000", 2024-09-17T21:30:16.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9985000000000003e-05 2024-09-17T21:30:16.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.877898+0000", 2024-09-17T21:30:16.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0523e-05 2024-09-17T21:30:16.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:16.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.884142+0000", 2024-09-17T21:30:16.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0062445039999999997 2024-09-17T21:30:16.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.884197+0000", 2024-09-17T21:30:16.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4542000000000001e-05 2024-09-17T21:30:16.482 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.482 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.482 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.483 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48295 3.9 e45/25 3:97228dae:::rbd_data.1120b21c3309.000000000000004f:head v 45'31, mlcod=45'29)", 2024-09-17T21:30:16.483 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.901458+0000", 2024-09-17T21:30:16.483 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.550778158, 2024-09-17T21:30:16.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038023079999999999, 2024-09-17T21:30:16.483 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.483 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.483 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:16.483 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.901458+0000", 2024-09-17T21:30:16.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.901458+0000", 2024-09-17T21:30:16.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.901459+0000", 2024-09-17T21:30:16.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0330000000000001e-06 2024-09-17T21:30:16.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.901466+0000", 2024-09-17T21:30:16.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2370000000000003e-06 2024-09-17T21:30:16.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.901482+0000", 2024-09-17T21:30:16.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5264000000000002e-05 2024-09-17T21:30:16.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.901489+0000", 2024-09-17T21:30:16.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0450000000000003e-06 2024-09-17T21:30:16.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.901555+0000", 2024-09-17T21:30:16.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6093000000000006e-05 2024-09-17T21:30:16.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.901574+0000", 2024-09-17T21:30:16.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9281999999999999e-05 2024-09-17T21:30:16.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:16.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.905205+0000", 2024-09-17T21:30:16.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036306300000000001 2024-09-17T21:30:16.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.905260+0000", 2024-09-17T21:30:16.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5723999999999997e-05 2024-09-17T21:30:16.487 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.487 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.487 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.487 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48304 3.0 e45/25 3:07d2aa03:::rbd_data.1120b21c3309.0000000000000056:head v 45'33, mlcod=43'31)", 2024-09-17T21:30:16.487 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.904321+0000", 2024-09-17T21:30:16.488 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5479156340000007, 2024-09-17T21:30:16.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0041472260000000004, 2024-09-17T21:30:16.488 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.488 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.488 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:16.488 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904321+0000", 2024-09-17T21:30:16.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904321+0000", 2024-09-17T21:30:16.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904321+0000", 2024-09-17T21:30:16.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8899999999999998e-07 2024-09-17T21:30:16.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904327+0000", 2024-09-17T21:30:16.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4380000000000003e-06 2024-09-17T21:30:16.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904338+0000", 2024-09-17T21:30:16.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1131999999999999e-05 2024-09-17T21:30:16.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904345+0000", 2024-09-17T21:30:16.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7089999999999996e-06 2024-09-17T21:30:16.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904399+0000", 2024-09-17T21:30:16.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4573000000000001e-05 2024-09-17T21:30:16.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904417+0000", 2024-09-17T21:30:16.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8159999999999999e-05 2024-09-17T21:30:16.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:16.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.908414+0000", 2024-09-17T21:30:16.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039968240000000004 2024-09-17T21:30:16.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.908468+0000", 2024-09-17T21:30:16.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3600999999999997e-05 2024-09-17T21:30:16.492 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.492 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.492 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.492 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "pg_update_log_missing(3.4 epoch 45/25 rep_tid 251 entries 45'135 (0'0) error 3:22ae8e31:::rbd_data.1120b21c3309.000000000000005a:head by client.4384.0:48303 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 45'133)", 2024-09-17T21:30:16.492 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.904364+0000", 2024-09-17T21:30:16.492 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5478722319999996, 2024-09-17T21:30:16.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004087696, 2024-09-17T21:30:16.493 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.493 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.493 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T21:30:16.493 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904364+0000", 2024-09-17T21:30:16.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904364+0000", 2024-09-17T21:30:16.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904364+0000", 2024-09-17T21:30:16.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0100000000000001e-07 2024-09-17T21:30:16.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904370+0000", 2024-09-17T21:30:16.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.19e-06 2024-09-17T21:30:16.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904385+0000", 2024-09-17T21:30:16.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4827999999999999e-05 2024-09-17T21:30:16.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904399+0000", 2024-09-17T21:30:16.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3549e-05 2024-09-17T21:30:16.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.904499+0000", 2024-09-17T21:30:16.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9815999999999994e-05 2024-09-17T21:30:16.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.908452+0000", 2024-09-17T21:30:16.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039530119999999997 2024-09-17T21:30:16.496 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.496 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.496 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.496 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48462 3.4 e45/25 3:24fc1191:::rbd_data.1120b21c3309.00000000000000a9:head v 45'142, mlcod=45'139)", 2024-09-17T21:30:16.496 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:06.964224+0000", 2024-09-17T21:30:16.496 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4880124289999994, 2024-09-17T21:30:16.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043470440000000004, 2024-09-17T21:30:16.496 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.496 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.497 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:16.497 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.964224+0000", 2024-09-17T21:30:16.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.964224+0000", 2024-09-17T21:30:16.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.964224+0000", 2024-09-17T21:30:16.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4300000000000002e-07 2024-09-17T21:30:16.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.964232+0000", 2024-09-17T21:30:16.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5669999999999997e-06 2024-09-17T21:30:16.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.964243+0000", 2024-09-17T21:30:16.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0981e-05 2024-09-17T21:30:16.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.964249+0000", 2024-09-17T21:30:16.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2079999999999997e-06 2024-09-17T21:30:16.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.964319+0000", 2024-09-17T21:30:16.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0152000000000002e-05 2024-09-17T21:30:16.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.964338+0000", 2024-09-17T21:30:16.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8448999999999999e-05 2024-09-17T21:30:16.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:16.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.968525+0000", 2024-09-17T21:30:16.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0041876220000000002 2024-09-17T21:30:16.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:06.968571+0000", 2024-09-17T21:30:16.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5321999999999997e-05 2024-09-17T21:30:16.500 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.501 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.501 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.501 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "pg_update_log_missing(3.5 epoch 45/37 rep_tid 844 entries 45'73 (0'0) error 3:a65c40ae:::rbd_data.1120b21c3309.0000000000000126:head by client.4384.0:48900 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 45'70)", 2024-09-17T21:30:16.501 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:16.501 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4051246759999998, 2024-09-17T21:30:16.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019554365000000001, 2024-09-17T21:30:16.501 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.501 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.501 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T21:30:16.501 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:16.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:16.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047112+0000", 2024-09-17T21:30:16.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:30:16.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047121+0000", 2024-09-17T21:30:16.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4379999999999995e-06 2024-09-17T21:30:16.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047139+0000", 2024-09-17T21:30:16.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8235000000000001e-05 2024-09-17T21:30:16.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047303+0000", 2024-09-17T21:30:16.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016389400000000001 2024-09-17T21:30:16.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047396+0000", 2024-09-17T21:30:16.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3355000000000003e-05 2024-09-17T21:30:16.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066666+0000", 2024-09-17T21:30:16.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019269649999999999 2024-09-17T21:30:16.504 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.504 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.504 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.505 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48906 3.4 e45/25 3:2503fe27:::rbd_data.1120b21c3309.0000000000000127:head v 45'148, mlcod=45'146)", 2024-09-17T21:30:16.505 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:16.505 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4029984970000005, 2024-09-17T21:30:16.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018169739000000001, 2024-09-17T21:30:16.505 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.505 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.505 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:16.505 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:16.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:16.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:16.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:30:16.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049244+0000", 2024-09-17T21:30:16.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8390000000000003e-06 2024-09-17T21:30:16.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066411+0000", 2024-09-17T21:30:16.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017166818 2024-09-17T21:30:16.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066426+0000", 2024-09-17T21:30:16.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4888999999999999e-05 2024-09-17T21:30:16.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066611+0000", 2024-09-17T21:30:16.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018464499999999999 2024-09-17T21:30:16.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066630+0000", 2024-09-17T21:30:16.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9201000000000001e-05 2024-09-17T21:30:16.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:16.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067373+0000", 2024-09-17T21:30:16.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00074322600000000004 2024-09-17T21:30:16.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067407+0000", 2024-09-17T21:30:16.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4357999999999999e-05 2024-09-17T21:30:16.509 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.509 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.509 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.509 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "pg_update_log_missing(3.1f epoch 45/37 rep_tid 1162 entries 45'293 (0'0) error 3:fb571695:::rbd_data.1120b21c3309.000000000000012d:head by client.4384.0:48915 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 45'291)", 2024-09-17T21:30:16.509 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.051791+0000", 2024-09-17T21:30:16.510 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.400445457, 2024-09-17T21:30:16.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016121626, 2024-09-17T21:30:16.510 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.510 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.510 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T21:30:16.510 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.051791+0000", 2024-09-17T21:30:16.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.051791+0000", 2024-09-17T21:30:16.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.051792+0000", 2024-09-17T21:30:16.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1500000000000003e-07 2024-09-17T21:30:16.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.051801+0000", 2024-09-17T21:30:16.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4469999999999995e-06 2024-09-17T21:30:16.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066420+0000", 2024-09-17T21:30:16.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.014619228 2024-09-17T21:30:16.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066442+0000", 2024-09-17T21:30:16.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1566999999999999e-05 2024-09-17T21:30:16.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066851+0000", 2024-09-17T21:30:16.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00040967599999999999 2024-09-17T21:30:16.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067912+0000", 2024-09-17T21:30:16.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001060893 2024-09-17T21:30:16.513 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.513 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.513 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.513 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:49780 3.4 3:24ae1c35:::rbd_data.1120d7424d8b.0000000000000039:e [read 36864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:16.513 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:10.896854+0000", 2024-09-17T21:30:16.513 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5553826649999998, 2024-09-17T21:30:16.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0037674459999999998, 2024-09-17T21:30:16.514 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.514 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.514 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:16.514 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:16.514 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:16.514 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:16.514 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 49780 2024-09-17T21:30:16.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.514 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.896854+0000", 2024-09-17T21:30:16.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.896854+0000", 2024-09-17T21:30:16.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.896854+0000", 2024-09-17T21:30:16.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7300000000000001e-07 2024-09-17T21:30:16.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.896856+0000", 2024-09-17T21:30:16.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-17T21:30:16.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.896863+0000", 2024-09-17T21:30:16.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.489e-06 2024-09-17T21:30:16.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.896868+0000", 2024-09-17T21:30:16.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:30:16.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.900401+0000", 2024-09-17T21:30:16.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0035336289999999999 2024-09-17T21:30:16.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.900510+0000", 2024-09-17T21:30:16.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000108584 2024-09-17T21:30:16.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.900621+0000", 2024-09-17T21:30:16.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011102799999999999 2024-09-17T21:30:16.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.518 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.518 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:49781 3.4 3:24ae1c35:::rbd_data.1120d7424d8b.0000000000000039:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:16.518 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:10.896876+0000", 2024-09-17T21:30:16.518 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5553605299999997, 2024-09-17T21:30:16.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039605899999999999, 2024-09-17T21:30:16.518 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.518 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.518 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:16.519 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:16.519 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:16.519 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:16.519 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 49781 2024-09-17T21:30:16.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.519 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.896876+0000", 2024-09-17T21:30:16.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.896876+0000", 2024-09-17T21:30:16.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.896876+0000", 2024-09-17T21:30:16.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7800000000000002e-07 2024-09-17T21:30:16.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.896878+0000", 2024-09-17T21:30:16.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.216e-06 2024-09-17T21:30:16.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.896885+0000", 2024-09-17T21:30:16.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2349999999999996e-06 2024-09-17T21:30:16.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.896889+0000", 2024-09-17T21:30:16.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6970000000000001e-06 2024-09-17T21:30:16.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.900649+0000", 2024-09-17T21:30:16.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003759994 2024-09-17T21:30:16.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.900726+0000", 2024-09-17T21:30:16.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6354999999999997e-05 2024-09-17T21:30:16.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.900836+0000", 2024-09-17T21:30:16.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000110615 2024-09-17T21:30:16.522 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.522 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.522 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.523 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:49942 3.4 3:23621007:::rbd_data.1120b21c3309.000000000000001f:f [sparse-read 0~61440] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:16.523 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:10.903101+0000", 2024-09-17T21:30:16.523 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5491356219999997, 2024-09-17T21:30:16.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042278960000000001, 2024-09-17T21:30:16.523 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.523 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.523 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:16.523 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:16.523 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:16.523 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:16.523 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 49942 2024-09-17T21:30:16.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.524 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.903101+0000", 2024-09-17T21:30:16.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.903101+0000", 2024-09-17T21:30:16.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.903101+0000", 2024-09-17T21:30:16.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T21:30:16.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.903103+0000", 2024-09-17T21:30:16.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.407e-06 2024-09-17T21:30:16.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.903109+0000", 2024-09-17T21:30:16.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3940000000000004e-06 2024-09-17T21:30:16.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.903113+0000", 2024-09-17T21:30:16.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5539999999999999e-06 2024-09-17T21:30:16.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.906585+0000", 2024-09-17T21:30:16.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0034718330000000001 2024-09-17T21:30:16.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.906670+0000", 2024-09-17T21:30:16.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4794000000000002e-05 2024-09-17T21:30:16.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.907328+0000", 2024-09-17T21:30:16.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00065855999999999998 2024-09-17T21:30:16.527 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.527 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.527 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.527 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:49943 3.4 3:23621007:::rbd_data.1120b21c3309.000000000000001f:f [sparse-read 212992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:16.527 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:10.903119+0000", 2024-09-17T21:30:16.527 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5491170170000004, 2024-09-17T21:30:16.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0044465340000000003, 2024-09-17T21:30:16.528 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.528 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.528 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:16.528 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:16.528 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:16.528 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:16.528 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 49943 2024-09-17T21:30:16.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.528 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.903119+0000", 2024-09-17T21:30:16.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.903119+0000", 2024-09-17T21:30:16.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.903119+0000", 2024-09-17T21:30:16.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.27e-07 2024-09-17T21:30:16.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.903122+0000", 2024-09-17T21:30:16.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1909999999999999e-06 2024-09-17T21:30:16.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.903127+0000", 2024-09-17T21:30:16.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4990000000000002e-06 2024-09-17T21:30:16.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.903131+0000", 2024-09-17T21:30:16.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999996e-06 2024-09-17T21:30:16.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.907363+0000", 2024-09-17T21:30:16.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004231182 2024-09-17T21:30:16.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.907444+0000", 2024-09-17T21:30:16.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1448999999999999e-05 2024-09-17T21:30:16.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.907566+0000", 2024-09-17T21:30:16.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012159600000000001 2024-09-17T21:30:16.532 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.532 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.532 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.532 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:49945 3.4 3:23621007:::rbd_data.1120b21c3309.000000000000001f:f [sparse-read 421888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:16.532 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:10.903138+0000", 2024-09-17T21:30:16.532 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5490984579999996, 2024-09-17T21:30:16.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0046558229999999999, 2024-09-17T21:30:16.532 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.532 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.532 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:16.533 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:16.533 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:16.533 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:16.533 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 49945 2024-09-17T21:30:16.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.533 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.903138+0000", 2024-09-17T21:30:16.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.903138+0000", 2024-09-17T21:30:16.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.903138+0000", 2024-09-17T21:30:16.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.27e-07 2024-09-17T21:30:16.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.903140+0000", 2024-09-17T21:30:16.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7519999999999999e-06 2024-09-17T21:30:16.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.903145+0000", 2024-09-17T21:30:16.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.48e-06 2024-09-17T21:30:16.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.903155+0000", 2024-09-17T21:30:16.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0036e-05 2024-09-17T21:30:16.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.907596+0000", 2024-09-17T21:30:16.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0044407509999999997 2024-09-17T21:30:16.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.907673+0000", 2024-09-17T21:30:16.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7355999999999996e-05 2024-09-17T21:30:16.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:10.907794+0000", 2024-09-17T21:30:16.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120121 2024-09-17T21:30:16.536 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.536 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.536 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.536 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50206 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'301, mlcod=45'299)", 2024-09-17T21:30:16.537 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:16.537 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.2735216529999995, 2024-09-17T21:30:16.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068285810000000002, 2024-09-17T21:30:16.537 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.537 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.537 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:16.537 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:16.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:16.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178716+0000", 2024-09-17T21:30:16.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.438e-06 2024-09-17T21:30:16.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192044+0000", 2024-09-17T21:30:16.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013328370000000001 2024-09-17T21:30:16.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192069+0000", 2024-09-17T21:30:16.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4496000000000001e-05 2024-09-17T21:30:16.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192085+0000", 2024-09-17T21:30:16.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5758999999999999e-05 2024-09-17T21:30:16.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192167+0000", 2024-09-17T21:30:16.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2842000000000004e-05 2024-09-17T21:30:16.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192201+0000", 2024-09-17T21:30:16.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3127999999999999e-05 2024-09-17T21:30:16.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:16.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246938+0000", 2024-09-17T21:30:16.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054736982000000003 2024-09-17T21:30:16.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247000+0000", 2024-09-17T21:30:16.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2794999999999998e-05 2024-09-17T21:30:16.541 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.541 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.541 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.541 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50207 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'303, mlcod=45'299)", 2024-09-17T21:30:16.541 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:16.541 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.2305746040000001, 2024-09-17T21:30:16.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026003433999999999, 2024-09-17T21:30:16.541 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:16.541 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:16.542 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:16.542 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:16.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:16.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:16.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:16.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:16.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:16.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:16.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221663+0000", 2024-09-17T21:30:16.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.271e-06 2024-09-17T21:30:16.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:16.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221672+0000", 2024-09-17T21:30:16.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6810000000000004e-06 2024-09-17T21:30:16.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:16.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221684+0000", 2024-09-17T21:30:16.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:30:16.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:16.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221691+0000", 2024-09-17T21:30:16.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:30:16.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:16.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221743+0000", 2024-09-17T21:30:16.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2868999999999998e-05 2024-09-17T21:30:16.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:16.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221765+0000", 2024-09-17T21:30:16.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1316999999999999e-05 2024-09-17T21:30:16.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:16.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247616+0000", 2024-09-17T21:30:16.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025851504000000001 2024-09-17T21:30:16.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:16.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:16.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:16.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247665+0000", 2024-09-17T21:30:16.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8739000000000002e-05 2024-09-17T21:30:16.545 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.545 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.545 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.545 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:16.545 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:16.546 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:16.547 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:30:16.553 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:16.551+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:16.654 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:16.652+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:16.657 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is set 2024-09-17T21:30:16.701 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:16.701 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:16.701 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:16.701 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:16.711 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:30:16.754 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:16.752+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:16.803 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-17T21:30:16.855 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:16.853+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:16.869 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:16.869 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:16.869 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:16.869 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:16.869 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:16.880 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:30:16.957 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:16.954+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:16.990 INFO:teuthology.orchestra.run.smithi129.stdout:2242 read 0xc10ab3 thru 0xc16aba (0x6008 bytes) 2024-09-17T21:30:16.991 INFO:teuthology.orchestra.run.smithi129.stdout:2243 punch from 0x1619d51 to 0x161c692, (0x2941 bytes) 2024-09-17T21:30:16.994 INFO:teuthology.orchestra.run.smithi129.stdout:2244 writesame 0x4015d74 thru 0x401a8a9 data_size 0x3(0x4b36 bytes) 2024-09-17T21:30:17.014 INFO:teuthology.orchestra.run.smithi129.stdout:2245 write 0xcd6cbd0 thru 0xcd72a76 (0x5ea7 bytes) 2024-09-17T21:30:17.018 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:17.018 INFO:teuthology.orchestra.run.smithi129.stdout:2247 write 0x5965a4f thru 0x596d3aa (0x795c bytes) 2024-09-17T21:30:17.022 INFO:teuthology.orchestra.run.smithi129.stdout:2249 punch from 0x5520721 to 0x552cad2, (0xc3b1 bytes) 2024-09-17T21:30:17.022 INFO:teuthology.orchestra.run.smithi129.stdout:2250 write 0x9c174d1 thru 0x9c1fc72 (0x87a2 bytes) 2024-09-17T21:30:17.025 INFO:teuthology.orchestra.run.smithi129.stdout:2251 read 0x670b1d1 thru 0x6710369 (0x5199 bytes) 2024-09-17T21:30:17.026 INFO:teuthology.orchestra.run.smithi129.stdout:2252 read 0x1e5c7c9 thru 0x1e6369a (0x6ed2 bytes) 2024-09-17T21:30:17.026 INFO:teuthology.orchestra.run.smithi129.stdout:2253 read 0x37cd0b8 thru 0x37cd36c (0x2b5 bytes) 2024-09-17T21:30:17.030 INFO:teuthology.orchestra.run.smithi129.stdout:2254 trunc from 0xcd72a77 to 0xa0a4ba8 2024-09-17T21:30:17.034 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:17.035 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:17.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:17.035 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:17.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.035 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:49361 3.19 e46/25 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:head v 46'135, mlcod=45'133)", 2024-09-17T21:30:17.035 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:17.035 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.2783009530000005, 2024-09-17T21:30:17.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043477744999999998, 2024-09-17T21:30:17.035 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.035 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.035 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:17.035 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:17.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:17.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:17.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-07 2024-09-17T21:30:17.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769703+0000", 2024-09-17T21:30:17.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016134381 2024-09-17T21:30:17.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769724+0000", 2024-09-17T21:30:17.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0781999999999999e-05 2024-09-17T21:30:17.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769734+0000", 2024-09-17T21:30:17.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6870000000000006e-06 2024-09-17T21:30:17.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769797+0000", 2024-09-17T21:30:17.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3418999999999997e-05 2024-09-17T21:30:17.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769822+0000", 2024-09-17T21:30:17.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4652999999999999e-05 2024-09-17T21:30:17.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:17.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.796982+0000", 2024-09-17T21:30:17.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027160205 2024-09-17T21:30:17.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.797047+0000", 2024-09-17T21:30:17.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4455999999999994e-05 2024-09-17T21:30:17.039 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.039 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.039 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.039 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50804 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 14155776~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.039 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495110+0000", 2024-09-17T21:30:17.040 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.536760149, 2024-09-17T21:30:17.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.034879440999999997, 2024-09-17T21:30:17.040 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.040 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.040 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.040 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.040 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.040 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.040 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50804 2024-09-17T21:30:17.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.040 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495110+0000", 2024-09-17T21:30:17.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495110+0000", 2024-09-17T21:30:17.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495110+0000", 2024-09-17T21:30:17.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2e-07 2024-09-17T21:30:17.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495112+0000", 2024-09-17T21:30:17.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7689999999999999e-06 2024-09-17T21:30:17.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495133+0000", 2024-09-17T21:30:17.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1319e-05 2024-09-17T21:30:17.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495138+0000", 2024-09-17T21:30:17.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:30:17.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529422+0000", 2024-09-17T21:30:17.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.034284199000000001 2024-09-17T21:30:17.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529492+0000", 2024-09-17T21:30:17.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9502e-05 2024-09-17T21:30:17.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529989+0000", 2024-09-17T21:30:17.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00049732900000000002 2024-09-17T21:30:17.044 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.044 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.044 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.044 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50805 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 17825792~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.044 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:30:17.044 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5367174590000001, 2024-09-17T21:30:17.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035591579999999998, 2024-09-17T21:30:17.044 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.044 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.044 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.044 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.044 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.045 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.045 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50805 2024-09-17T21:30:17.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.045 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:30:17.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:30:17.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495153+0000", 2024-09-17T21:30:17.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:30:17.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495155+0000", 2024-09-17T21:30:17.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1849999999999998e-06 2024-09-17T21:30:17.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495162+0000", 2024-09-17T21:30:17.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.64e-06 2024-09-17T21:30:17.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495166+0000", 2024-09-17T21:30:17.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:30:17.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530020+0000", 2024-09-17T21:30:17.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.034853824999999998 2024-09-17T21:30:17.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530076+0000", 2024-09-17T21:30:17.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5235999999999998e-05 2024-09-17T21:30:17.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530744+0000", 2024-09-17T21:30:17.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066835500000000003 2024-09-17T21:30:17.048 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.048 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.048 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.048 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50806 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19398656~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.048 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:17.048 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5366969910000003, 2024-09-17T21:30:17.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036408459999999997, 2024-09-17T21:30:17.049 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.049 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.049 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.049 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.049 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.049 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.049 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50806 2024-09-17T21:30:17.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.049 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:17.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:17.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:17.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.72e-07 2024-09-17T21:30:17.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495175+0000", 2024-09-17T21:30:17.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:30:17.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495182+0000", 2024-09-17T21:30:17.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8079999999999999e-06 2024-09-17T21:30:17.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495186+0000", 2024-09-17T21:30:17.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5549999999999996e-06 2024-09-17T21:30:17.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530791+0000", 2024-09-17T21:30:17.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035604963000000003 2024-09-17T21:30:17.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530855+0000", 2024-09-17T21:30:17.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3286000000000004e-05 2024-09-17T21:30:17.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531581+0000", 2024-09-17T21:30:17.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00072655699999999996 2024-09-17T21:30:17.052 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.053 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.053 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.053 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50807 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23068672~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.053 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:17.053 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5366781060000001, 2024-09-17T21:30:17.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037183314000000002, 2024-09-17T21:30:17.053 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.053 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.053 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.053 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.053 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.054 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.054 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50807 2024-09-17T21:30:17.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.054 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:17.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:17.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:17.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9999999999999999e-07 2024-09-17T21:30:17.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495194+0000", 2024-09-17T21:30:17.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.779e-06 2024-09-17T21:30:17.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495199+0000", 2024-09-17T21:30:17.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.468e-06 2024-09-17T21:30:17.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495203+0000", 2024-09-17T21:30:17.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0010000000000003e-06 2024-09-17T21:30:17.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531636+0000", 2024-09-17T21:30:17.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036432665000000003 2024-09-17T21:30:17.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531727+0000", 2024-09-17T21:30:17.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1607999999999994e-05 2024-09-17T21:30:17.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532375+0000", 2024-09-17T21:30:17.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00064749299999999996 2024-09-17T21:30:17.057 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.057 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.057 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.057 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50808 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24641536~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.057 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:30:17.058 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5366613630000003, 2024-09-17T21:30:17.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037870190999999997, 2024-09-17T21:30:17.058 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.058 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.058 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.058 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.058 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.058 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.058 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50808 2024-09-17T21:30:17.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.058 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:30:17.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:30:17.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495209+0000", 2024-09-17T21:30:17.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:30:17.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495215+0000", 2024-09-17T21:30:17.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1630000000000001e-06 2024-09-17T21:30:17.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495221+0000", 2024-09-17T21:30:17.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-17T21:30:17.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495226+0000", 2024-09-17T21:30:17.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8189999999999999e-06 2024-09-17T21:30:17.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532425+0000", 2024-09-17T21:30:17.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037199135000000001 2024-09-17T21:30:17.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532531+0000", 2024-09-17T21:30:17.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106314 2024-09-17T21:30:17.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533079+0000", 2024-09-17T21:30:17.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00054739700000000003 2024-09-17T21:30:17.061 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.062 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.062 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.062 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50809 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28311552~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.062 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:30:17.062 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5366383969999999, 2024-09-17T21:30:17.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038597605, 2024-09-17T21:30:17.062 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.062 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.062 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.063 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50809 2024-09-17T21:30:17.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.063 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:30:17.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:30:17.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495232+0000", 2024-09-17T21:30:17.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.39e-07 2024-09-17T21:30:17.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495234+0000", 2024-09-17T21:30:17.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T21:30:17.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495239+0000", 2024-09-17T21:30:17.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7969999999999997e-06 2024-09-17T21:30:17.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495244+0000", 2024-09-17T21:30:17.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999996e-06 2024-09-17T21:30:17.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533125+0000", 2024-09-17T21:30:17.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037881523 2024-09-17T21:30:17.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533237+0000", 2024-09-17T21:30:17.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000111879 2024-09-17T21:30:17.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533829+0000", 2024-09-17T21:30:17.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000591933 2024-09-17T21:30:17.066 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.066 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.066 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.067 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50810 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 29884416~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.067 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:30:17.067 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5366204789999998, 2024-09-17T21:30:17.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039149809000000001, 2024-09-17T21:30:17.067 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.067 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.067 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.067 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50810 2024-09-17T21:30:17.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.068 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:30:17.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:30:17.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495250+0000", 2024-09-17T21:30:17.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:30:17.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495251+0000", 2024-09-17T21:30:17.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.81e-06 2024-09-17T21:30:17.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495257+0000", 2024-09-17T21:30:17.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1399999999999999e-06 2024-09-17T21:30:17.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495261+0000", 2024-09-17T21:30:17.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4229999999999998e-06 2024-09-17T21:30:17.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533879+0000", 2024-09-17T21:30:17.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038618292999999998 2024-09-17T21:30:17.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533939+0000", 2024-09-17T21:30:17.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.012e-05 2024-09-17T21:30:17.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534399+0000", 2024-09-17T21:30:17.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00045968099999999998 2024-09-17T21:30:17.071 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.071 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.071 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.071 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50850 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 2359296~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.071 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:17.071 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.536602104, 2024-09-17T21:30:17.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039731491000000001, 2024-09-17T21:30:17.071 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.072 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.072 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.072 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50850 2024-09-17T21:30:17.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.072 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:17.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:17.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:17.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3200000000000001e-07 2024-09-17T21:30:17.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495270+0000", 2024-09-17T21:30:17.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.694e-06 2024-09-17T21:30:17.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495275+0000", 2024-09-17T21:30:17.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5890000000000002e-06 2024-09-17T21:30:17.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495279+0000", 2024-09-17T21:30:17.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-17T21:30:17.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534440+0000", 2024-09-17T21:30:17.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039160805999999999 2024-09-17T21:30:17.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534526+0000", 2024-09-17T21:30:17.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5327999999999993e-05 2024-09-17T21:30:17.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534999+0000", 2024-09-17T21:30:17.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00047353400000000001 2024-09-17T21:30:17.075 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.075 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.075 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.076 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50851 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 3932160~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.076 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:30:17.076 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5365844920000002, 2024-09-17T21:30:17.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040211991000000002, 2024-09-17T21:30:17.076 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.076 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.076 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.076 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50851 2024-09-17T21:30:17.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.077 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:30:17.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:30:17.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495286+0000", 2024-09-17T21:30:17.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:17.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495287+0000", 2024-09-17T21:30:17.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6640000000000001e-06 2024-09-17T21:30:17.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495293+0000", 2024-09-17T21:30:17.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6710000000000004e-06 2024-09-17T21:30:17.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495298+0000", 2024-09-17T21:30:17.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6700000000000002e-06 2024-09-17T21:30:17.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535017+0000", 2024-09-17T21:30:17.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039719059000000001 2024-09-17T21:30:17.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535067+0000", 2024-09-17T21:30:17.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0046000000000002e-05 2024-09-17T21:30:17.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535497+0000", 2024-09-17T21:30:17.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043050299999999999 2024-09-17T21:30:17.080 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.080 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.080 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.080 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50852 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 7602176~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.080 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:30:17.080 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.536565575, 2024-09-17T21:30:17.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041166337999999997, 2024-09-17T21:30:17.081 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.081 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.081 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.081 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.081 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.081 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.081 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50852 2024-09-17T21:30:17.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.081 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:30:17.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:30:17.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495305+0000", 2024-09-17T21:30:17.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1600000000000002e-07 2024-09-17T21:30:17.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495307+0000", 2024-09-17T21:30:17.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0980000000000001e-06 2024-09-17T21:30:17.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495312+0000", 2024-09-17T21:30:17.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6210000000000001e-06 2024-09-17T21:30:17.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495317+0000", 2024-09-17T21:30:17.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T21:30:17.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535531+0000", 2024-09-17T21:30:17.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040214580999999999 2024-09-17T21:30:17.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535618+0000", 2024-09-17T21:30:17.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6648000000000001e-05 2024-09-17T21:30:17.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536471+0000", 2024-09-17T21:30:17.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00085251599999999999 2024-09-17T21:30:17.084 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.084 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.085 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.085 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50853 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 9175040~204800] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.085 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:17.085 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5365472440000003, 2024-09-17T21:30:17.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041673496999999997, 2024-09-17T21:30:17.085 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.085 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.085 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.085 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.085 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.085 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.085 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50853 2024-09-17T21:30:17.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.086 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:17.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:17.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:17.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:30:17.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495334+0000", 2024-09-17T21:30:17.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.076e-05 2024-09-17T21:30:17.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495340+0000", 2024-09-17T21:30:17.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:30:17.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495345+0000", 2024-09-17T21:30:17.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:30:17.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536514+0000", 2024-09-17T21:30:17.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041168534999999999 2024-09-17T21:30:17.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536572+0000", 2024-09-17T21:30:17.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8742000000000001e-05 2024-09-17T21:30:17.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536996+0000", 2024-09-17T21:30:17.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042368899999999999 2024-09-17T21:30:17.089 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.089 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.089 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.089 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50854 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 12845056~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.090 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:17.090 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5365181850000003, 2024-09-17T21:30:17.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042777397000000002, 2024-09-17T21:30:17.090 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.090 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.090 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.090 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.090 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.090 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.090 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50854 2024-09-17T21:30:17.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.090 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:17.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:17.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:17.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.82e-07 2024-09-17T21:30:17.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495354+0000", 2024-09-17T21:30:17.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.091e-06 2024-09-17T21:30:17.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495360+0000", 2024-09-17T21:30:17.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.716e-06 2024-09-17T21:30:17.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495365+0000", 2024-09-17T21:30:17.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2939999999999996e-06 2024-09-17T21:30:17.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537020+0000", 2024-09-17T21:30:17.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041654419999999998 2024-09-17T21:30:17.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537105+0000", 2024-09-17T21:30:17.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5857000000000001e-05 2024-09-17T21:30:17.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538129+0000", 2024-09-17T21:30:17.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010235369999999999 2024-09-17T21:30:17.094 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.094 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.094 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.094 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50855 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 14417920~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.094 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:17.094 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5364982200000004, 2024-09-17T21:30:17.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043254325000000003, 2024-09-17T21:30:17.094 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.094 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.094 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.095 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.095 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.095 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.095 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50855 2024-09-17T21:30:17.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.095 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:17.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:17.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:17.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:30:17.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495374+0000", 2024-09-17T21:30:17.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.922e-06 2024-09-17T21:30:17.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495379+0000", 2024-09-17T21:30:17.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-17T21:30:17.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495384+0000", 2024-09-17T21:30:17.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-17T21:30:17.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538167+0000", 2024-09-17T21:30:17.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042782845 2024-09-17T21:30:17.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538218+0000", 2024-09-17T21:30:17.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1335000000000002e-05 2024-09-17T21:30:17.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538626+0000", 2024-09-17T21:30:17.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000407656 2024-09-17T21:30:17.098 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.098 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.098 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.099 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50856 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 18087936~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.099 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:17.099 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5364797650000002, 2024-09-17T21:30:17.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043911444000000001, 2024-09-17T21:30:17.099 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.099 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.099 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.099 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.099 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.099 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.099 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50856 2024-09-17T21:30:17.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.100 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:17.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:17.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:17.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0499999999999999e-07 2024-09-17T21:30:17.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495392+0000", 2024-09-17T21:30:17.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7430000000000001e-06 2024-09-17T21:30:17.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495398+0000", 2024-09-17T21:30:17.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3040000000000001e-06 2024-09-17T21:30:17.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495403+0000", 2024-09-17T21:30:17.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5600000000000004e-06 2024-09-17T21:30:17.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538660+0000", 2024-09-17T21:30:17.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043256832000000002 2024-09-17T21:30:17.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538746+0000", 2024-09-17T21:30:17.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6448999999999999e-05 2024-09-17T21:30:17.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539301+0000", 2024-09-17T21:30:17.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055525099999999999 2024-09-17T21:30:17.103 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.103 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.103 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.103 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50857 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19660800~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.103 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:30:17.103 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5364603599999995, 2024-09-17T21:30:17.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044408156999999997, 2024-09-17T21:30:17.103 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.104 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.104 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.104 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.104 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.104 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.104 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50857 2024-09-17T21:30:17.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.104 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:30:17.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:30:17.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495410+0000", 2024-09-17T21:30:17.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:30:17.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495412+0000", 2024-09-17T21:30:17.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9580000000000001e-06 2024-09-17T21:30:17.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495418+0000", 2024-09-17T21:30:17.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1909999999999997e-06 2024-09-17T21:30:17.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495423+0000", 2024-09-17T21:30:17.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2429999999999996e-06 2024-09-17T21:30:17.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539338+0000", 2024-09-17T21:30:17.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043914478 2024-09-17T21:30:17.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539393+0000", 2024-09-17T21:30:17.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5226999999999998e-05 2024-09-17T21:30:17.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539818+0000", 2024-09-17T21:30:17.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042469300000000002 2024-09-17T21:30:17.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.107 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.108 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50858 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23330816~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.108 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:17.108 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5364392630000001, 2024-09-17T21:30:17.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045010798999999997, 2024-09-17T21:30:17.108 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.108 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.108 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.108 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.108 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.108 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.108 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50858 2024-09-17T21:30:17.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.109 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:17.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:17.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:17.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:30:17.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495433+0000", 2024-09-17T21:30:17.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.787e-06 2024-09-17T21:30:17.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495438+0000", 2024-09-17T21:30:17.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:30:17.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495442+0000", 2024-09-17T21:30:17.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2640000000000002e-06 2024-09-17T21:30:17.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539852+0000", 2024-09-17T21:30:17.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044409592999999997 2024-09-17T21:30:17.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539940+0000", 2024-09-17T21:30:17.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7696999999999997e-05 2024-09-17T21:30:17.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540441+0000", 2024-09-17T21:30:17.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050164299999999997 2024-09-17T21:30:17.112 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.112 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.112 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.112 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50859 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24903680~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.112 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:30:17.112 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5364215850000003, 2024-09-17T21:30:17.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045571569999999999, 2024-09-17T21:30:17.113 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.113 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.113 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.113 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.113 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.113 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.113 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50859 2024-09-17T21:30:17.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.113 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:30:17.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:30:17.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495449+0000", 2024-09-17T21:30:17.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:30:17.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495450+0000", 2024-09-17T21:30:17.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7889999999999999e-06 2024-09-17T21:30:17.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495456+0000", 2024-09-17T21:30:17.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5679999999999999e-06 2024-09-17T21:30:17.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495460+0000", 2024-09-17T21:30:17.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4989999999999996e-06 2024-09-17T21:30:17.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540472+0000", 2024-09-17T21:30:17.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045011585999999999 2024-09-17T21:30:17.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540554+0000", 2024-09-17T21:30:17.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2443999999999999e-05 2024-09-17T21:30:17.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541020+0000", 2024-09-17T21:30:17.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00046534300000000001 2024-09-17T21:30:17.116 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.116 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.117 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.117 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50860 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28573696~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.117 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:17.117 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5364032319999996, 2024-09-17T21:30:17.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046167899999999998, 2024-09-17T21:30:17.117 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.117 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.117 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.117 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.117 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.117 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.117 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50860 2024-09-17T21:30:17.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.118 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:17.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:17.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:17.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:30:17.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495469+0000", 2024-09-17T21:30:17.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.99e-06 2024-09-17T21:30:17.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495475+0000", 2024-09-17T21:30:17.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:30:17.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495480+0000", 2024-09-17T21:30:17.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:30:17.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541050+0000", 2024-09-17T21:30:17.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045569853 2024-09-17T21:30:17.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541138+0000", 2024-09-17T21:30:17.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7997000000000005e-05 2024-09-17T21:30:17.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541634+0000", 2024-09-17T21:30:17.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00049685500000000004 2024-09-17T21:30:17.121 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.121 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.121 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.121 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50861 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 30146560~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:17.121 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:17.121 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.5363840550000001, 2024-09-17T21:30:17.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046690553000000003, 2024-09-17T21:30:17.122 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:17.122 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:17.122 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:17.122 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:17.122 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:17.122 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.122 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50861 2024-09-17T21:30:17.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.122 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:17.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:17.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:17.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:17.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:17.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:17.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:17.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:17.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:30:17.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:17.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495488+0000", 2024-09-17T21:30:17.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:30:17.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:17.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495495+0000", 2024-09-17T21:30:17.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:30:17.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495500+0000", 2024-09-17T21:30:17.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8620000000000002e-06 2024-09-17T21:30:17.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:17.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541687+0000", 2024-09-17T21:30:17.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046186434999999998 2024-09-17T21:30:17.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:17.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541741+0000", 2024-09-17T21:30:17.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4228999999999999e-05 2024-09-17T21:30:17.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:17.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:17.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:17.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.542176+0000", 2024-09-17T21:30:17.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043531100000000002 2024-09-17T21:30:17.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.125 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.126 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.126 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:17.126 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:17.126 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:17.127 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:17.128 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:30:17.129 INFO:teuthology.orchestra.run.smithi129.stdout:2256 writesame 0xe640817 thru 0xe64e82e data_size 0x47(0xe018 bytes) 2024-09-17T21:30:17.129 INFO:teuthology.orchestra.run.smithi129.stdout:2257 write 0x8679d06 thru 0x8686ae3 (0xcdde bytes) 2024-09-17T21:30:17.129 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:17.129 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:17.129 INFO:teuthology.orchestra.run.smithi129.stdout:2260 trunc from 0xe64e82f to 0x4a63120 2024-09-17T21:30:17.129 INFO:teuthology.orchestra.run.smithi129.stdout:2261 trunc from 0x4a63120 to 0xa897e17 2024-09-17T21:30:17.130 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:17.130 INFO:teuthology.orchestra.run.smithi129.stdout:2263 punch from 0x7b31176 to 0x7b35ae8, (0x4972 bytes) 2024-09-17T21:30:17.130 INFO:teuthology.orchestra.run.smithi129.stdout:2264 flatten 2024-09-17T21:30:17.130 INFO:teuthology.orchestra.run.smithi129.stdout:2265 read 0x1fa085c thru 0x1faabf9 (0xa39e bytes) 2024-09-17T21:30:17.130 INFO:teuthology.orchestra.run.smithi129.stdout:2266 read 0x711ce00 thru 0x7120bdb (0x3ddc bytes) 2024-09-17T21:30:17.130 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:17.130 INFO:teuthology.orchestra.run.smithi129.stdout:2270 punch from 0x3fce45c to 0x3fd8984, (0xa528 bytes) 2024-09-17T21:30:17.130 INFO:teuthology.orchestra.run.smithi129.stdout:2271 read 0x51836c7 thru 0x5191f1e (0xe858 bytes) 2024-09-17T21:30:17.130 INFO:teuthology.orchestra.run.smithi129.stdout:2272 read 0x50f9ae thru 0x5145f5 (0x4c48 bytes) 2024-09-17T21:30:17.130 INFO:teuthology.orchestra.run.smithi129.stdout:2274 read 0x96abb6 thru 0x975001 (0xa44c bytes) 2024-09-17T21:30:17.130 INFO:teuthology.orchestra.run.smithi129.stdout:2275 trunc from 0xa897e17 to 0x67642a 2024-09-17T21:30:17.131 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:17.125+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:17.136 INFO:teuthology.orchestra.run.smithi129.stdout:2277 read 0x610de thru 0x683d4 (0x72f7 bytes) 2024-09-17T21:30:17.137 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:17.152 INFO:teuthology.orchestra.run.smithi129.stdout:2279 trunc from 0x67642a to 0xd350c16 2024-09-17T21:30:17.157 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:17.157 INFO:teuthology.orchestra.run.smithi129.stdout:2283 read 0x2e3cb11 thru 0x2e419dd (0x4ecd bytes) 2024-09-17T21:30:17.157 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:17.157 INFO:teuthology.orchestra.run.smithi129.stdout:2287 write 0xca20419 thru 0xca20539 (0x121 bytes) 2024-09-17T21:30:17.160 INFO:teuthology.orchestra.run.smithi129.stdout:2288 writesame 0xc49aa1b thru 0xc4a1212 data_size 0xc(0x67f8 bytes) 2024-09-17T21:30:17.164 INFO:teuthology.orchestra.run.smithi129.stdout:2289 writesame 0xb873cb4 thru 0xb882721 data_size 0x2(0xea6e bytes) 2024-09-17T21:30:17.174 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:17.174 INFO:teuthology.orchestra.run.smithi129.stdout:2291 read 0x58fc54d thru 0x58fd1a4 (0xc58 bytes) 2024-09-17T21:30:17.174 INFO:teuthology.orchestra.run.smithi129.stdout:2292 write 0x2dacda thru 0x2e5318 (0xa63f bytes) 2024-09-17T21:30:17.176 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:17.176 INFO:teuthology.orchestra.run.smithi129.stdout:2296 read 0x28feed3 thru 0x28ffd62 (0xe90 bytes) 2024-09-17T21:30:17.176 INFO:teuthology.orchestra.run.smithi129.stdout:2297 writesame 0x4fb140a thru 0x4fb4794 data_size 0x23(0x338b bytes) 2024-09-17T21:30:17.179 INFO:teuthology.orchestra.run.smithi129.stdout:2298 trunc from 0xd350c16 to 0xdafe2a1 2024-09-17T21:30:17.184 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:17.191 INFO:teuthology.orchestra.run.smithi129.stdout:2300 write 0xec11653 thru 0xec1efa1 (0xd94f bytes) 2024-09-17T21:30:17.210 INFO:teuthology.orchestra.run.smithi129.stdout:2301 trunc from 0xec1efa2 to 0x1085411 2024-09-17T21:30:17.227 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:17.225+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:17.237 INFO:teuthology.orchestra.run.smithi129.stdout:2302 write 0xadba5be thru 0xadbf617 (0x505a bytes) 2024-09-17T21:30:17.240 INFO:teuthology.orchestra.run.smithi129.stdout:2303 read 0xa4b623f thru 0xa4c5e12 (0xfbd4 bytes) 2024-09-17T21:30:17.240 INFO:teuthology.orchestra.run.smithi129.stdout:2304 read 0x59a0f20 thru 0x59a88a2 (0x7983 bytes) 2024-09-17T21:30:17.244 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:17.244 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:17.244 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:17.244 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:17.246 INFO:teuthology.orchestra.run.smithi129.stdout:2305 write 0xb41b299 thru 0xb424785 (0x94ed bytes) 2024-09-17T21:30:17.249 INFO:teuthology.orchestra.run.smithi129.stdout:2308 read 0xa62fe14 thru 0xa63e087 (0xe274 bytes) 2024-09-17T21:30:17.249 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:17.250 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:17.250 INFO:teuthology.orchestra.run.smithi129.stdout:2312 read 0x8fa2e7b thru 0x8fad022 (0xa1a8 bytes) 2024-09-17T21:30:17.250 INFO:teuthology.orchestra.run.smithi129.stdout:2313 write 0x25bcf53 thru 0x25bdcc1 (0xd6f bytes) 2024-09-17T21:30:17.251 INFO:teuthology.orchestra.run.smithi129.stdout:2315 write 0xe51dd4 thru 0xe57900 (0x5b2d bytes) 2024-09-17T21:30:17.253 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:30:17.254 INFO:teuthology.orchestra.run.smithi129.stdout:2316 write 0xaecb02c thru 0xaed24e5 (0x74ba bytes) 2024-09-17T21:30:17.258 INFO:teuthology.orchestra.run.smithi129.stdout:2317 punch from 0xd9179 to 0xdc093, (0x2f1a bytes) 2024-09-17T21:30:17.266 INFO:teuthology.orchestra.run.smithi129.stdout:2320 writesame 0xd29370c thru 0xd29bf14 data_size 0x23(0x8809 bytes) 2024-09-17T21:30:17.266 INFO:teuthology.orchestra.run.smithi129.stdout:2321 writesame 0x1f73b7d thru 0x1f7ba42 data_size 0x9(0x7ec6 bytes) 2024-09-17T21:30:17.268 INFO:teuthology.orchestra.run.smithi129.stdout:2322 read 0xcde2c6e thru 0xcde92e0 (0x6673 bytes) 2024-09-17T21:30:17.268 INFO:teuthology.orchestra.run.smithi129.stdout:2324 punch from 0xc988fb8 to 0xc98e3f2, (0x543a bytes) 2024-09-17T21:30:17.273 INFO:teuthology.orchestra.run.smithi129.stdout:2326 write 0xdf1b6c9 thru 0xdf26e8b (0xb7c3 bytes) 2024-09-17T21:30:17.280 INFO:teuthology.orchestra.run.smithi129.stdout:2327 trunc from 0xdf26e8c to 0xabc94ed 2024-09-17T21:30:17.291 INFO:teuthology.orchestra.run.smithi129.stdout:2328 punch from 0x64d2399 to 0x64dc745, (0xa3ac bytes) 2024-09-17T21:30:17.291 INFO:teuthology.orchestra.run.smithi129.stdout:2329 read 0x1b31dc7 thru 0x1b3f428 (0xd662 bytes) 2024-09-17T21:30:17.291 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:17.292 INFO:teuthology.orchestra.run.smithi129.stdout:2332 writesame 0xaf2349 thru 0xb00f2d data_size 0x1(0xebe5 bytes) 2024-09-17T21:30:17.313 INFO:teuthology.orchestra.run.smithi129.stdout:2333 read 0x2ce96a9 thru 0x2ceb5ce (0x1f26 bytes) 2024-09-17T21:30:17.313 INFO:teuthology.orchestra.run.smithi129.stdout:2337 punch from 0x3bf7105 to 0x3bfd8e9, (0x67e4 bytes) 2024-09-17T21:30:17.321 INFO:teuthology.orchestra.run.smithi129.stdout:2338 writesame 0xca73c15 thru 0xca7b163 data_size 0x1(0x754f bytes) 2024-09-17T21:30:17.321 INFO:teuthology.orchestra.run.smithi129.stdout:2339 read 0x6335b57 thru 0x633d615 (0x7abf bytes) 2024-09-17T21:30:17.321 INFO:teuthology.orchestra.run.smithi129.stdout:2340 writesame 0xae89f01 thru 0xae96620 data_size 0x76(0xc720 bytes) 2024-09-17T21:30:17.323 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:17.323 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:17.323 INFO:teuthology.orchestra.run.smithi129.stdout:2344 read 0xb89a9d3 thru 0xb8a5402 (0xaa30 bytes) 2024-09-17T21:30:17.323 INFO:teuthology.orchestra.run.smithi129.stdout:2346 read 0x86465a6 thru 0x864b71d (0x5178 bytes) 2024-09-17T21:30:17.323 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:17.323 INFO:teuthology.orchestra.run.smithi129.stdout:2348 clone 19 order 19 su 131072 sc 15 2024-09-17T21:30:17.329 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:17.327+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:17.358 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:17.358 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:17.358 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:17.358 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:17.358 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:17.367 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:30:17.430 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:17.427+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:17.472 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21342 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2082816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.792612221999999, 2024-09-17T21:30:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022083523000000001, 2024-09-17T21:30:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21342 2024-09-17T21:30:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:30:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676659+0000", 2024-09-17T21:30:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0950000000000001e-06 2024-09-17T21:30:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676662+0000", 2024-09-17T21:30:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.971e-06 2024-09-17T21:30:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676665+0000", 2024-09-17T21:30:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6189999999999998e-06 2024-09-17T21:30:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698568+0000", 2024-09-17T21:30:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021903549000000001 2024-09-17T21:30:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698668+0000", 2024-09-17T21:30:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9778999999999995e-05 2024-09-17T21:30:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698741+0000", 2024-09-17T21:30:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000006e-05 2024-09-17T21:30:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21343 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2095104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.792602284000001, 2024-09-17T21:30:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022504024000000001, 2024-09-17T21:30:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21343 2024-09-17T21:30:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:30:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676669+0000", 2024-09-17T21:30:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.003e-06 2024-09-17T21:30:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676672+0000", 2024-09-17T21:30:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9280000000000002e-06 2024-09-17T21:30:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676674+0000", 2024-09-17T21:30:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5469999999999999e-06 2024-09-17T21:30:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699015+0000", 2024-09-17T21:30:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022340563000000001 2024-09-17T21:30:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699102+0000", 2024-09-17T21:30:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7425999999999999e-05 2024-09-17T21:30:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699172+0000", 2024-09-17T21:30:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9346999999999999e-05 2024-09-17T21:30:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45576 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.764368484, 2024-09-17T21:30:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022024660000000001, 2024-09-17T21:30:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45576 2024-09-17T21:30:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704902+0000", 2024-09-17T21:30:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9100000000000004e-07 2024-09-17T21:30:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704905+0000", 2024-09-17T21:30:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6139999999999999e-06 2024-09-17T21:30:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704911+0000", 2024-09-17T21:30:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8619999999999997e-06 2024-09-17T21:30:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704917+0000", 2024-09-17T21:30:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:30:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726681+0000", 2024-09-17T21:30:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021764569000000001 2024-09-17T21:30:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726726+0000", 2024-09-17T21:30:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4218999999999998e-05 2024-09-17T21:30:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726926+0000", 2024-09-17T21:30:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000200466 2024-09-17T21:30:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45577 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.764344828, 2024-09-17T21:30:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022269694999999999, 2024-09-17T21:30:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45577 2024-09-17T21:30:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704926+0000", 2024-09-17T21:30:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:30:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704928+0000", 2024-09-17T21:30:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9989999999999998e-06 2024-09-17T21:30:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704934+0000", 2024-09-17T21:30:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:30:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704939+0000", 2024-09-17T21:30:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5909999999999998e-06 2024-09-17T21:30:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726954+0000", 2024-09-17T21:30:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022014885000000001 2024-09-17T21:30:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726995+0000", 2024-09-17T21:30:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1474000000000003e-05 2024-09-17T21:30:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727195+0000", 2024-09-17T21:30:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019968300000000001 2024-09-17T21:30:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45578 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.764323584, 2024-09-17T21:30:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022484006000000001, 2024-09-17T21:30:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45578 2024-09-17T21:30:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704947+0000", 2024-09-17T21:30:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1399999999999997e-07 2024-09-17T21:30:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704949+0000", 2024-09-17T21:30:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0959999999999999e-06 2024-09-17T21:30:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704956+0000", 2024-09-17T21:30:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9330000000000004e-06 2024-09-17T21:30:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704961+0000", 2024-09-17T21:30:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6759999999999998e-06 2024-09-17T21:30:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727213+0000", 2024-09-17T21:30:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022252568 2024-09-17T21:30:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727259+0000", 2024-09-17T21:30:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5238000000000003e-05 2024-09-17T21:30:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727430+0000", 2024-09-17T21:30:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000171881 2024-09-17T21:30:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45579 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.764301656000001, 2024-09-17T21:30:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022749080000000001, 2024-09-17T21:30:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45579 2024-09-17T21:30:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704969+0000", 2024-09-17T21:30:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:30:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704971+0000", 2024-09-17T21:30:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.097e-06 2024-09-17T21:30:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704977+0000", 2024-09-17T21:30:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6329999999999999e-06 2024-09-17T21:30:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704983+0000", 2024-09-17T21:30:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:30:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727457+0000", 2024-09-17T21:30:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022474521000000001 2024-09-17T21:30:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727498+0000", 2024-09-17T21:30:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0558e-05 2024-09-17T21:30:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727717+0000", 2024-09-17T21:30:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219805 2024-09-17T21:30:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45650 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 20480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.763178349, 2024-09-17T21:30:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021996219000000001, 2024-09-17T21:30:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45650 2024-09-17T21:30:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5199999999999997e-07 2024-09-17T21:30:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706095+0000", 2024-09-17T21:30:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6079999999999998e-06 2024-09-17T21:30:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706101+0000", 2024-09-17T21:30:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:30:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706106+0000", 2024-09-17T21:30:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T21:30:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728000+0000", 2024-09-17T21:30:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021893378000000002 2024-09-17T21:30:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728039+0000", 2024-09-17T21:30:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9249000000000003e-05 2024-09-17T21:30:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728088+0000", 2024-09-17T21:30:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8785000000000001e-05 2024-09-17T21:30:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45652 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.763155518, 2024-09-17T21:30:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022100465999999999, 2024-09-17T21:30:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45652 2024-09-17T21:30:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706115+0000", 2024-09-17T21:30:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:30:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706117+0000", 2024-09-17T21:30:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-06 2024-09-17T21:30:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706122+0000", 2024-09-17T21:30:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.288e-06 2024-09-17T21:30:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706136+0000", 2024-09-17T21:30:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3409e-05 2024-09-17T21:30:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728112+0000", 2024-09-17T21:30:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021976724999999999 2024-09-17T21:30:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728151+0000", 2024-09-17T21:30:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8318999999999997e-05 2024-09-17T21:30:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728215+0000", 2024-09-17T21:30:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4172999999999997e-05 2024-09-17T21:30:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45653 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.763126308, 2024-09-17T21:30:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022249517999999999, 2024-09-17T21:30:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45653 2024-09-17T21:30:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T21:30:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706147+0000", 2024-09-17T21:30:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4430000000000002e-06 2024-09-17T21:30:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706152+0000", 2024-09-17T21:30:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5350000000000004e-06 2024-09-17T21:30:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706157+0000", 2024-09-17T21:30:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8420000000000004e-06 2024-09-17T21:30:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728246+0000", 2024-09-17T21:30:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022089237000000001 2024-09-17T21:30:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728309+0000", 2024-09-17T21:30:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2356999999999999e-05 2024-09-17T21:30:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:30:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4596000000000001e-05 2024-09-17T21:30:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45657 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 36864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.763095044, 2024-09-17T21:30:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022218252000000001, 2024-09-17T21:30:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45657 2024-09-17T21:30:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8299999999999997e-07 2024-09-17T21:30:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706178+0000", 2024-09-17T21:30:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3700000000000002e-06 2024-09-17T21:30:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706184+0000", 2024-09-17T21:30:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.144e-06 2024-09-17T21:30:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706190+0000", 2024-09-17T21:30:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8590000000000001e-06 2024-09-17T21:30:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728240+0000", 2024-09-17T21:30:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022049804999999999 2024-09-17T21:30:17.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728307+0000", 2024-09-17T21:30:17.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7477999999999993e-05 2024-09-17T21:30:17.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:30:17.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6112999999999994e-05 2024-09-17T21:30:17.520 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.520 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.520 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.520 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45658 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:17.520 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:17.521 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.763071924, 2024-09-17T21:30:17.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022376455, 2024-09-17T21:30:17.521 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.521 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.521 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:17.521 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:17.521 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:17.521 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.521 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45658 2024-09-17T21:30:17.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.521 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:17.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:17.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:17.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:30:17.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706200+0000", 2024-09-17T21:30:17.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:30:17.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706206+0000", 2024-09-17T21:30:17.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7039999999999999e-06 2024-09-17T21:30:17.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706211+0000", 2024-09-17T21:30:17.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9429999999999999e-06 2024-09-17T21:30:17.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728418+0000", 2024-09-17T21:30:17.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022206760999999998 2024-09-17T21:30:17.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728497+0000", 2024-09-17T21:30:17.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9493999999999995e-05 2024-09-17T21:30:17.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728574+0000", 2024-09-17T21:30:17.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7180000000000003e-05 2024-09-17T21:30:17.526 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.526 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.526 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.526 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45707 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 212992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:17.526 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:17.526 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.761367858, 2024-09-17T21:30:17.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022235550999999999, 2024-09-17T21:30:17.526 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.526 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.526 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:17.526 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:17.527 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:17.527 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.527 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45707 2024-09-17T21:30:17.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.527 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:17.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:17.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:17.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:30:17.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707905+0000", 2024-09-17T21:30:17.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2120000000000002e-06 2024-09-17T21:30:17.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707911+0000", 2024-09-17T21:30:17.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1929999999999998e-06 2024-09-17T21:30:17.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707915+0000", 2024-09-17T21:30:17.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:30:17.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729897+0000", 2024-09-17T21:30:17.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021981186 2024-09-17T21:30:17.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729944+0000", 2024-09-17T21:30:17.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571000000000002e-05 2024-09-17T21:30:17.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730138+0000", 2024-09-17T21:30:17.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000193451 2024-09-17T21:30:17.531 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.531 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.531 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.531 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45708 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:17.531 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:17.531 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.761347726, 2024-09-17T21:30:17.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022533742999999998, 2024-09-17T21:30:17.531 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.531 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.531 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:17.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:17.532 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:17.532 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.532 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45708 2024-09-17T21:30:17.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.532 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:17.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:17.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707923+0000", 2024-09-17T21:30:17.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:30:17.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707932+0000", 2024-09-17T21:30:17.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6430000000000007e-06 2024-09-17T21:30:17.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707939+0000", 2024-09-17T21:30:17.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3539999999999996e-06 2024-09-17T21:30:17.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707944+0000", 2024-09-17T21:30:17.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3249999999999998e-06 2024-09-17T21:30:17.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730176+0000", 2024-09-17T21:30:17.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022232393 2024-09-17T21:30:17.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730263+0000", 2024-09-17T21:30:17.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6638999999999995e-05 2024-09-17T21:30:17.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730456+0000", 2024-09-17T21:30:17.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019304899999999999 2024-09-17T21:30:17.535 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.535 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.535 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.536 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45752 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:17.536 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:17.536 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.761092506000001, 2024-09-17T21:30:17.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022426878000000001, 2024-09-17T21:30:17.536 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.536 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.536 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:17.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:17.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:17.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.536 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45752 2024-09-17T21:30:17.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.536 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:17.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:17.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708178+0000", 2024-09-17T21:30:17.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:30:17.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708181+0000", 2024-09-17T21:30:17.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.909e-06 2024-09-17T21:30:17.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708187+0000", 2024-09-17T21:30:17.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4999999999999996e-06 2024-09-17T21:30:17.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708193+0000", 2024-09-17T21:30:17.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:30:17.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730484+0000", 2024-09-17T21:30:17.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022291765000000002 2024-09-17T21:30:17.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730541+0000", 2024-09-17T21:30:17.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6483000000000001e-05 2024-09-17T21:30:17.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730604+0000", 2024-09-17T21:30:17.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3429000000000005e-05 2024-09-17T21:30:17.540 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.540 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.540 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.540 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45761 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:17.540 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:17.540 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.761064596000001, 2024-09-17T21:30:17.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022614408999999999, 2024-09-17T21:30:17.540 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.541 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.541 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:17.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:17.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:17.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.541 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45761 2024-09-17T21:30:17.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.541 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:17.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:17.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708206+0000", 2024-09-17T21:30:17.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4700000000000001e-07 2024-09-17T21:30:17.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708208+0000", 2024-09-17T21:30:17.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1509999999999998e-06 2024-09-17T21:30:17.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708214+0000", 2024-09-17T21:30:17.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.49e-06 2024-09-17T21:30:17.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708219+0000", 2024-09-17T21:30:17.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8119999999999998e-06 2024-09-17T21:30:17.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730641+0000", 2024-09-17T21:30:17.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022422765000000001 2024-09-17T21:30:17.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730731+0000", 2024-09-17T21:30:17.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9598999999999994e-05 2024-09-17T21:30:17.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730820+0000", 2024-09-17T21:30:17.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8844999999999994e-05 2024-09-17T21:30:17.545 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.545 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.545 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.545 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45778 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:17.545 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:17.545 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.760320992, 2024-09-17T21:30:17.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022018544000000001, 2024-09-17T21:30:17.545 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.545 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.546 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:17.546 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:17.546 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:17.546 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.546 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45778 2024-09-17T21:30:17.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.546 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:17.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:17.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708950+0000", 2024-09-17T21:30:17.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-17T21:30:17.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709307+0000", 2024-09-17T21:30:17.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000356848 2024-09-17T21:30:17.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709316+0000", 2024-09-17T21:30:17.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1430000000000004e-06 2024-09-17T21:30:17.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709328+0000", 2024-09-17T21:30:17.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2629999999999999e-05 2024-09-17T21:30:17.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730861+0000", 2024-09-17T21:30:17.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021532057 2024-09-17T21:30:17.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730905+0000", 2024-09-17T21:30:17.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4907000000000002e-05 2024-09-17T21:30:17.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730968+0000", 2024-09-17T21:30:17.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2108999999999998e-05 2024-09-17T21:30:17.550 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.550 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.550 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.550 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45785 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:17.550 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:17.550 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.759842809, 2024-09-17T21:30:17.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022044113000000001, 2024-09-17T21:30:17.550 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.550 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.550 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:17.551 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:17.551 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:17.551 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.551 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45785 2024-09-17T21:30:17.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.551 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:17.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:17.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709428+0000", 2024-09-17T21:30:17.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:30:17.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709430+0000", 2024-09-17T21:30:17.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:30:17.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709451+0000", 2024-09-17T21:30:17.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1372000000000001e-05 2024-09-17T21:30:17.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709472+0000", 2024-09-17T21:30:17.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1007999999999999e-05 2024-09-17T21:30:17.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731293+0000", 2024-09-17T21:30:17.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021821320000000002 2024-09-17T21:30:17.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731378+0000", 2024-09-17T21:30:17.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5059e-05 2024-09-17T21:30:17.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731471+0000", 2024-09-17T21:30:17.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2867000000000004e-05 2024-09-17T21:30:17.554 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.555 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.555 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.555 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45786 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:17.555 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:17.555 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.759784012000001, 2024-09-17T21:30:17.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022181821000000001, 2024-09-17T21:30:17.555 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.555 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.555 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:17.555 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:17.555 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:17.555 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:17.556 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45786 2024-09-17T21:30:17.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.556 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:17.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:17.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709487+0000", 2024-09-17T21:30:17.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1299999999999999e-07 2024-09-17T21:30:17.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709491+0000", 2024-09-17T21:30:17.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2089999999999999e-06 2024-09-17T21:30:17.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709498+0000", 2024-09-17T21:30:17.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3980000000000002e-06 2024-09-17T21:30:17.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709504+0000", 2024-09-17T21:30:17.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7219999999999996e-06 2024-09-17T21:30:17.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731499+0000", 2024-09-17T21:30:17.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021995297 2024-09-17T21:30:17.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731591+0000", 2024-09-17T21:30:17.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1373999999999996e-05 2024-09-17T21:30:17.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731668+0000", 2024-09-17T21:30:17.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7107999999999994e-05 2024-09-17T21:30:17.559 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.559 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.559 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.559 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50202 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'139, mlcod=45'137)", 2024-09-17T21:30:17.559 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:30:17.559 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 6.3450481429999996, 2024-09-17T21:30:17.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059659441000000001, 2024-09-17T21:30:17.560 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.560 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.560 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:17.560 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:30:17.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:30:17.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124224+0000", 2024-09-17T21:30:17.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5000000000000002e-06 2024-09-17T21:30:17.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147385+0000", 2024-09-17T21:30:17.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023160544000000002 2024-09-17T21:30:17.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147411+0000", 2024-09-17T21:30:17.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6095999999999999e-05 2024-09-17T21:30:17.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147438+0000", 2024-09-17T21:30:17.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6982999999999999e-05 2024-09-17T21:30:17.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147504+0000", 2024-09-17T21:30:17.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6156999999999997e-05 2024-09-17T21:30:17.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147531+0000", 2024-09-17T21:30:17.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6741999999999999e-05 2024-09-17T21:30:17.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:17.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183820+0000", 2024-09-17T21:30:17.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036289090000000003 2024-09-17T21:30:17.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183881+0000", 2024-09-17T21:30:17.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1328999999999995e-05 2024-09-17T21:30:17.563 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.563 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.564 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.564 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50203 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'141, mlcod=45'137)", 2024-09-17T21:30:17.564 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:30:17.564 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 6.3077854000000002, 2024-09-17T21:30:17.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023138823999999999, 2024-09-17T21:30:17.564 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:17.564 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:17.564 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:17.564 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:17.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:17.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:30:17.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:17.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:30:17.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:17.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:17.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161486+0000", 2024-09-17T21:30:17.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5290000000000001e-06 2024-09-17T21:30:17.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:17.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161496+0000", 2024-09-17T21:30:17.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0063e-05 2024-09-17T21:30:17.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:17.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161514+0000", 2024-09-17T21:30:17.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7728e-05 2024-09-17T21:30:17.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:17.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161524+0000", 2024-09-17T21:30:17.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0339e-05 2024-09-17T21:30:17.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:17.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170872+0000", 2024-09-17T21:30:17.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093473640000000004 2024-09-17T21:30:17.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:17.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170899+0000", 2024-09-17T21:30:17.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7321000000000002e-05 2024-09-17T21:30:17.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:17.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184580+0000", 2024-09-17T21:30:17.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013680988999999999 2024-09-17T21:30:17.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:17.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:17.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:17.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184623+0000", 2024-09-17T21:30:17.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3491000000000001e-05 2024-09-17T21:30:17.568 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.568 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.568 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.568 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:17.568 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:17.568 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:17.569 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:30:17.570 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:17.567+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:17.571 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:30:17.670 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:17.668+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:17.680 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:17.680 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:17.680 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:17.680 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:17.690 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:30:17.701 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:17.701 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:17.701 INFO:teuthology.orchestra.run.smithi129.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-17T21:30:17.719 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:17.719 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:17.736 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:17.736 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:17.753 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:17.753 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:17.769 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:17.769 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:17.774 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:17.772+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:17.786 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:17.786 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:17.803 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:17.803 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:17.807 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:30:17.807 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:30:17.836 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:17.836 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:17.846 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:30:17.859 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-17T21:30:17.872 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:17.870+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:17.973 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:17.971+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:17.987 INFO:teuthology.orchestra.run.smithi129.stdout:2349 clone 20 order 22 su 65536 sc 2 2024-09-17T21:30:17.990 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:17.990 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:17.999 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:30:18.083 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:18.081+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:18.144 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:18.144 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:18.153 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:30:18.194 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:18.192+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:18.276 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:18.274+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:18.297 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:18.297 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:18.306 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:30:18.390 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:18.388+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:18.450 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:18.451 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:18.459 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:30:18.479 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:18.477+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:18.580 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:18.578+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:18.603 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:18.603 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:18.613 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:30:18.676 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:30:18.697 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:18.695+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:18.737 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone19 from 0xca7b164 (overlap 0xca7b164) to 0x653d8b2 2024-09-17T21:30:18.770 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:18.771 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:18.783 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:30:18.783 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:18.780+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:18.884 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:18.881+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:18.933 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:18.934 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:18.942 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:30:18.973 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-17T21:30:18.984 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:18.982+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:19.057 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:19.058 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:19.067 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:30:19.109 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:19.106+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:19.187 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:19.184+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:19.200 INFO:teuthology.orchestra.run.smithi129.stdout:2350 writesame 0x1172cb1 thru 0x117b313 data_size 0x1(0x8663 bytes) 2024-09-17T21:30:19.217 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:19.217 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:19.226 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:30:19.236 INFO:teuthology.orchestra.run.smithi129.stdout:2351 trunc from 0xca7b164 to 0x644faa2 2024-09-17T21:30:19.245 INFO:teuthology.orchestra.run.smithi129.stdout:2352 read 0x366237 thru 0x368838 (0x2602 bytes) 2024-09-17T21:30:19.246 INFO:teuthology.orchestra.run.smithi129.stdout:2354 write 0x1fcd128 thru 0x1fdbc0c (0xeae5 bytes) 2024-09-17T21:30:19.270 INFO:teuthology.orchestra.run.smithi129.stdout:2355 write 0x78079fe thru 0x78178ba (0xfebd bytes) 2024-09-17T21:30:19.273 INFO:teuthology.orchestra.run.smithi129.stdout:2356 writesame 0x2561874 thru 0x25717f1 data_size 0x21(0xff7e bytes) 2024-09-17T21:30:19.287 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:19.285+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:19.305 INFO:teuthology.orchestra.run.smithi129.stdout:2357 writesame 0xc9b13e8 thru 0xc9b5e6f data_size 0x5a(0x4a88 bytes) 2024-09-17T21:30:19.306 INFO:teuthology.orchestra.run.smithi129.stdout:2358 punch from 0x261aa2b to 0x261e11f, (0x36f4 bytes) 2024-09-17T21:30:19.306 INFO:teuthology.orchestra.run.smithi129.stdout:2359 write 0xacdf31a thru 0xacdf9a9 (0x690 bytes) 2024-09-17T21:30:19.308 INFO:teuthology.orchestra.run.smithi129.stdout:2360 trunc from 0xc9b5e70 to 0xd89fd40 2024-09-17T21:30:19.329 INFO:teuthology.orchestra.run.smithi129.stdout:2361 trunc from 0xd89fd40 to 0x8419c1 2024-09-17T21:30:19.351 INFO:teuthology.orchestra.run.smithi129.stdout:2363 punch from 0x27cf49 to 0x280547, (0x35fe bytes) 2024-09-17T21:30:19.352 INFO:teuthology.orchestra.run.smithi129.stdout:2365 read 0x33bc41 thru 0x34648e (0xa84e bytes) 2024-09-17T21:30:19.352 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:19.363 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:19.363 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:19.365 INFO:teuthology.orchestra.run.smithi129.stdout:2367 writesame 0x805c9c1 thru 0x805e806 data_size 0x32(0x1e46 bytes) 2024-09-17T21:30:19.365 INFO:teuthology.orchestra.run.smithi129.stdout:2369 read 0x3262bc2 thru 0x326c3e5 (0x9824 bytes) 2024-09-17T21:30:19.371 INFO:teuthology.orchestra.run.smithi129.stdout:2371 trunc from 0x805e807 to 0xd327000 2024-09-17T21:30:19.374 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:30:19.377 INFO:teuthology.orchestra.run.smithi129.stdout:2373 read 0xcb22f43 thru 0xcb2c7d2 (0x9890 bytes) 2024-09-17T21:30:19.377 INFO:teuthology.orchestra.run.smithi129.stdout:2374 punch from 0xb23af3b to 0xb244595, (0x965a bytes) 2024-09-17T21:30:19.377 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:19.378 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:19.378 INFO:teuthology.orchestra.run.smithi129.stdout:2377 writesame 0xb1c0e74 thru 0xb1c6d59 data_size 0x3(0x5ee6 bytes) 2024-09-17T21:30:19.382 INFO:teuthology.orchestra.run.smithi129.stdout:2378 write 0xad4083f thru 0xad42d0f (0x24d1 bytes) 2024-09-17T21:30:19.385 INFO:teuthology.orchestra.run.smithi129.stdout:2379 write 0x1665147 thru 0x16680e1 (0x2f9b bytes) 2024-09-17T21:30:19.388 INFO:teuthology.orchestra.run.smithi129.stdout:2381 write 0x9f1a698 thru 0x9f28adc (0xe445 bytes) 2024-09-17T21:30:19.388 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:19.386+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:19.393 INFO:teuthology.orchestra.run.smithi129.stdout:2384 punch from 0x1b34bfa to 0x1b40d01, (0xc107 bytes) 2024-09-17T21:30:19.393 INFO:teuthology.orchestra.run.smithi129.stdout:2387 read 0xafa0418 thru 0xafac41a (0xc003 bytes) 2024-09-17T21:30:19.394 INFO:teuthology.orchestra.run.smithi129.stdout:2388 punch from 0x3698fdc to 0x36a52da, (0xc2fe bytes) 2024-09-17T21:30:19.394 INFO:teuthology.orchestra.run.smithi129.stdout:2389 write 0x5c3bfd thru 0x5ce7c6 (0xabca bytes) 2024-09-17T21:30:19.407 INFO:teuthology.orchestra.run.smithi129.stdout:2390 clone 21 order 25 su 262144 sc 3 2024-09-17T21:30:19.484 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:19.485 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:19.492 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:19.487+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:19.493 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:30:19.590 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:19.588+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:19.636 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:19.637 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:19.645 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:30:19.681 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is set 2024-09-17T21:30:19.691 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:19.689+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:19.742 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone20 from 0xd327000 (overlap 0x8419c1) to 0x34d71a 2024-09-17T21:30:19.792 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:19.789+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:19.796 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:19.796 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:19.807 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:30:19.807 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:30:19.828 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-17T21:30:19.892 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:19.890+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:19.975 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:19.975 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:19.981 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:19.981 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:19.982 INFO:teuthology.orchestra.run.smithi129.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-17T21:30:19.987 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:30:20.000 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:20.000 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:20.000 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:19.997+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:20.008 INFO:teuthology.orchestra.run.smithi129.stdout:2391 write 0xd4133a thru 0xd42789 (0x1450 bytes) 2024-09-17T21:30:20.028 INFO:teuthology.orchestra.run.smithi129.stdout:2392 punch from 0x40ad926 to 0x40b0c34, (0x330e bytes) 2024-09-17T21:30:20.028 INFO:teuthology.orchestra.run.smithi129.stdout:2395 read 0x8eae1dd thru 0x8ebdbe6 (0xfa0a bytes) 2024-09-17T21:30:20.028 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:20.028 INFO:teuthology.orchestra.run.smithi129.stdout:2397 read 0x3871dd6 thru 0x387c718 (0xa943 bytes) 2024-09-17T21:30:20.028 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:20.028 INFO:teuthology.orchestra.run.smithi129.stdout:2399 clone 22 order 20 su 65536 sc 12 2024-09-17T21:30:20.093 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:20.093 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:20.094 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:20.091+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:20.102 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:30:20.194 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:20.193+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:20.218 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:20.218 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:20.249 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:20.249 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:20.267 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:20.267 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:49361 3.19 e46/25 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:head v 46'135, mlcod=45'133)", 2024-09-17T21:30:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 12.511080853999999, 2024-09-17T21:30:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043477744999999998, 2024-09-17T21:30:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-07 2024-09-17T21:30:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769703+0000", 2024-09-17T21:30:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016134381 2024-09-17T21:30:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769724+0000", 2024-09-17T21:30:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0781999999999999e-05 2024-09-17T21:30:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769734+0000", 2024-09-17T21:30:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6870000000000006e-06 2024-09-17T21:30:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769797+0000", 2024-09-17T21:30:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3418999999999997e-05 2024-09-17T21:30:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769822+0000", 2024-09-17T21:30:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4652999999999999e-05 2024-09-17T21:30:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.796982+0000", 2024-09-17T21:30:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027160205 2024-09-17T21:30:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.797047+0000", 2024-09-17T21:30:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4455999999999994e-05 2024-09-17T21:30:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50804 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 14155776~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495110+0000", 2024-09-17T21:30:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.7695400499999998, 2024-09-17T21:30:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.034879440999999997, 2024-09-17T21:30:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50804 2024-09-17T21:30:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495110+0000", 2024-09-17T21:30:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495110+0000", 2024-09-17T21:30:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495110+0000", 2024-09-17T21:30:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2e-07 2024-09-17T21:30:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495112+0000", 2024-09-17T21:30:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7689999999999999e-06 2024-09-17T21:30:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495133+0000", 2024-09-17T21:30:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1319e-05 2024-09-17T21:30:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495138+0000", 2024-09-17T21:30:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:30:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529422+0000", 2024-09-17T21:30:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.034284199000000001 2024-09-17T21:30:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529492+0000", 2024-09-17T21:30:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9502e-05 2024-09-17T21:30:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529989+0000", 2024-09-17T21:30:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00049732900000000002 2024-09-17T21:30:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50805 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 17825792~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:30:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.7694973600000008, 2024-09-17T21:30:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035591579999999998, 2024-09-17T21:30:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50805 2024-09-17T21:30:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:30:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:30:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495153+0000", 2024-09-17T21:30:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:30:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495155+0000", 2024-09-17T21:30:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1849999999999998e-06 2024-09-17T21:30:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495162+0000", 2024-09-17T21:30:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.64e-06 2024-09-17T21:30:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495166+0000", 2024-09-17T21:30:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:30:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530020+0000", 2024-09-17T21:30:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.034853824999999998 2024-09-17T21:30:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530076+0000", 2024-09-17T21:30:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5235999999999998e-05 2024-09-17T21:30:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530744+0000", 2024-09-17T21:30:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066835500000000003 2024-09-17T21:30:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50806 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19398656~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.7694768920000001, 2024-09-17T21:30:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036408459999999997, 2024-09-17T21:30:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50806 2024-09-17T21:30:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.72e-07 2024-09-17T21:30:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495175+0000", 2024-09-17T21:30:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:30:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495182+0000", 2024-09-17T21:30:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8079999999999999e-06 2024-09-17T21:30:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495186+0000", 2024-09-17T21:30:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5549999999999996e-06 2024-09-17T21:30:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530791+0000", 2024-09-17T21:30:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035604963000000003 2024-09-17T21:30:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530855+0000", 2024-09-17T21:30:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3286000000000004e-05 2024-09-17T21:30:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531581+0000", 2024-09-17T21:30:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00072655699999999996 2024-09-17T21:30:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50807 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23068672~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.7694580070000008, 2024-09-17T21:30:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037183314000000002, 2024-09-17T21:30:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50807 2024-09-17T21:30:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9999999999999999e-07 2024-09-17T21:30:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495194+0000", 2024-09-17T21:30:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.779e-06 2024-09-17T21:30:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495199+0000", 2024-09-17T21:30:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.468e-06 2024-09-17T21:30:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495203+0000", 2024-09-17T21:30:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0010000000000003e-06 2024-09-17T21:30:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531636+0000", 2024-09-17T21:30:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036432665000000003 2024-09-17T21:30:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531727+0000", 2024-09-17T21:30:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1607999999999994e-05 2024-09-17T21:30:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532375+0000", 2024-09-17T21:30:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00064749299999999996 2024-09-17T21:30:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50808 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24641536~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:30:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.7694412639999992, 2024-09-17T21:30:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037870190999999997, 2024-09-17T21:30:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50808 2024-09-17T21:30:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:30:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:30:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495209+0000", 2024-09-17T21:30:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:30:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495215+0000", 2024-09-17T21:30:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1630000000000001e-06 2024-09-17T21:30:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495221+0000", 2024-09-17T21:30:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-17T21:30:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495226+0000", 2024-09-17T21:30:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8189999999999999e-06 2024-09-17T21:30:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532425+0000", 2024-09-17T21:30:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037199135000000001 2024-09-17T21:30:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532531+0000", 2024-09-17T21:30:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106314 2024-09-17T21:30:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533079+0000", 2024-09-17T21:30:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00054739700000000003 2024-09-17T21:30:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50809 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28311552~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:30:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.7694182979999997, 2024-09-17T21:30:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038597605, 2024-09-17T21:30:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50809 2024-09-17T21:30:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.297 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:30:20.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:30:20.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495232+0000", 2024-09-17T21:30:20.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.39e-07 2024-09-17T21:30:20.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495234+0000", 2024-09-17T21:30:20.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T21:30:20.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495239+0000", 2024-09-17T21:30:20.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7969999999999997e-06 2024-09-17T21:30:20.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495244+0000", 2024-09-17T21:30:20.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999996e-06 2024-09-17T21:30:20.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533125+0000", 2024-09-17T21:30:20.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037881523 2024-09-17T21:30:20.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533237+0000", 2024-09-17T21:30:20.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000111879 2024-09-17T21:30:20.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533829+0000", 2024-09-17T21:30:20.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000591933 2024-09-17T21:30:20.300 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.300 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.300 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.300 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50810 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 29884416~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.300 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:30:20.300 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.7694003800000004, 2024-09-17T21:30:20.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039149809000000001, 2024-09-17T21:30:20.301 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.301 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.301 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.301 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.301 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.301 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.301 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50810 2024-09-17T21:30:20.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.301 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:30:20.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:30:20.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495250+0000", 2024-09-17T21:30:20.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:30:20.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495251+0000", 2024-09-17T21:30:20.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.81e-06 2024-09-17T21:30:20.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495257+0000", 2024-09-17T21:30:20.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1399999999999999e-06 2024-09-17T21:30:20.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495261+0000", 2024-09-17T21:30:20.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4229999999999998e-06 2024-09-17T21:30:20.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533879+0000", 2024-09-17T21:30:20.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038618292999999998 2024-09-17T21:30:20.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533939+0000", 2024-09-17T21:30:20.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.012e-05 2024-09-17T21:30:20.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534399+0000", 2024-09-17T21:30:20.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00045968099999999998 2024-09-17T21:30:20.304 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.305 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.305 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.305 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50850 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 2359296~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.305 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:20.305 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.7693820050000006, 2024-09-17T21:30:20.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039731491000000001, 2024-09-17T21:30:20.305 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.305 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.305 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50850 2024-09-17T21:30:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:20.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3200000000000001e-07 2024-09-17T21:30:20.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495270+0000", 2024-09-17T21:30:20.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.694e-06 2024-09-17T21:30:20.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495275+0000", 2024-09-17T21:30:20.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5890000000000002e-06 2024-09-17T21:30:20.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495279+0000", 2024-09-17T21:30:20.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-17T21:30:20.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534440+0000", 2024-09-17T21:30:20.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039160805999999999 2024-09-17T21:30:20.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534526+0000", 2024-09-17T21:30:20.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5327999999999993e-05 2024-09-17T21:30:20.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534999+0000", 2024-09-17T21:30:20.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00047353400000000001 2024-09-17T21:30:20.309 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.309 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.309 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.309 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50851 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 3932160~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.309 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:30:20.309 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.769364393, 2024-09-17T21:30:20.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040211991000000002, 2024-09-17T21:30:20.310 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.310 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.310 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.310 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.310 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.310 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.310 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50851 2024-09-17T21:30:20.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.310 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:30:20.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:30:20.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495286+0000", 2024-09-17T21:30:20.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:20.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495287+0000", 2024-09-17T21:30:20.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6640000000000001e-06 2024-09-17T21:30:20.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495293+0000", 2024-09-17T21:30:20.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6710000000000004e-06 2024-09-17T21:30:20.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495298+0000", 2024-09-17T21:30:20.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6700000000000002e-06 2024-09-17T21:30:20.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535017+0000", 2024-09-17T21:30:20.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039719059000000001 2024-09-17T21:30:20.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535067+0000", 2024-09-17T21:30:20.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0046000000000002e-05 2024-09-17T21:30:20.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535497+0000", 2024-09-17T21:30:20.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043050299999999999 2024-09-17T21:30:20.314 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.314 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.314 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.314 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50852 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 7602176~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.314 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:30:20.314 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.7693454759999998, 2024-09-17T21:30:20.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041166337999999997, 2024-09-17T21:30:20.314 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.314 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.315 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.315 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.315 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.315 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.315 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50852 2024-09-17T21:30:20.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.315 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:30:20.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:30:20.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495305+0000", 2024-09-17T21:30:20.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1600000000000002e-07 2024-09-17T21:30:20.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495307+0000", 2024-09-17T21:30:20.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0980000000000001e-06 2024-09-17T21:30:20.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495312+0000", 2024-09-17T21:30:20.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6210000000000001e-06 2024-09-17T21:30:20.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495317+0000", 2024-09-17T21:30:20.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T21:30:20.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535531+0000", 2024-09-17T21:30:20.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040214580999999999 2024-09-17T21:30:20.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535618+0000", 2024-09-17T21:30:20.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6648000000000001e-05 2024-09-17T21:30:20.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536471+0000", 2024-09-17T21:30:20.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00085251599999999999 2024-09-17T21:30:20.318 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.318 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.318 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.319 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50853 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 9175040~204800] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.319 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:20.319 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.7693271450000001, 2024-09-17T21:30:20.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041673496999999997, 2024-09-17T21:30:20.319 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.319 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.319 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.319 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.319 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.319 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.319 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50853 2024-09-17T21:30:20.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.319 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:20.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:20.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:20.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:30:20.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495334+0000", 2024-09-17T21:30:20.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.076e-05 2024-09-17T21:30:20.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495340+0000", 2024-09-17T21:30:20.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:30:20.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495345+0000", 2024-09-17T21:30:20.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:30:20.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536514+0000", 2024-09-17T21:30:20.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041168534999999999 2024-09-17T21:30:20.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536572+0000", 2024-09-17T21:30:20.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8742000000000001e-05 2024-09-17T21:30:20.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536996+0000", 2024-09-17T21:30:20.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042368899999999999 2024-09-17T21:30:20.323 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.323 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.323 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.323 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50854 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 12845056~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.323 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:20.323 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.7692980859999992, 2024-09-17T21:30:20.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042777397000000002, 2024-09-17T21:30:20.323 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.323 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.323 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.323 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.324 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50854 2024-09-17T21:30:20.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.324 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:20.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:20.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:20.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.82e-07 2024-09-17T21:30:20.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495354+0000", 2024-09-17T21:30:20.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.091e-06 2024-09-17T21:30:20.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495360+0000", 2024-09-17T21:30:20.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.716e-06 2024-09-17T21:30:20.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495365+0000", 2024-09-17T21:30:20.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2939999999999996e-06 2024-09-17T21:30:20.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537020+0000", 2024-09-17T21:30:20.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041654419999999998 2024-09-17T21:30:20.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537105+0000", 2024-09-17T21:30:20.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5857000000000001e-05 2024-09-17T21:30:20.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538129+0000", 2024-09-17T21:30:20.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010235369999999999 2024-09-17T21:30:20.327 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.327 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.327 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.327 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50855 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 14417920~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.328 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:20.328 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.7692781209999993, 2024-09-17T21:30:20.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043254325000000003, 2024-09-17T21:30:20.328 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.328 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.328 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.328 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.328 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.328 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.328 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50855 2024-09-17T21:30:20.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.328 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:20.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:20.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:20.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:30:20.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495374+0000", 2024-09-17T21:30:20.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.922e-06 2024-09-17T21:30:20.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495379+0000", 2024-09-17T21:30:20.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-17T21:30:20.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495384+0000", 2024-09-17T21:30:20.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-17T21:30:20.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538167+0000", 2024-09-17T21:30:20.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042782845 2024-09-17T21:30:20.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538218+0000", 2024-09-17T21:30:20.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1335000000000002e-05 2024-09-17T21:30:20.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538626+0000", 2024-09-17T21:30:20.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000407656 2024-09-17T21:30:20.332 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.332 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.332 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.332 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50856 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 18087936~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.332 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:20.332 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.769259666, 2024-09-17T21:30:20.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043911444000000001, 2024-09-17T21:30:20.332 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.333 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.333 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.333 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.333 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.333 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.333 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50856 2024-09-17T21:30:20.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.333 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:20.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:20.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:20.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0499999999999999e-07 2024-09-17T21:30:20.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495392+0000", 2024-09-17T21:30:20.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7430000000000001e-06 2024-09-17T21:30:20.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495398+0000", 2024-09-17T21:30:20.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3040000000000001e-06 2024-09-17T21:30:20.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495403+0000", 2024-09-17T21:30:20.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5600000000000004e-06 2024-09-17T21:30:20.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538660+0000", 2024-09-17T21:30:20.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043256832000000002 2024-09-17T21:30:20.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538746+0000", 2024-09-17T21:30:20.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6448999999999999e-05 2024-09-17T21:30:20.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539301+0000", 2024-09-17T21:30:20.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055525099999999999 2024-09-17T21:30:20.336 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.336 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.336 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.337 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50857 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19660800~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.337 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:30:20.337 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.7692402610000002, 2024-09-17T21:30:20.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044408156999999997, 2024-09-17T21:30:20.337 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.337 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.337 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.337 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.337 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.337 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.337 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50857 2024-09-17T21:30:20.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.337 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:30:20.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:30:20.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495410+0000", 2024-09-17T21:30:20.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:30:20.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495412+0000", 2024-09-17T21:30:20.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9580000000000001e-06 2024-09-17T21:30:20.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495418+0000", 2024-09-17T21:30:20.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1909999999999997e-06 2024-09-17T21:30:20.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495423+0000", 2024-09-17T21:30:20.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2429999999999996e-06 2024-09-17T21:30:20.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539338+0000", 2024-09-17T21:30:20.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043914478 2024-09-17T21:30:20.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539393+0000", 2024-09-17T21:30:20.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5226999999999998e-05 2024-09-17T21:30:20.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539818+0000", 2024-09-17T21:30:20.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042469300000000002 2024-09-17T21:30:20.341 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.341 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.341 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.341 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50858 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23330816~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.341 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:20.341 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.7692191640000008, 2024-09-17T21:30:20.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045010798999999997, 2024-09-17T21:30:20.341 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.342 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.342 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.342 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.342 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.342 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.342 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50858 2024-09-17T21:30:20.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.342 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:20.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:20.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:20.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:30:20.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495433+0000", 2024-09-17T21:30:20.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.787e-06 2024-09-17T21:30:20.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495438+0000", 2024-09-17T21:30:20.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:30:20.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495442+0000", 2024-09-17T21:30:20.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2640000000000002e-06 2024-09-17T21:30:20.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539852+0000", 2024-09-17T21:30:20.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044409592999999997 2024-09-17T21:30:20.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539940+0000", 2024-09-17T21:30:20.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7696999999999997e-05 2024-09-17T21:30:20.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540441+0000", 2024-09-17T21:30:20.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050164299999999997 2024-09-17T21:30:20.345 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.345 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.345 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.346 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50859 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24903680~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.346 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:30:20.346 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.769201486, 2024-09-17T21:30:20.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045571569999999999, 2024-09-17T21:30:20.346 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.346 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.346 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.346 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50859 2024-09-17T21:30:20.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.346 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:30:20.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:30:20.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495449+0000", 2024-09-17T21:30:20.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:30:20.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495450+0000", 2024-09-17T21:30:20.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7889999999999999e-06 2024-09-17T21:30:20.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495456+0000", 2024-09-17T21:30:20.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5679999999999999e-06 2024-09-17T21:30:20.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495460+0000", 2024-09-17T21:30:20.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4989999999999996e-06 2024-09-17T21:30:20.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540472+0000", 2024-09-17T21:30:20.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045011585999999999 2024-09-17T21:30:20.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540554+0000", 2024-09-17T21:30:20.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2443999999999999e-05 2024-09-17T21:30:20.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541020+0000", 2024-09-17T21:30:20.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00046534300000000001 2024-09-17T21:30:20.350 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.350 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.350 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.350 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50860 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28573696~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.350 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:20.350 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.7691831330000003, 2024-09-17T21:30:20.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046167899999999998, 2024-09-17T21:30:20.350 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.350 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.351 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.351 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.351 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.351 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.351 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50860 2024-09-17T21:30:20.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.351 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:20.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:20.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:20.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:30:20.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495469+0000", 2024-09-17T21:30:20.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.99e-06 2024-09-17T21:30:20.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495475+0000", 2024-09-17T21:30:20.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:30:20.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495480+0000", 2024-09-17T21:30:20.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:30:20.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541050+0000", 2024-09-17T21:30:20.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045569853 2024-09-17T21:30:20.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541138+0000", 2024-09-17T21:30:20.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7997000000000005e-05 2024-09-17T21:30:20.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541634+0000", 2024-09-17T21:30:20.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00049685500000000004 2024-09-17T21:30:20.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.354 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.355 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.355 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50861 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 30146560~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:20.355 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:20.355 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.7691639559999999, 2024-09-17T21:30:20.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046690553000000003, 2024-09-17T21:30:20.355 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:20.355 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:20.355 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:20.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:20.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:20.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.355 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50861 2024-09-17T21:30:20.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.356 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:20.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:20.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:20.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:20.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:20.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:20.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:20.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:20.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:30:20.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:20.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495488+0000", 2024-09-17T21:30:20.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:30:20.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:20.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495495+0000", 2024-09-17T21:30:20.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:30:20.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495500+0000", 2024-09-17T21:30:20.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8620000000000002e-06 2024-09-17T21:30:20.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:20.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541687+0000", 2024-09-17T21:30:20.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046186434999999998 2024-09-17T21:30:20.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:20.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541741+0000", 2024-09-17T21:30:20.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4228999999999999e-05 2024-09-17T21:30:20.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:20.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:20.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:20.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.542176+0000", 2024-09-17T21:30:20.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043531100000000002 2024-09-17T21:30:20.359 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.359 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.359 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.359 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:20.360 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:20.360 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:20.361 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:30:20.362 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:20.362 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:20.362 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:20.362 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:20.363 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:20.363 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:30:20.363 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:20.364 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:30:20.364 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [1] live_osds: [0, 2, 3, 4, 5, 6, 7] 2024-09-17T21:30:20.364 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:30:20.364 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-17T21:30:20.364 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T21:30:20.364 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T21:30:20.365 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:20.359+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:20.367 INFO:tasks.ceph.osd.1:Started 2024-09-17T21:30:20.367 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:30:20.462 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:20.459+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:20.465 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-17T21:30:20.465 INFO:teuthology.orchestra.run.smithi129.stderr:0 2024-09-17T21:30:20.465 INFO:teuthology.orchestra.run.smithi129.stderr:1 2024-09-17T21:30:20.465 INFO:teuthology.orchestra.run.smithi129.stderr:2 2024-09-17T21:30:20.465 INFO:teuthology.orchestra.run.smithi129.stderr:abort 2024-09-17T21:30:20.465 INFO:teuthology.orchestra.run.smithi129.stderr:assert 2024-09-17T21:30:20.465 INFO:teuthology.orchestra.run.smithi129.stderr:config diff 2024-09-17T21:30:20.465 INFO:teuthology.orchestra.run.smithi129.stderr:config diff get 2024-09-17T21:30:20.465 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-17T21:30:20.466 INFO:teuthology.orchestra.run.smithi129.stderr:config help [] 2024-09-17T21:30:20.466 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-17T21:30:20.466 INFO:teuthology.orchestra.run.smithi129.stderr:admin_socket: invalid command 2024-09-17T21:30:20.467 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:30:20.468 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-17T21:30:20.469 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:20.469 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:20.469 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:20.469 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:20.477 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:30:20.562 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:20.560+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:20.621 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:20.621 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:20.621 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:20.621 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:20.621 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:20.629 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:30:20.673 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:20.671+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:20.738 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:20.738 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:20.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:20.739 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:20.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.739 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21342 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2082816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:20.739 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:20.739 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.058789262000001, 2024-09-17T21:30:20.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022083523000000001, 2024-09-17T21:30:20.739 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.739 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.739 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:20.739 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:20.739 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:20.739 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.740 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21342 2024-09-17T21:30:20.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.740 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:20.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:20.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:20.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:30:20.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676659+0000", 2024-09-17T21:30:20.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0950000000000001e-06 2024-09-17T21:30:20.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676662+0000", 2024-09-17T21:30:20.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.971e-06 2024-09-17T21:30:20.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676665+0000", 2024-09-17T21:30:20.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6189999999999998e-06 2024-09-17T21:30:20.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698568+0000", 2024-09-17T21:30:20.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021903549000000001 2024-09-17T21:30:20.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698668+0000", 2024-09-17T21:30:20.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9778999999999995e-05 2024-09-17T21:30:20.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698741+0000", 2024-09-17T21:30:20.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000006e-05 2024-09-17T21:30:20.743 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.743 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.743 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.743 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21343 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2095104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:20.743 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:20.744 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.058779324, 2024-09-17T21:30:20.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022504024000000001, 2024-09-17T21:30:20.744 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.744 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.744 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:20.744 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:20.744 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:20.744 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.744 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21343 2024-09-17T21:30:20.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.744 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:20.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:20.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:20.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:30:20.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676669+0000", 2024-09-17T21:30:20.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.003e-06 2024-09-17T21:30:20.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676672+0000", 2024-09-17T21:30:20.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9280000000000002e-06 2024-09-17T21:30:20.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676674+0000", 2024-09-17T21:30:20.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5469999999999999e-06 2024-09-17T21:30:20.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699015+0000", 2024-09-17T21:30:20.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022340563000000001 2024-09-17T21:30:20.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699102+0000", 2024-09-17T21:30:20.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7425999999999999e-05 2024-09-17T21:30:20.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699172+0000", 2024-09-17T21:30:20.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9346999999999999e-05 2024-09-17T21:30:20.747 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.747 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.748 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.748 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45576 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:20.748 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:20.748 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.030545524000001, 2024-09-17T21:30:20.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022024660000000001, 2024-09-17T21:30:20.748 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.748 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.748 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:20.748 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:20.748 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:20.748 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.748 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45576 2024-09-17T21:30:20.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.749 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:20.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:20.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704902+0000", 2024-09-17T21:30:20.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9100000000000004e-07 2024-09-17T21:30:20.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704905+0000", 2024-09-17T21:30:20.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6139999999999999e-06 2024-09-17T21:30:20.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704911+0000", 2024-09-17T21:30:20.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8619999999999997e-06 2024-09-17T21:30:20.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704917+0000", 2024-09-17T21:30:20.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:30:20.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726681+0000", 2024-09-17T21:30:20.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021764569000000001 2024-09-17T21:30:20.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726726+0000", 2024-09-17T21:30:20.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4218999999999998e-05 2024-09-17T21:30:20.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726926+0000", 2024-09-17T21:30:20.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000200466 2024-09-17T21:30:20.752 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.752 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.752 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.752 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45577 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:20.752 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:20.752 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.030521867999999, 2024-09-17T21:30:20.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022269694999999999, 2024-09-17T21:30:20.752 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.752 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.753 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:20.753 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:20.753 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:20.753 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.753 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45577 2024-09-17T21:30:20.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.753 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:20.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:20.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704926+0000", 2024-09-17T21:30:20.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:30:20.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704928+0000", 2024-09-17T21:30:20.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9989999999999998e-06 2024-09-17T21:30:20.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704934+0000", 2024-09-17T21:30:20.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:30:20.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704939+0000", 2024-09-17T21:30:20.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5909999999999998e-06 2024-09-17T21:30:20.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726954+0000", 2024-09-17T21:30:20.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022014885000000001 2024-09-17T21:30:20.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726995+0000", 2024-09-17T21:30:20.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1474000000000003e-05 2024-09-17T21:30:20.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727195+0000", 2024-09-17T21:30:20.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019968300000000001 2024-09-17T21:30:20.756 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.756 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.756 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.756 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45578 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:20.756 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:20.757 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.030500624, 2024-09-17T21:30:20.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022484006000000001, 2024-09-17T21:30:20.757 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.757 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.757 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:20.757 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:20.757 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:20.757 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.757 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45578 2024-09-17T21:30:20.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.757 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:20.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:20.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704947+0000", 2024-09-17T21:30:20.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1399999999999997e-07 2024-09-17T21:30:20.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704949+0000", 2024-09-17T21:30:20.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0959999999999999e-06 2024-09-17T21:30:20.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704956+0000", 2024-09-17T21:30:20.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9330000000000004e-06 2024-09-17T21:30:20.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704961+0000", 2024-09-17T21:30:20.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6759999999999998e-06 2024-09-17T21:30:20.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727213+0000", 2024-09-17T21:30:20.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022252568 2024-09-17T21:30:20.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727259+0000", 2024-09-17T21:30:20.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5238000000000003e-05 2024-09-17T21:30:20.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727430+0000", 2024-09-17T21:30:20.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000171881 2024-09-17T21:30:20.760 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.760 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.761 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.761 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45579 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:20.761 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:20.761 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.030478695999999, 2024-09-17T21:30:20.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022749080000000001, 2024-09-17T21:30:20.761 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.761 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.761 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:20.761 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:20.761 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:20.761 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.761 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45579 2024-09-17T21:30:20.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.762 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:20.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:20.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704969+0000", 2024-09-17T21:30:20.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:30:20.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704971+0000", 2024-09-17T21:30:20.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.097e-06 2024-09-17T21:30:20.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704977+0000", 2024-09-17T21:30:20.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6329999999999999e-06 2024-09-17T21:30:20.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704983+0000", 2024-09-17T21:30:20.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:30:20.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727457+0000", 2024-09-17T21:30:20.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022474521000000001 2024-09-17T21:30:20.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727498+0000", 2024-09-17T21:30:20.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0558e-05 2024-09-17T21:30:20.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727717+0000", 2024-09-17T21:30:20.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219805 2024-09-17T21:30:20.765 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.765 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.765 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.765 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45650 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 20480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:20.765 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:20.765 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.029355388999999, 2024-09-17T21:30:20.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021996219000000001, 2024-09-17T21:30:20.765 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.765 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.765 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:20.766 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:20.766 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:20.766 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.766 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45650 2024-09-17T21:30:20.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.766 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:20.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:20.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:20.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5199999999999997e-07 2024-09-17T21:30:20.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706095+0000", 2024-09-17T21:30:20.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6079999999999998e-06 2024-09-17T21:30:20.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706101+0000", 2024-09-17T21:30:20.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:30:20.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706106+0000", 2024-09-17T21:30:20.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T21:30:20.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728000+0000", 2024-09-17T21:30:20.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021893378000000002 2024-09-17T21:30:20.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728039+0000", 2024-09-17T21:30:20.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9249000000000003e-05 2024-09-17T21:30:20.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728088+0000", 2024-09-17T21:30:20.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8785000000000001e-05 2024-09-17T21:30:20.769 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.769 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.769 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.769 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45652 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:20.769 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:20.770 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.029332558, 2024-09-17T21:30:20.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022100465999999999, 2024-09-17T21:30:20.770 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.770 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.770 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:20.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:20.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:20.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.770 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45652 2024-09-17T21:30:20.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.770 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:20.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:20.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706115+0000", 2024-09-17T21:30:20.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:30:20.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706117+0000", 2024-09-17T21:30:20.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-06 2024-09-17T21:30:20.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706122+0000", 2024-09-17T21:30:20.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.288e-06 2024-09-17T21:30:20.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706136+0000", 2024-09-17T21:30:20.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3409e-05 2024-09-17T21:30:20.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728112+0000", 2024-09-17T21:30:20.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021976724999999999 2024-09-17T21:30:20.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728151+0000", 2024-09-17T21:30:20.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8318999999999997e-05 2024-09-17T21:30:20.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728215+0000", 2024-09-17T21:30:20.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4172999999999997e-05 2024-09-17T21:30:20.774 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.774 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.774 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.774 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45653 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:20.774 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:20.774 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.029303347999999, 2024-09-17T21:30:20.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022249517999999999, 2024-09-17T21:30:20.774 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.774 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.774 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:20.774 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:20.774 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:20.775 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.775 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45653 2024-09-17T21:30:20.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.775 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:20.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:20.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:20.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T21:30:20.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706147+0000", 2024-09-17T21:30:20.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4430000000000002e-06 2024-09-17T21:30:20.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706152+0000", 2024-09-17T21:30:20.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5350000000000004e-06 2024-09-17T21:30:20.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706157+0000", 2024-09-17T21:30:20.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8420000000000004e-06 2024-09-17T21:30:20.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728246+0000", 2024-09-17T21:30:20.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022089237000000001 2024-09-17T21:30:20.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728309+0000", 2024-09-17T21:30:20.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2356999999999999e-05 2024-09-17T21:30:20.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:30:20.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4596000000000001e-05 2024-09-17T21:30:20.778 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.778 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.778 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.779 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45657 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 36864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:20.779 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:20.779 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.029272084, 2024-09-17T21:30:20.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022218252000000001, 2024-09-17T21:30:20.779 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.779 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.779 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:20.779 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:20.779 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:20.779 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.779 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45657 2024-09-17T21:30:20.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.779 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:20.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:20.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:20.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8299999999999997e-07 2024-09-17T21:30:20.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706178+0000", 2024-09-17T21:30:20.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3700000000000002e-06 2024-09-17T21:30:20.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706184+0000", 2024-09-17T21:30:20.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.144e-06 2024-09-17T21:30:20.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706190+0000", 2024-09-17T21:30:20.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8590000000000001e-06 2024-09-17T21:30:20.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728240+0000", 2024-09-17T21:30:20.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022049804999999999 2024-09-17T21:30:20.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728307+0000", 2024-09-17T21:30:20.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7477999999999993e-05 2024-09-17T21:30:20.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:30:20.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6112999999999994e-05 2024-09-17T21:30:20.783 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.783 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.783 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.783 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45658 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:20.783 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:20.783 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.029248964000001, 2024-09-17T21:30:20.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022376455, 2024-09-17T21:30:20.783 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.783 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.783 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:20.783 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:20.783 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:20.784 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.784 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45658 2024-09-17T21:30:20.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.784 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:20.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:20.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:20.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:30:20.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706200+0000", 2024-09-17T21:30:20.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:30:20.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706206+0000", 2024-09-17T21:30:20.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7039999999999999e-06 2024-09-17T21:30:20.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706211+0000", 2024-09-17T21:30:20.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9429999999999999e-06 2024-09-17T21:30:20.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728418+0000", 2024-09-17T21:30:20.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022206760999999998 2024-09-17T21:30:20.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728497+0000", 2024-09-17T21:30:20.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9493999999999995e-05 2024-09-17T21:30:20.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728574+0000", 2024-09-17T21:30:20.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7180000000000003e-05 2024-09-17T21:30:20.787 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.787 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.787 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.787 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45707 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 212992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:20.787 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:20.787 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.027544898, 2024-09-17T21:30:20.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022235550999999999, 2024-09-17T21:30:20.787 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.787 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.788 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:20.788 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:20.788 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:20.788 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.788 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45707 2024-09-17T21:30:20.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.788 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:20.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:20.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:20.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:30:20.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707905+0000", 2024-09-17T21:30:20.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2120000000000002e-06 2024-09-17T21:30:20.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707911+0000", 2024-09-17T21:30:20.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1929999999999998e-06 2024-09-17T21:30:20.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707915+0000", 2024-09-17T21:30:20.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:30:20.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729897+0000", 2024-09-17T21:30:20.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021981186 2024-09-17T21:30:20.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729944+0000", 2024-09-17T21:30:20.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571000000000002e-05 2024-09-17T21:30:20.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730138+0000", 2024-09-17T21:30:20.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000193451 2024-09-17T21:30:20.791 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.791 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.791 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.791 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45708 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:20.792 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:20.792 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.027524766000001, 2024-09-17T21:30:20.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022533742999999998, 2024-09-17T21:30:20.792 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.792 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.792 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:20.792 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:20.792 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:20.792 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.792 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45708 2024-09-17T21:30:20.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.792 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:20.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:20.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707923+0000", 2024-09-17T21:30:20.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:30:20.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707932+0000", 2024-09-17T21:30:20.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6430000000000007e-06 2024-09-17T21:30:20.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707939+0000", 2024-09-17T21:30:20.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3539999999999996e-06 2024-09-17T21:30:20.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707944+0000", 2024-09-17T21:30:20.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3249999999999998e-06 2024-09-17T21:30:20.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730176+0000", 2024-09-17T21:30:20.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022232393 2024-09-17T21:30:20.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730263+0000", 2024-09-17T21:30:20.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6638999999999995e-05 2024-09-17T21:30:20.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730456+0000", 2024-09-17T21:30:20.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019304899999999999 2024-09-17T21:30:20.796 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.796 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.796 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.796 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45752 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:20.796 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:20.796 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.027269545999999, 2024-09-17T21:30:20.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022426878000000001, 2024-09-17T21:30:20.796 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.796 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.797 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:20.797 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:20.797 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:20.797 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.797 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45752 2024-09-17T21:30:20.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.797 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:20.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:20.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708178+0000", 2024-09-17T21:30:20.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:30:20.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708181+0000", 2024-09-17T21:30:20.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.909e-06 2024-09-17T21:30:20.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708187+0000", 2024-09-17T21:30:20.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4999999999999996e-06 2024-09-17T21:30:20.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708193+0000", 2024-09-17T21:30:20.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:30:20.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730484+0000", 2024-09-17T21:30:20.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022291765000000002 2024-09-17T21:30:20.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730541+0000", 2024-09-17T21:30:20.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6483000000000001e-05 2024-09-17T21:30:20.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730604+0000", 2024-09-17T21:30:20.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3429000000000005e-05 2024-09-17T21:30:20.800 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.800 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.800 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.800 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45761 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:20.800 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:20.800 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.027241635999999, 2024-09-17T21:30:20.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022614408999999999, 2024-09-17T21:30:20.801 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.801 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.801 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:20.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:20.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:20.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.801 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45761 2024-09-17T21:30:20.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.801 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:20.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:20.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708206+0000", 2024-09-17T21:30:20.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4700000000000001e-07 2024-09-17T21:30:20.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708208+0000", 2024-09-17T21:30:20.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1509999999999998e-06 2024-09-17T21:30:20.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708214+0000", 2024-09-17T21:30:20.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.49e-06 2024-09-17T21:30:20.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708219+0000", 2024-09-17T21:30:20.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8119999999999998e-06 2024-09-17T21:30:20.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730641+0000", 2024-09-17T21:30:20.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022422765000000001 2024-09-17T21:30:20.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730731+0000", 2024-09-17T21:30:20.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9598999999999994e-05 2024-09-17T21:30:20.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730820+0000", 2024-09-17T21:30:20.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8844999999999994e-05 2024-09-17T21:30:20.804 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.804 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.804 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.805 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45778 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:20.805 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:20.805 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.026498031999999, 2024-09-17T21:30:20.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022018544000000001, 2024-09-17T21:30:20.805 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.805 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.805 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:20.805 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:20.805 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:20.805 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.805 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45778 2024-09-17T21:30:20.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.805 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:20.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:20.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708950+0000", 2024-09-17T21:30:20.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-17T21:30:20.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709307+0000", 2024-09-17T21:30:20.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000356848 2024-09-17T21:30:20.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709316+0000", 2024-09-17T21:30:20.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1430000000000004e-06 2024-09-17T21:30:20.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709328+0000", 2024-09-17T21:30:20.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2629999999999999e-05 2024-09-17T21:30:20.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730861+0000", 2024-09-17T21:30:20.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021532057 2024-09-17T21:30:20.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730905+0000", 2024-09-17T21:30:20.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4907000000000002e-05 2024-09-17T21:30:20.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730968+0000", 2024-09-17T21:30:20.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2108999999999998e-05 2024-09-17T21:30:20.809 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.809 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.809 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.809 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45785 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:20.809 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:20.809 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.026019849000001, 2024-09-17T21:30:20.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022044113000000001, 2024-09-17T21:30:20.809 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.809 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.809 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:20.809 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:20.809 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:20.809 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.810 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45785 2024-09-17T21:30:20.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.810 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:20.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:20.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709428+0000", 2024-09-17T21:30:20.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:30:20.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709430+0000", 2024-09-17T21:30:20.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:30:20.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709451+0000", 2024-09-17T21:30:20.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1372000000000001e-05 2024-09-17T21:30:20.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709472+0000", 2024-09-17T21:30:20.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1007999999999999e-05 2024-09-17T21:30:20.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731293+0000", 2024-09-17T21:30:20.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021821320000000002 2024-09-17T21:30:20.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731378+0000", 2024-09-17T21:30:20.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5059e-05 2024-09-17T21:30:20.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731471+0000", 2024-09-17T21:30:20.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2867000000000004e-05 2024-09-17T21:30:20.813 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.813 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.813 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.813 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45786 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.025961052, 2024-09-17T21:30:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022181821000000001, 2024-09-17T21:30:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45786 2024-09-17T21:30:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:20.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709487+0000", 2024-09-17T21:30:20.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1299999999999999e-07 2024-09-17T21:30:20.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709491+0000", 2024-09-17T21:30:20.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2089999999999999e-06 2024-09-17T21:30:20.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709498+0000", 2024-09-17T21:30:20.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3980000000000002e-06 2024-09-17T21:30:20.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709504+0000", 2024-09-17T21:30:20.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7219999999999996e-06 2024-09-17T21:30:20.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731499+0000", 2024-09-17T21:30:20.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021995297 2024-09-17T21:30:20.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731591+0000", 2024-09-17T21:30:20.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1373999999999996e-05 2024-09-17T21:30:20.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731668+0000", 2024-09-17T21:30:20.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7107999999999994e-05 2024-09-17T21:30:20.817 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.817 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.817 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.818 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50202 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'139, mlcod=45'137)", 2024-09-17T21:30:20.818 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:30:20.818 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.6112251830000002, 2024-09-17T21:30:20.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059659441000000001, 2024-09-17T21:30:20.818 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.818 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.818 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:20.818 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:30:20.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:30:20.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124224+0000", 2024-09-17T21:30:20.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5000000000000002e-06 2024-09-17T21:30:20.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147385+0000", 2024-09-17T21:30:20.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023160544000000002 2024-09-17T21:30:20.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147411+0000", 2024-09-17T21:30:20.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6095999999999999e-05 2024-09-17T21:30:20.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147438+0000", 2024-09-17T21:30:20.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6982999999999999e-05 2024-09-17T21:30:20.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147504+0000", 2024-09-17T21:30:20.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6156999999999997e-05 2024-09-17T21:30:20.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147531+0000", 2024-09-17T21:30:20.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6741999999999999e-05 2024-09-17T21:30:20.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:20.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183820+0000", 2024-09-17T21:30:20.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036289090000000003 2024-09-17T21:30:20.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183881+0000", 2024-09-17T21:30:20.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1328999999999995e-05 2024-09-17T21:30:20.822 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.822 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.822 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.822 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50203 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'141, mlcod=45'137)", 2024-09-17T21:30:20.822 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:30:20.822 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.5739624400000007, 2024-09-17T21:30:20.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023138823999999999, 2024-09-17T21:30:20.822 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:20.822 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:20.822 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:20.822 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:20.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:20.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:30:20.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:20.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:30:20.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:20.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:20.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161486+0000", 2024-09-17T21:30:20.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5290000000000001e-06 2024-09-17T21:30:20.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:20.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161496+0000", 2024-09-17T21:30:20.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0063e-05 2024-09-17T21:30:20.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:20.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161514+0000", 2024-09-17T21:30:20.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7728e-05 2024-09-17T21:30:20.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:20.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161524+0000", 2024-09-17T21:30:20.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0339e-05 2024-09-17T21:30:20.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:20.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170872+0000", 2024-09-17T21:30:20.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093473640000000004 2024-09-17T21:30:20.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:20.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170899+0000", 2024-09-17T21:30:20.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7321000000000002e-05 2024-09-17T21:30:20.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:20.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184580+0000", 2024-09-17T21:30:20.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013680988999999999 2024-09-17T21:30:20.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:20.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:20.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:20.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184623+0000", 2024-09-17T21:30:20.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3491000000000001e-05 2024-09-17T21:30:20.826 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.826 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.826 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.826 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:20.826 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:20.826 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:20.827 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone21 from 0xd327000 (overlap 0xd327000) to 0x0 2024-09-17T21:30:20.828 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:30:20.828 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:20.825+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:20.861 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-17T21:30:20.886 INFO:teuthology.orchestra.run.smithi129.stdout:2400 trunc from 0xd327000 to 0x1655609 2024-09-17T21:30:20.928 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:20.925+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:20.931 INFO:teuthology.orchestra.run.smithi129.stdout:2401 trunc from 0x1655609 to 0xc6bbb61 2024-09-17T21:30:20.938 INFO:teuthology.orchestra.run.smithi129.stdout:2403 punch from 0x33b5a07 to 0x33bdd0d, (0x8306 bytes) 2024-09-17T21:30:20.938 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:20.938 INFO:teuthology.orchestra.run.smithi129.stdout:2405 write 0x9be4b6b thru 0x9bed952 (0x8de8 bytes) 2024-09-17T21:30:20.941 INFO:teuthology.orchestra.run.smithi129.stdout:2407 punch from 0xb480d90 to 0xb489236, (0x84a6 bytes) 2024-09-17T21:30:20.953 INFO:teuthology.orchestra.run.smithi129.stdout:2408 trunc from 0xc6bbb61 to 0x264368b 2024-09-17T21:30:20.963 INFO:teuthology.orchestra.run.smithi129.stdout:2410 read 0x18b2880 thru 0x18bb1f4 (0x8975 bytes) 2024-09-17T21:30:20.973 INFO:teuthology.orchestra.run.smithi129.stdout:2412 write 0x7171357 thru 0x7179595 (0x823f bytes) 2024-09-17T21:30:20.980 INFO:teuthology.orchestra.run.smithi129.stdout:2413 trunc from 0x7179596 to 0x9d9d9f9 2024-09-17T21:30:20.982 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:20.982 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:20.983 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:20.983 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:20.984 INFO:teuthology.orchestra.run.smithi129.stdout:2414 write 0x8b3a6ab thru 0x8b3cf3e (0x2894 bytes) 2024-09-17T21:30:20.988 INFO:teuthology.orchestra.run.smithi129.stdout:2415 write 0x4b32512 thru 0x4b35bc9 (0x36b8 bytes) 2024-09-17T21:30:20.991 INFO:teuthology.orchestra.run.smithi129.stdout:2416 write 0xe56fc4 thru 0xe65165 (0xe1a2 bytes) 2024-09-17T21:30:20.992 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:30:20.993 INFO:teuthology.orchestra.run.smithi129.stdout:2418 write 0x6ac4a8e thru 0x6acdf4c (0x94bf bytes) 2024-09-17T21:30:20.995 INFO:teuthology.orchestra.run.smithi129.stdout:2419 read 0x323a4ad thru 0x324871f (0xe273 bytes) 2024-09-17T21:30:20.995 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:20.995 INFO:teuthology.orchestra.run.smithi129.stdout:2423 read 0x88b0fe7 thru 0x88b2f36 (0x1f50 bytes) 2024-09-17T21:30:20.995 INFO:teuthology.orchestra.run.smithi129.stdout:2428 read 0x162a86a thru 0x1634be6 (0xa37d bytes) 2024-09-17T21:30:21.006 INFO:teuthology.orchestra.run.smithi129.stdout:2429 write 0xdd2f5d8 thru 0xdd36525 (0x6f4e bytes) 2024-09-17T21:30:21.010 INFO:teuthology.orchestra.run.smithi129.stdout:2430 read 0x4e93671 thru 0x4ea1428 (0xddb8 bytes) 2024-09-17T21:30:21.010 INFO:teuthology.orchestra.run.smithi129.stdout:2431 read 0x5514299 thru 0x5516219 (0x1f81 bytes) 2024-09-17T21:30:21.010 INFO:teuthology.orchestra.run.smithi129.stdout:2432 read 0x28099c thru 0x282f45 (0x25aa bytes) 2024-09-17T21:30:21.010 INFO:teuthology.orchestra.run.smithi129.stdout:2434 read 0x9513742 thru 0x9516e53 (0x3712 bytes) 2024-09-17T21:30:21.010 INFO:teuthology.orchestra.run.smithi129.stdout:2435 write 0x240cb4f thru 0x2411f85 (0x5437 bytes) 2024-09-17T21:30:21.016 INFO:teuthology.orchestra.run.smithi129.stdout:2436 trunc from 0xdd36526 to 0xba9b511 2024-09-17T21:30:21.025 INFO:teuthology.orchestra.run.smithi129.stdout:2437 write 0x28aa4d8 thru 0x28b865a (0xe183 bytes) 2024-09-17T21:30:21.029 INFO:teuthology.orchestra.run.smithi129.stdout:2439 write 0x4267d15 thru 0x426b92d (0x3c19 bytes) 2024-09-17T21:30:21.036 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:21.034+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:21.039 INFO:teuthology.orchestra.run.smithi129.stdout:2440 trunc from 0xba9b511 to 0x53e5cce 2024-09-17T21:30:21.057 INFO:teuthology.orchestra.run.smithi129.stdout:2441 trunc from 0x53e5cce to 0xca76ff4 2024-09-17T21:30:21.062 INFO:teuthology.orchestra.run.smithi129.stdout:2442 punch from 0x37948dc to 0x37a2647, (0xdd6b bytes) 2024-09-17T21:30:21.062 INFO:teuthology.orchestra.run.smithi129.stdout:2443 read 0x296b0f2 thru 0x2976285 (0xb194 bytes) 2024-09-17T21:30:21.063 INFO:teuthology.orchestra.run.smithi129.stdout:2444 read 0xbe1a997 thru 0xbe20e1c (0x6486 bytes) 2024-09-17T21:30:21.063 INFO:teuthology.orchestra.run.smithi129.stdout:2445 read 0x90fc706 thru 0x91077e1 (0xb0dc bytes) 2024-09-17T21:30:21.063 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:21.063 INFO:teuthology.orchestra.run.smithi129.stdout:2449 read 0x9c46b42 thru 0x9c4cc61 (0x6120 bytes) 2024-09-17T21:30:21.063 INFO:teuthology.orchestra.run.smithi129.stdout:2450 write 0x15f8b49 thru 0x1608752 (0xfc0a bytes) 2024-09-17T21:30:21.064 INFO:teuthology.orchestra.run.smithi129.stdout:2451 write 0x5da783 thru 0x5e1ec2 (0x7740 bytes) 2024-09-17T21:30:21.078 INFO:teuthology.orchestra.run.smithi129.stdout:2452 trunc from 0xca76ff4 to 0xd5b6218 2024-09-17T21:30:21.083 INFO:teuthology.orchestra.run.smithi129.stdout:2453 punch from 0x116c014 to 0x117217d, (0x6169 bytes) 2024-09-17T21:30:21.083 INFO:teuthology.orchestra.run.smithi129.stdout:2454 punch from 0x2259002 to 0x2261c93, (0x8c91 bytes) 2024-09-17T21:30:21.083 INFO:teuthology.orchestra.run.smithi129.stdout:2455 trunc from 0xd5b6218 to 0xcdd32e5 2024-09-17T21:30:21.102 INFO:teuthology.orchestra.run.smithi129.stdout:2457 trunc from 0xcdd32e5 to 0xe87538 2024-09-17T21:30:21.103 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:21.104 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:21.104 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:21.104 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:21.104 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:21.114 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:30:21.129 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:21.127+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:21.131 INFO:teuthology.orchestra.run.smithi129.stdout:2458 write 0x78f6712 thru 0x78f67d0 (0xbf bytes) 2024-09-17T21:30:21.134 INFO:teuthology.orchestra.run.smithi129.stdout:2461 punch from 0x7100bb8 to 0x710c6a1, (0xbae9 bytes) 2024-09-17T21:30:21.140 INFO:teuthology.orchestra.run.smithi129.stdout:2462 writesame 0x84a8e8c thru 0x84aa5ec data_size 0x15(0x1761 bytes) 2024-09-17T21:30:21.147 INFO:teuthology.orchestra.run.smithi129.stdout:2463 trunc from 0x84aa5ed to 0x29987b5 2024-09-17T21:30:21.157 INFO:teuthology.orchestra.run.smithi129.stdout:2465 punch from 0x1b7128e to 0x1b7772f, (0x64a1 bytes) 2024-09-17T21:30:21.157 INFO:teuthology.orchestra.run.smithi129.stdout:2466 read 0x1f334c1 thru 0x1f3bf38 (0x8a78 bytes) 2024-09-17T21:30:21.170 INFO:teuthology.orchestra.run.smithi129.stdout:2467 write 0x9e2219c thru 0x9e29532 (0x7397 bytes) 2024-09-17T21:30:21.173 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:21.176 INFO:teuthology.orchestra.run.smithi129.stdout:2471 trunc from 0x9e29533 to 0xca7ecb2 2024-09-17T21:30:21.181 INFO:teuthology.orchestra.run.smithi129.stdout:2473 write 0x6f424c9 thru 0x6f462b6 (0x3dee bytes) 2024-09-17T21:30:21.184 INFO:teuthology.orchestra.run.smithi129.stdout:2474 punch from 0x1717e7f to 0x172363e, (0xb7bf bytes) 2024-09-17T21:30:21.184 INFO:teuthology.orchestra.run.smithi129.stdout:2475 read 0xb8939e4 thru 0xb899130 (0x574d bytes) 2024-09-17T21:30:21.185 INFO:teuthology.orchestra.run.smithi129.stdout:2476 write 0x180c1fb thru 0x181501c (0x8e22 bytes) 2024-09-17T21:30:21.192 INFO:teuthology.orchestra.run.smithi129.stdout:2477 read 0x3a544f2 thru 0x3a62d96 (0xe8a5 bytes) 2024-09-17T21:30:21.192 INFO:teuthology.orchestra.run.smithi129.stdout:2478 read 0x4462403 thru 0x446a351 (0x7f4f bytes) 2024-09-17T21:30:21.192 INFO:teuthology.orchestra.run.smithi129.stdout:2479 punch from 0x3bd0c to 0x49738, (0xda2c bytes) 2024-09-17T21:30:21.192 INFO:teuthology.orchestra.run.smithi129.stdout:2480 write 0x4a62aba thru 0x4a6ffcc (0xd513 bytes) 2024-09-17T21:30:21.203 INFO:teuthology.orchestra.run.smithi129.stdout:2481 write 0xe757f20 thru 0xe763985 (0xba66 bytes) 2024-09-17T21:30:21.207 INFO:teuthology.orchestra.run.smithi129.stdout:2482 write 0x7d49b3e thru 0x7d543ec (0xa8af bytes) 2024-09-17T21:30:21.214 INFO:teuthology.orchestra.run.smithi129.stdout:2483 punch from 0x6730c0c to 0x6738c65, (0x8059 bytes) 2024-09-17T21:30:21.214 INFO:teuthology.orchestra.run.smithi129.stdout:2485 write 0x736772d thru 0x7368471 (0xd45 bytes) 2024-09-17T21:30:21.216 INFO:teuthology.orchestra.run.smithi129.stdout:2486 write 0xae555ea thru 0xae5faca (0xa4e1 bytes) 2024-09-17T21:30:21.220 INFO:teuthology.orchestra.run.smithi129.stdout:2488 read 0x6077d43 thru 0x607b2a1 (0x355f bytes) 2024-09-17T21:30:21.220 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:21.230 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:21.228+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:21.235 INFO:teuthology.orchestra.run.smithi129.stdout:2492 trunc from 0xe763986 to 0x1d4bd20 2024-09-17T21:30:21.246 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:21.246 INFO:teuthology.orchestra.run.smithi129.stdout:2494 punch from 0x1aa9ad1 to 0x1ab0174, (0x66a3 bytes) 2024-09-17T21:30:21.258 INFO:teuthology.orchestra.run.smithi129.stdout:2495 write 0x7172a9e thru 0x71814f2 (0xea55 bytes) 2024-09-17T21:30:21.263 INFO:teuthology.orchestra.run.smithi129.stdout:2496 punch from 0x4a86dc9 to 0x4a9338b, (0xc5c2 bytes) 2024-09-17T21:30:21.263 INFO:teuthology.orchestra.run.smithi129.stdout:2497 read 0x2ce397e thru 0x2ce8633 (0x4cb6 bytes) 2024-09-17T21:30:21.263 INFO:teuthology.orchestra.run.smithi129.stdout:2499 read 0x598869b thru 0x59947d9 (0xc13f bytes) 2024-09-17T21:30:21.263 INFO:teuthology.orchestra.run.smithi129.stdout:2500 punch from 0x7311d6 to 0x73fdeb, (0xec15 bytes) 2024-09-17T21:30:21.263 INFO:teuthology.orchestra.run.smithi129.stdout:2503 clone 23 order 21 su 524288 sc 5 2024-09-17T21:30:21.267 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:21.267 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:21.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10799 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:30:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.199518781000002, 2024-09-17T21:30:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039122745, 2024-09-17T21:30:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10799 2024-09-17T21:30:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:30:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:30:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065469+0000", 2024-09-17T21:30:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:30:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065477+0000", 2024-09-17T21:30:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1319999999999994e-06 2024-09-17T21:30:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065483+0000", 2024-09-17T21:30:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2040000000000002e-06 2024-09-17T21:30:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065487+0000", 2024-09-17T21:30:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.42e-06 2024-09-17T21:30:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099984+0000", 2024-09-17T21:30:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034496484000000001 2024-09-17T21:30:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100065+0000", 2024-09-17T21:30:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1200999999999997e-05 2024-09-17T21:30:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100422+0000", 2024-09-17T21:30:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035657700000000001 2024-09-17T21:30:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104214+0000", 2024-09-17T21:30:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003792045 2024-09-17T21:30:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104505+0000", 2024-09-17T21:30:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029141399999999999 2024-09-17T21:30:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104556+0000", 2024-09-17T21:30:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1388999999999999e-05 2024-09-17T21:30:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104591+0000", 2024-09-17T21:30:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4455999999999997e-05 2024-09-17T21:30:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12187 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.125426050000002, 2024-09-17T21:30:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038825750999999999, 2024-09-17T21:30:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12187 2024-09-17T21:30:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:30:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139563+0000", 2024-09-17T21:30:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.624e-06 2024-09-17T21:30:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139569+0000", 2024-09-17T21:30:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1090000000000003e-06 2024-09-17T21:30:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139573+0000", 2024-09-17T21:30:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4429999999999996e-06 2024-09-17T21:30:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178206+0000", 2024-09-17T21:30:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038632376000000003 2024-09-17T21:30:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178282+0000", 2024-09-17T21:30:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6076999999999997e-05 2024-09-17T21:30:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178387+0000", 2024-09-17T21:30:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104682 2024-09-17T21:30:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12188 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:30:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.125407869, 2024-09-17T21:30:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039053826999999999, 2024-09-17T21:30:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12188 2024-09-17T21:30:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:30:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:30:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139746+0000", 2024-09-17T21:30:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000167281 2024-09-17T21:30:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139749+0000", 2024-09-17T21:30:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9660000000000001e-06 2024-09-17T21:30:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139757+0000", 2024-09-17T21:30:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8029999999999997e-06 2024-09-17T21:30:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139771+0000", 2024-09-17T21:30:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3541e-05 2024-09-17T21:30:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178415+0000", 2024-09-17T21:30:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038644803999999998 2024-09-17T21:30:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178517+0000", 2024-09-17T21:30:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101298 2024-09-17T21:30:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178633+0000", 2024-09-17T21:30:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116134 2024-09-17T21:30:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12189 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:30:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.125207392, 2024-09-17T21:30:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039135991000000002, 2024-09-17T21:30:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12189 2024-09-17T21:30:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:30:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:30:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139780+0000", 2024-09-17T21:30:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:30:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139782+0000", 2024-09-17T21:30:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1950000000000002e-06 2024-09-17T21:30:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139788+0000", 2024-09-17T21:30:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0639999999999999e-06 2024-09-17T21:30:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139792+0000", 2024-09-17T21:30:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1960000000000001e-06 2024-09-17T21:30:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178671+0000", 2024-09-17T21:30:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038878176 2024-09-17T21:30:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178778+0000", 2024-09-17T21:30:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107278 2024-09-17T21:30:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178915+0000", 2024-09-17T21:30:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013757900000000001 2024-09-17T21:30:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12190 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.125188209000001, 2024-09-17T21:30:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039343781000000001, 2024-09-17T21:30:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12190 2024-09-17T21:30:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139801+0000", 2024-09-17T21:30:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.916e-06 2024-09-17T21:30:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139807+0000", 2024-09-17T21:30:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4540000000000004e-06 2024-09-17T21:30:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139812+0000", 2024-09-17T21:30:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9729999999999997e-06 2024-09-17T21:30:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178960+0000", 2024-09-17T21:30:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039147567000000001 2024-09-17T21:30:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179051+0000", 2024-09-17T21:30:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0754000000000006e-05 2024-09-17T21:30:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179142+0000", 2024-09-17T21:30:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1712999999999994e-05 2024-09-17T21:30:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12191 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.125168350999999, 2024-09-17T21:30:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039464336000000003, 2024-09-17T21:30:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12191 2024-09-17T21:30:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:30:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139821+0000", 2024-09-17T21:30:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1260000000000001e-06 2024-09-17T21:30:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139827+0000", 2024-09-17T21:30:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T21:30:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139840+0000", 2024-09-17T21:30:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2607e-05 2024-09-17T21:30:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179162+0000", 2024-09-17T21:30:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039322863999999999 2024-09-17T21:30:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179204+0000", 2024-09-17T21:30:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1938999999999999e-05 2024-09-17T21:30:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179283+0000", 2024-09-17T21:30:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8547000000000005e-05 2024-09-17T21:30:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12192 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:30:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.125140639000001, 2024-09-17T21:30:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039652318999999998, 2024-09-17T21:30:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12192 2024-09-17T21:30:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:30:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:30:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141458+0000", 2024-09-17T21:30:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001612098 2024-09-17T21:30:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141469+0000", 2024-09-17T21:30:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-17T21:30:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141478+0000", 2024-09-17T21:30:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T21:30:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141485+0000", 2024-09-17T21:30:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4760000000000003e-06 2024-09-17T21:30:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179317+0000", 2024-09-17T21:30:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037831832000000003 2024-09-17T21:30:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179395+0000", 2024-09-17T21:30:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8349000000000005e-05 2024-09-17T21:30:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179499+0000", 2024-09-17T21:30:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103628 2024-09-17T21:30:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12257 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.122951831000002, 2024-09-17T21:30:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038862751000000001, 2024-09-17T21:30:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12257 2024-09-17T21:30:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T21:30:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142037+0000", 2024-09-17T21:30:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.03e-06 2024-09-17T21:30:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142043+0000", 2024-09-17T21:30:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6319999999999996e-06 2024-09-17T21:30:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142052+0000", 2024-09-17T21:30:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2709999999999998e-06 2024-09-17T21:30:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180794+0000", 2024-09-17T21:30:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038741758000000001 2024-09-17T21:30:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180843+0000", 2024-09-17T21:30:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9021999999999999e-05 2024-09-17T21:30:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180898+0000", 2024-09-17T21:30:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673000000000003e-05 2024-09-17T21:30:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12258 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.122928313999999, 2024-09-17T21:30:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038950717000000003, 2024-09-17T21:30:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12258 2024-09-17T21:30:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199999999999999e-07 2024-09-17T21:30:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142061+0000", 2024-09-17T21:30:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8279999999999999e-06 2024-09-17T21:30:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142067+0000", 2024-09-17T21:30:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5600000000000001e-06 2024-09-17T21:30:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142071+0000", 2024-09-17T21:30:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6240000000000001e-06 2024-09-17T21:30:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180921+0000", 2024-09-17T21:30:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038849366000000003 2024-09-17T21:30:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180961+0000", 2024-09-17T21:30:21.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0219000000000002e-05 2024-09-17T21:30:21.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181009+0000", 2024-09-17T21:30:21.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8437999999999999e-05 2024-09-17T21:30:21.312 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.312 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.312 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.312 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12259 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.122901067000001, 2024-09-17T21:30:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039116832999999997, 2024-09-17T21:30:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12259 2024-09-17T21:30:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2200000000000004e-07 2024-09-17T21:30:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142088+0000", 2024-09-17T21:30:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0329999999999998e-06 2024-09-17T21:30:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142095+0000", 2024-09-17T21:30:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4320000000000004e-06 2024-09-17T21:30:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142100+0000", 2024-09-17T21:30:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:30:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181037+0000", 2024-09-17T21:30:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038937601000000002 2024-09-17T21:30:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181114+0000", 2024-09-17T21:30:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6857e-05 2024-09-17T21:30:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181203+0000", 2024-09-17T21:30:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8437999999999996e-05 2024-09-17T21:30:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12260 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:30:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.122880780000003, 2024-09-17T21:30:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039204184000000003, 2024-09-17T21:30:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12260 2024-09-17T21:30:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:30:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:30:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142107+0000", 2024-09-17T21:30:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:30:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142108+0000", 2024-09-17T21:30:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.753e-06 2024-09-17T21:30:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142114+0000", 2024-09-17T21:30:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7660000000000004e-06 2024-09-17T21:30:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142119+0000", 2024-09-17T21:30:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:30:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181216+0000", 2024-09-17T21:30:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039097191000000003 2024-09-17T21:30:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181260+0000", 2024-09-17T21:30:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4731000000000003e-05 2024-09-17T21:30:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181310+0000", 2024-09-17T21:30:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9818000000000003e-05 2024-09-17T21:30:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12261 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:30:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.122862456, 2024-09-17T21:30:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039292200999999999, 2024-09-17T21:30:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12261 2024-09-17T21:30:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:30:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:30:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142125+0000", 2024-09-17T21:30:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:30:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142127+0000", 2024-09-17T21:30:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.792e-06 2024-09-17T21:30:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142132+0000", 2024-09-17T21:30:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3979999999999998e-06 2024-09-17T21:30:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142136+0000", 2024-09-17T21:30:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.211e-06 2024-09-17T21:30:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181324+0000", 2024-09-17T21:30:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039187560000000003 2024-09-17T21:30:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181363+0000", 2024-09-17T21:30:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9375000000000002e-05 2024-09-17T21:30:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181417+0000", 2024-09-17T21:30:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3372000000000003e-05 2024-09-17T21:30:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12262 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:30:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.122844663000002, 2024-09-17T21:30:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039384875, 2024-09-17T21:30:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12262 2024-09-17T21:30:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:30:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:30:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142143+0000", 2024-09-17T21:30:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:30:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142148+0000", 2024-09-17T21:30:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4360000000000001e-06 2024-09-17T21:30:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142154+0000", 2024-09-17T21:30:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8719999999999999e-06 2024-09-17T21:30:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142158+0000", 2024-09-17T21:30:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.476e-06 2024-09-17T21:30:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181436+0000", 2024-09-17T21:30:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039277919000000001 2024-09-17T21:30:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181480+0000", 2024-09-17T21:30:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3649999999999997e-05 2024-09-17T21:30:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181527+0000", 2024-09-17T21:30:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7131e-05 2024-09-17T21:30:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12263 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1079296~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.122822227999997, 2024-09-17T21:30:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039473339000000003, 2024-09-17T21:30:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12263 2024-09-17T21:30:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:30:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142167+0000", 2024-09-17T21:30:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.81e-06 2024-09-17T21:30:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142172+0000", 2024-09-17T21:30:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:30:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142177+0000", 2024-09-17T21:30:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T21:30:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181553+0000", 2024-09-17T21:30:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039376130000000002 2024-09-17T21:30:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181592+0000", 2024-09-17T21:30:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9153999999999998e-05 2024-09-17T21:30:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181638+0000", 2024-09-17T21:30:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5864e-05 2024-09-17T21:30:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12278 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1095680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.122804252999998, 2024-09-17T21:30:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039454222999999997, 2024-09-17T21:30:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12278 2024-09-17T21:30:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:30:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142185+0000", 2024-09-17T21:30:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.697e-06 2024-09-17T21:30:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142191+0000", 2024-09-17T21:30:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:30:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142196+0000", 2024-09-17T21:30:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4829999999999999e-06 2024-09-17T21:30:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181548+0000", 2024-09-17T21:30:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039351200000000003 2024-09-17T21:30:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181591+0000", 2024-09-17T21:30:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3492000000000003e-05 2024-09-17T21:30:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181637+0000", 2024-09-17T21:30:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5847000000000003e-05 2024-09-17T21:30:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12279 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.122780964, 2024-09-17T21:30:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039539801999999999, 2024-09-17T21:30:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12279 2024-09-17T21:30:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.89e-07 2024-09-17T21:30:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142208+0000", 2024-09-17T21:30:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7579999999999999e-06 2024-09-17T21:30:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142214+0000", 2024-09-17T21:30:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:30:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142218+0000", 2024-09-17T21:30:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3429999999999998e-06 2024-09-17T21:30:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181653+0000", 2024-09-17T21:30:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039435322000000002 2024-09-17T21:30:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181698+0000", 2024-09-17T21:30:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4978999999999997e-05 2024-09-17T21:30:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181746+0000", 2024-09-17T21:30:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571999999999998e-05 2024-09-17T21:30:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12280 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1124352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:30:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.122763444999997, 2024-09-17T21:30:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039704358000000002, 2024-09-17T21:30:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12280 2024-09-17T21:30:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:30:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:30:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142224+0000", 2024-09-17T21:30:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:30:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142226+0000", 2024-09-17T21:30:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:30:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142231+0000", 2024-09-17T21:30:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:30:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142240+0000", 2024-09-17T21:30:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4590000000000004e-06 2024-09-17T21:30:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181774+0000", 2024-09-17T21:30:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039533833999999997 2024-09-17T21:30:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181859+0000", 2024-09-17T21:30:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4328999999999998e-05 2024-09-17T21:30:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181928+0000", 2024-09-17T21:30:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9139000000000003e-05 2024-09-17T21:30:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12281 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1136640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:30:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.122740538999999, 2024-09-17T21:30:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039787799999999998, 2024-09-17T21:30:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12281 2024-09-17T21:30:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:30:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:30:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142309+0000", 2024-09-17T21:30:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2323999999999995e-05 2024-09-17T21:30:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142317+0000", 2024-09-17T21:30:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1859999999999992e-06 2024-09-17T21:30:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142324+0000", 2024-09-17T21:30:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4699999999999996e-06 2024-09-17T21:30:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142330+0000", 2024-09-17T21:30:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4650000000000002e-06 2024-09-17T21:30:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181948+0000", 2024-09-17T21:30:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039618056999999998 2024-09-17T21:30:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181986+0000", 2024-09-17T21:30:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8442999999999998e-05 2024-09-17T21:30:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182034+0000", 2024-09-17T21:30:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7855000000000002e-05 2024-09-17T21:30:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12282 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1150976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.122648310000002, 2024-09-17T21:30:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039871919999999998, 2024-09-17T21:30:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12282 2024-09-17T21:30:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:30:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142341+0000", 2024-09-17T21:30:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.232e-06 2024-09-17T21:30:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142348+0000", 2024-09-17T21:30:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7320000000000001e-06 2024-09-17T21:30:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142353+0000", 2024-09-17T21:30:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:30:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182062+0000", 2024-09-17T21:30:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039709739000000001 2024-09-17T21:30:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182128+0000", 2024-09-17T21:30:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5739000000000002e-05 2024-09-17T21:30:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182210+0000", 2024-09-17T21:30:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:30:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12283 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:30:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.122627899000001, 2024-09-17T21:30:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039896434000000001, 2024-09-17T21:30:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:30:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12283 2024-09-17T21:30:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:30:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:30:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144852+0000", 2024-09-17T21:30:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024934079999999999 2024-09-17T21:30:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144856+0000", 2024-09-17T21:30:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:30:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144864+0000", 2024-09-17T21:30:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0160000000000008e-06 2024-09-17T21:30:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144869+0000", 2024-09-17T21:30:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:30:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182246+0000", 2024-09-17T21:30:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037377207000000003 2024-09-17T21:30:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182255+0000", 2024-09-17T21:30:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1840000000000005e-06 2024-09-17T21:30:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.367 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:21.368 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:30:21.369 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:21.366+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:21.468 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:21.466+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:21.512 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:21.513 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:21.513 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:21.513 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:21.521 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:30:21.605 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:21.603+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:21.665 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:21.665 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:21.665 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:21.665 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:21.665 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:21.671 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:21.668+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:21.673 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:30:21.698 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:30:21.771 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:21.769+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:21.778 INFO:teuthology.orchestra.run.smithi129.stdout:flattening image image_client.0-clone22 2024-09-17T21:30:21.828 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:21.828 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:21.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:21.828 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:21.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.828 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:36935 3.a 3:54c8759b:::rbd_data.1120e62eea89.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-17T21:30:21.828 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:21.828 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 24.866402062999999, 2024-09-17T21:30:21.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020540151999999999, 2024-09-17T21:30:21.828 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.828 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.829 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:21.829 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.829 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.829 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.829 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 36935 2024-09-17T21:30:21.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.829 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:21.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:21.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958910+0000", 2024-09-17T21:30:21.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:30:21.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958918+0000", 2024-09-17T21:30:21.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6469999999999998e-06 2024-09-17T21:30:21.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958933+0000", 2024-09-17T21:30:21.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4791e-05 2024-09-17T21:30:21.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958940+0000", 2024-09-17T21:30:21.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-17T21:30:21.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958990+0000", 2024-09-17T21:30:21.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9759000000000003e-05 2024-09-17T21:30:21.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.959039+0000", 2024-09-17T21:30:21.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9487000000000002e-05 2024-09-17T21:30:21.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:21.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.962098+0000", 2024-09-17T21:30:21.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003059258 2024-09-17T21:30:21.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:21.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979325+0000", 2024-09-17T21:30:21.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017226841 2024-09-17T21:30:21.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:21.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979343+0000", 2024-09-17T21:30:21.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7486000000000002e-05 2024-09-17T21:30:21.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:21.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979401+0000", 2024-09-17T21:30:21.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7980999999999999e-05 2024-09-17T21:30:21.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979449+0000", 2024-09-17T21:30:21.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8639e-05 2024-09-17T21:30:21.834 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.834 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.834 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.834 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39447 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 909312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.834 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:21.834 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.857464163, 2024-09-17T21:30:21.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017419153999999999, 2024-09-17T21:30:21.834 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.834 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.834 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.834 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.835 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.835 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.835 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39447 2024-09-17T21:30:21.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.835 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:21.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:21.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:21.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:30:21.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967849+0000", 2024-09-17T21:30:21.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4339999999999999e-06 2024-09-17T21:30:21.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967853+0000", 2024-09-17T21:30:21.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:30:21.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967857+0000", 2024-09-17T21:30:21.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:30:21.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985193+0000", 2024-09-17T21:30:21.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017335973000000001 2024-09-17T21:30:21.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985228+0000", 2024-09-17T21:30:21.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5154999999999998e-05 2024-09-17T21:30:21.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985266+0000", 2024-09-17T21:30:21.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8448000000000002e-05 2024-09-17T21:30:21.838 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.838 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.838 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.839 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39448 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 974848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.839 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:21.839 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.857451347000001, 2024-09-17T21:30:21.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017517066000000001, 2024-09-17T21:30:21.839 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.839 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.839 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.839 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.839 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.839 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.839 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39448 2024-09-17T21:30:21.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.839 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:21.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:21.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:21.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.72e-07 2024-09-17T21:30:21.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967861+0000", 2024-09-17T21:30:21.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.096e-06 2024-09-17T21:30:21.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967865+0000", 2024-09-17T21:30:21.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9339999999999999e-06 2024-09-17T21:30:21.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967868+0000", 2024-09-17T21:30:21.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.508e-06 2024-09-17T21:30:21.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985285+0000", 2024-09-17T21:30:21.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017417208 2024-09-17T21:30:21.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985339+0000", 2024-09-17T21:30:21.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3931000000000003e-05 2024-09-17T21:30:21.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985377+0000", 2024-09-17T21:30:21.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8016999999999999e-05 2024-09-17T21:30:21.843 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.843 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.843 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.843 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39449 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.843 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:21.843 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.857439237000001, 2024-09-17T21:30:21.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017595137, 2024-09-17T21:30:21.843 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.843 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.843 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.843 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.843 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.843 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.844 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39449 2024-09-17T21:30:21.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.844 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:21.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:21.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:21.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2700000000000001e-07 2024-09-17T21:30:21.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967873+0000", 2024-09-17T21:30:21.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.164e-06 2024-09-17T21:30:21.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967877+0000", 2024-09-17T21:30:21.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.23e-06 2024-09-17T21:30:21.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967880+0000", 2024-09-17T21:30:21.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.286e-06 2024-09-17T21:30:21.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985397+0000", 2024-09-17T21:30:21.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017516982 2024-09-17T21:30:21.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985431+0000", 2024-09-17T21:30:21.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3899999999999997e-05 2024-09-17T21:30:21.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985467+0000", 2024-09-17T21:30:21.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6347999999999998e-05 2024-09-17T21:30:21.847 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.847 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.847 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.848 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39450 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1105920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.848 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:21.848 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.857418979999998, 2024-09-17T21:30:21.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017677824000000002, 2024-09-17T21:30:21.848 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.848 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.848 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.848 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.848 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.848 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.848 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39450 2024-09-17T21:30:21.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.849 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:21.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:21.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967893+0000", 2024-09-17T21:30:21.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9100000000000004e-07 2024-09-17T21:30:21.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967895+0000", 2024-09-17T21:30:21.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:30:21.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967901+0000", 2024-09-17T21:30:21.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.083e-06 2024-09-17T21:30:21.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967906+0000", 2024-09-17T21:30:21.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:30:21.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985493+0000", 2024-09-17T21:30:21.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017587565999999999 2024-09-17T21:30:21.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985531+0000", 2024-09-17T21:30:21.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7490000000000002e-05 2024-09-17T21:30:21.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985570+0000", 2024-09-17T21:30:21.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9205000000000002e-05 2024-09-17T21:30:21.851 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.852 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.852 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.852 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39451 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1171456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.852 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:21.852 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.8573986, 2024-09-17T21:30:21.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017752337999999999, 2024-09-17T21:30:21.852 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.852 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.852 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.852 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.852 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.852 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.853 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39451 2024-09-17T21:30:21.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.853 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:21.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:21.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:21.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:30:21.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967915+0000", 2024-09-17T21:30:21.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780000000000001e-06 2024-09-17T21:30:21.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967922+0000", 2024-09-17T21:30:21.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7020000000000004e-06 2024-09-17T21:30:21.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967926+0000", 2024-09-17T21:30:21.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.809e-06 2024-09-17T21:30:21.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985590+0000", 2024-09-17T21:30:21.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017663913999999999 2024-09-17T21:30:21.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985627+0000", 2024-09-17T21:30:21.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6896e-05 2024-09-17T21:30:21.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985665+0000", 2024-09-17T21:30:21.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7793999999999998e-05 2024-09-17T21:30:21.856 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.856 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.856 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.856 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39452 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1236992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.856 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:21.856 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.857378658000002, 2024-09-17T21:30:21.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017829602999999999, 2024-09-17T21:30:21.856 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.857 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.857 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.857 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.857 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.857 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.857 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39452 2024-09-17T21:30:21.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.857 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:21.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:21.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:21.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:30:21.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967947+0000", 2024-09-17T21:30:21.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4280999999999999e-05 2024-09-17T21:30:21.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967954+0000", 2024-09-17T21:30:21.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3249999999999996e-06 2024-09-17T21:30:21.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967959+0000", 2024-09-17T21:30:21.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2299999999999999e-06 2024-09-17T21:30:21.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985687+0000", 2024-09-17T21:30:21.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017727844999999999 2024-09-17T21:30:21.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985725+0000", 2024-09-17T21:30:21.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8243e-05 2024-09-17T21:30:21.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985762+0000", 2024-09-17T21:30:21.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7023000000000001e-05 2024-09-17T21:30:21.860 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.860 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.860 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.860 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39453 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.860 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:21.860 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.857342169999999, 2024-09-17T21:30:21.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017893793000000002, 2024-09-17T21:30:21.861 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.861 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.861 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.861 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.861 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.861 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.861 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39453 2024-09-17T21:30:21.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.861 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:21.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:21.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967970+0000", 2024-09-17T21:30:21.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:30:21.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967972+0000", 2024-09-17T21:30:21.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0229999999999999e-06 2024-09-17T21:30:21.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967978+0000", 2024-09-17T21:30:21.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0430000000000004e-06 2024-09-17T21:30:21.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967982+0000", 2024-09-17T21:30:21.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2570000000000001e-06 2024-09-17T21:30:21.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985782+0000", 2024-09-17T21:30:21.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017799683 2024-09-17T21:30:21.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985818+0000", 2024-09-17T21:30:21.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6304999999999999e-05 2024-09-17T21:30:21.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985863+0000", 2024-09-17T21:30:21.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4953999999999997e-05 2024-09-17T21:30:21.865 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.865 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.865 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.865 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39454 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1368064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.865 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:21.865 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.857322841999999, 2024-09-17T21:30:21.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017968886, 2024-09-17T21:30:21.865 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.865 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.866 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.866 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.866 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.866 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.866 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39454 2024-09-17T21:30:21.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.866 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:21.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:21.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967989+0000", 2024-09-17T21:30:21.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.15e-07 2024-09-17T21:30:21.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967991+0000", 2024-09-17T21:30:21.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.945e-06 2024-09-17T21:30:21.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967997+0000", 2024-09-17T21:30:21.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0190000000000003e-06 2024-09-17T21:30:21.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968001+0000", 2024-09-17T21:30:21.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6650000000000002e-06 2024-09-17T21:30:21.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985883+0000", 2024-09-17T21:30:21.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017881971999999999 2024-09-17T21:30:21.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985920+0000", 2024-09-17T21:30:21.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-05 2024-09-17T21:30:21.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985957+0000", 2024-09-17T21:30:21.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7370000000000003e-05 2024-09-17T21:30:21.869 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.870 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.870 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.870 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39455 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.870 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:21.870 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.857303668, 2024-09-17T21:30:21.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018044437999999999, 2024-09-17T21:30:21.870 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.870 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.870 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.870 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.870 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.871 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.871 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39455 2024-09-17T21:30:21.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.871 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:21.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:21.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:21.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:30:21.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968010+0000", 2024-09-17T21:30:21.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.886e-06 2024-09-17T21:30:21.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968017+0000", 2024-09-17T21:30:21.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:30:21.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968021+0000", 2024-09-17T21:30:21.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850000000000004e-06 2024-09-17T21:30:21.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985977+0000", 2024-09-17T21:30:21.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017956368 2024-09-17T21:30:21.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986015+0000", 2024-09-17T21:30:21.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7610000000000001e-05 2024-09-17T21:30:21.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986052+0000", 2024-09-17T21:30:21.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7109e-05 2024-09-17T21:30:21.874 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.874 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.874 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39456 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1499136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.857283368000001, 2024-09-17T21:30:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018114514000000002, 2024-09-17T21:30:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39456 2024-09-17T21:30:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-17T21:30:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968030+0000", 2024-09-17T21:30:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0049999999999999e-06 2024-09-17T21:30:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968036+0000", 2024-09-17T21:30:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0129999999999999e-06 2024-09-17T21:30:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968041+0000", 2024-09-17T21:30:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8300000000000003e-06 2024-09-17T21:30:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986071+0000", 2024-09-17T21:30:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018029620999999999 2024-09-17T21:30:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986106+0000", 2024-09-17T21:30:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4867999999999999e-05 2024-09-17T21:30:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986142+0000", 2024-09-17T21:30:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6566000000000002e-05 2024-09-17T21:30:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39457 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1564672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.857263551999999, 2024-09-17T21:30:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018188644, 2024-09-17T21:30:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39457 2024-09-17T21:30:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7199999999999999e-07 2024-09-17T21:30:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968050+0000", 2024-09-17T21:30:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1110000000000002e-06 2024-09-17T21:30:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968057+0000", 2024-09-17T21:30:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6170000000000003e-06 2024-09-17T21:30:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968062+0000", 2024-09-17T21:30:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:30:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986163+0000", 2024-09-17T21:30:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018101435999999999 2024-09-17T21:30:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986199+0000", 2024-09-17T21:30:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.591e-05 2024-09-17T21:30:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986236+0000", 2024-09-17T21:30:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7416999999999998e-05 2024-09-17T21:30:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39458 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1630208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.857236612000001, 2024-09-17T21:30:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018255856000000001, 2024-09-17T21:30:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39458 2024-09-17T21:30:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:30:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968077+0000", 2024-09-17T21:30:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.779e-06 2024-09-17T21:30:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968081+0000", 2024-09-17T21:30:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.532e-06 2024-09-17T21:30:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968085+0000", 2024-09-17T21:30:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999999e-06 2024-09-17T21:30:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986256+0000", 2024-09-17T21:30:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018171273000000002 2024-09-17T21:30:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986293+0000", 2024-09-17T21:30:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6534e-05 2024-09-17T21:30:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986330+0000", 2024-09-17T21:30:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7536000000000001e-05 2024-09-17T21:30:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39459 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1695744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.857222033999999, 2024-09-17T21:30:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334493, 2024-09-17T21:30:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39459 2024-09-17T21:30:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968090+0000", 2024-09-17T21:30:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:30:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968091+0000", 2024-09-17T21:30:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3319999999999999e-06 2024-09-17T21:30:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968094+0000", 2024-09-17T21:30:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2590000000000001e-06 2024-09-17T21:30:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968097+0000", 2024-09-17T21:30:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:30:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986350+0000", 2024-09-17T21:30:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018253364000000001 2024-09-17T21:30:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986387+0000", 2024-09-17T21:30:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6350999999999998e-05 2024-09-17T21:30:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986424+0000", 2024-09-17T21:30:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6897999999999998e-05 2024-09-17T21:30:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39460 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1761280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.857210898000002, 2024-09-17T21:30:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018415918999999999, 2024-09-17T21:30:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39460 2024-09-17T21:30:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968101+0000", 2024-09-17T21:30:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T21:30:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968102+0000", 2024-09-17T21:30:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1689999999999999e-06 2024-09-17T21:30:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968105+0000", 2024-09-17T21:30:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:30:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968108+0000", 2024-09-17T21:30:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6630000000000002e-06 2024-09-17T21:30:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986442+0000", 2024-09-17T21:30:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334487 2024-09-17T21:30:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986479+0000", 2024-09-17T21:30:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6557000000000003e-05 2024-09-17T21:30:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986516+0000", 2024-09-17T21:30:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7648999999999998e-05 2024-09-17T21:30:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39461 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1826816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.857200138, 2024-09-17T21:30:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018499768, 2024-09-17T21:30:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39461 2024-09-17T21:30:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4699999999999998e-07 2024-09-17T21:30:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968112+0000", 2024-09-17T21:30:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-06 2024-09-17T21:30:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968116+0000", 2024-09-17T21:30:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1080000000000001e-06 2024-09-17T21:30:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968122+0000", 2024-09-17T21:30:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1070000000000001e-06 2024-09-17T21:30:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986537+0000", 2024-09-17T21:30:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018414834000000001 2024-09-17T21:30:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986572+0000", 2024-09-17T21:30:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5726000000000003e-05 2024-09-17T21:30:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986611+0000", 2024-09-17T21:30:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8584000000000002e-05 2024-09-17T21:30:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39462 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1892352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.857185774000001, 2024-09-17T21:30:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018583252000000001, 2024-09-17T21:30:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39462 2024-09-17T21:30:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968126+0000", 2024-09-17T21:30:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968127+0000", 2024-09-17T21:30:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.229e-06 2024-09-17T21:30:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968131+0000", 2024-09-17T21:30:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8010000000000001e-06 2024-09-17T21:30:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968134+0000", 2024-09-17T21:30:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8880000000000001e-06 2024-09-17T21:30:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986634+0000", 2024-09-17T21:30:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018500501999999999 2024-09-17T21:30:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986671+0000", 2024-09-17T21:30:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6881000000000001e-05 2024-09-17T21:30:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986709+0000", 2024-09-17T21:30:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7546999999999998e-05 2024-09-17T21:30:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39463 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1957888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.857174158999999, 2024-09-17T21:30:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018667186999999998, 2024-09-17T21:30:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39463 2024-09-17T21:30:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:30:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968138+0000", 2024-09-17T21:30:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:30:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968142+0000", 2024-09-17T21:30:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.145e-06 2024-09-17T21:30:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968144+0000", 2024-09-17T21:30:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6869999999999999e-06 2024-09-17T21:30:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986728+0000", 2024-09-17T21:30:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018584163000000001 2024-09-17T21:30:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986766+0000", 2024-09-17T21:30:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7987000000000001e-05 2024-09-17T21:30:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986804+0000", 2024-09-17T21:30:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7892000000000002e-05 2024-09-17T21:30:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39464 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.857163849999999, 2024-09-17T21:30:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018753723999999999, 2024-09-17T21:30:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39464 2024-09-17T21:30:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968148+0000", 2024-09-17T21:30:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:30:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968152+0000", 2024-09-17T21:30:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3749999999999996e-06 2024-09-17T21:30:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968155+0000", 2024-09-17T21:30:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.371e-06 2024-09-17T21:30:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968158+0000", 2024-09-17T21:30:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T21:30:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986824+0000", 2024-09-17T21:30:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018665779 2024-09-17T21:30:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986863+0000", 2024-09-17T21:30:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9273000000000002e-05 2024-09-17T21:30:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986901+0000", 2024-09-17T21:30:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7978000000000001e-05 2024-09-17T21:30:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39465 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.857148451, 2024-09-17T21:30:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018825341999999998, 2024-09-17T21:30:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39465 2024-09-17T21:30:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-17T21:30:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968164+0000", 2024-09-17T21:30:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1289999999999999e-06 2024-09-17T21:30:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968167+0000", 2024-09-17T21:30:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1379999999999999e-06 2024-09-17T21:30:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968170+0000", 2024-09-17T21:30:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:30:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986914+0000", 2024-09-17T21:30:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018743829 2024-09-17T21:30:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986950+0000", 2024-09-17T21:30:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5819999999999999e-05 2024-09-17T21:30:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986988+0000", 2024-09-17T21:30:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8362000000000003e-05 2024-09-17T21:30:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.922 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.922 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:21.922 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:21.922 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:21.923 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-17T21:30:21.923 INFO:teuthology.orchestra.run.smithi129.stdout:2504 read 0x6931e1b thru 0x693385b (0x1a41 bytes) 2024-09-17T21:30:21.924 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:21.924 INFO:teuthology.orchestra.run.smithi129.stdout:2506 trunc from 0x71814f3 to 0x6a95a64 2024-09-17T21:30:21.924 INFO:teuthology.orchestra.run.smithi129.stdout:2509 trunc from 0x6a95a64 to 0x7b4499f 2024-09-17T21:30:21.924 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:30:21.924 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:21.921+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:21.925 INFO:teuthology.orchestra.run.smithi129.stdout:2510 read 0x105ce67 thru 0x1064cc1 (0x7e5b bytes) 2024-09-17T21:30:21.936 INFO:teuthology.orchestra.run.smithi129.stdout:2513 write 0xc9b701e thru 0xc9c2d7d (0xbd60 bytes) 2024-09-17T21:30:21.939 INFO:teuthology.orchestra.run.smithi129.stdout:2515 write 0x2529848 thru 0x252c99b (0x3154 bytes) 2024-09-17T21:30:21.942 INFO:teuthology.orchestra.run.smithi129.stdout:2516 writesame 0x75d467f thru 0x75d7abd data_size 0x19(0x343f bytes) 2024-09-17T21:30:21.945 INFO:teuthology.orchestra.run.smithi129.stdout:2517 punch from 0x7778d55 to 0x777bf21, (0x31cc bytes) 2024-09-17T21:30:21.945 INFO:teuthology.orchestra.run.smithi129.stdout:2519 read 0x3166b2e thru 0x31756c3 (0xeb96 bytes) 2024-09-17T21:30:21.945 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:21.945 INFO:teuthology.orchestra.run.smithi129.stdout:2521 write 0x71be51f thru 0x71cdff3 (0xfad5 bytes) 2024-09-17T21:30:21.956 INFO:teuthology.orchestra.run.smithi129.stdout:2522 write 0xeca729f thru 0xecabb9c (0x48fe bytes) 2024-09-17T21:30:21.959 INFO:teuthology.orchestra.run.smithi129.stdout:2523 read 0xa23ac5a thru 0xa241eeb (0x7292 bytes) 2024-09-17T21:30:21.959 INFO:teuthology.orchestra.run.smithi129.stdout:2524 punch from 0xbc2f293 to 0xbc39ce4, (0xaa51 bytes) 2024-09-17T21:30:21.973 INFO:teuthology.orchestra.run.smithi129.stdout:2526 trunc from 0xecabb9d to 0x28f97b9 2024-09-17T21:30:21.993 INFO:teuthology.orchestra.run.smithi129.stdout:2527 punch from 0x24d08ae to 0x24d6af2, (0x6244 bytes) 2024-09-17T21:30:21.993 INFO:teuthology.orchestra.run.smithi129.stdout:2528 read 0x45956b thru 0x45bba2 (0x2638 bytes) 2024-09-17T21:30:22.014 INFO:teuthology.orchestra.run.smithi129.stdout:2529 write 0xd8ad77a thru 0xd8b9278 (0xbaff bytes) 2024-09-17T21:30:22.017 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:22.017 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:22.017 INFO:teuthology.orchestra.run.smithi129.stdout:2534 read 0x4a2d3e7 thru 0x4a3105f (0x3c79 bytes) 2024-09-17T21:30:22.017 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:22.017 INFO:teuthology.orchestra.run.smithi129.stdout:2537 punch from 0x7ecd80d to 0x7ed17ca, (0x3fbd bytes) 2024-09-17T21:30:22.017 INFO:teuthology.orchestra.run.smithi129.stdout:2538 writesame 0x1776e0f thru 0x178451b data_size 0x9(0xd70d bytes) 2024-09-17T21:30:22.023 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:22.021+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:22.024 INFO:teuthology.orchestra.run.smithi129.stdout:2539 write 0xb490134 thru 0xb49b347 (0xb214 bytes) 2024-09-17T21:30:22.034 INFO:teuthology.orchestra.run.smithi129.stdout:2540 write 0xdc22cec thru 0xdc23d0c (0x1021 bytes) 2024-09-17T21:30:22.037 INFO:teuthology.orchestra.run.smithi129.stdout:2541 read 0x323af24 thru 0x323ee20 (0x3efd bytes) 2024-09-17T21:30:22.039 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:22.036+0000 7f7a098ed640 -1 Falling back to public interface 2024-09-17T21:30:22.043 INFO:teuthology.orchestra.run.smithi129.stdout:2544 write 0xe0d9d9a thru 0xe0e5681 (0xb8e8 bytes) 2024-09-17T21:30:22.046 INFO:teuthology.orchestra.run.smithi129.stdout:2546 punch from 0xcad0385 to 0xcad05ad, (0x228 bytes) 2024-09-17T21:30:22.046 INFO:teuthology.orchestra.run.smithi129.stdout:2548 write 0x227a586 thru 0x2289a1d (0xf498 bytes) 2024-09-17T21:30:22.049 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:22.050 INFO:teuthology.orchestra.run.smithi129.stdout:2551 read 0x29ac343 thru 0x29ac4fd (0x1bb bytes) 2024-09-17T21:30:22.050 INFO:teuthology.orchestra.run.smithi129.stdout:2552 read 0x6704b3f thru 0x6709ae1 (0x4fa3 bytes) 2024-09-17T21:30:22.050 INFO:teuthology.orchestra.run.smithi129.stdout:2553 read 0x929262d thru 0x92a0dd3 (0xe7a7 bytes) 2024-09-17T21:30:22.050 INFO:teuthology.orchestra.run.smithi129.stdout:2555 write 0x9c589ab thru 0x9c61a41 (0x9097 bytes) 2024-09-17T21:30:22.052 INFO:teuthology.orchestra.run.smithi129.stdout:2557 punch from 0xdf5c0 to 0xe1b34, (0x2574 bytes) 2024-09-17T21:30:22.052 INFO:teuthology.orchestra.run.smithi129.stdout:2558 read 0x1800c1b thru 0x180fc43 (0xf029 bytes) 2024-09-17T21:30:22.053 INFO:teuthology.orchestra.run.smithi129.stdout:2559 read 0x4c718cc thru 0x4c791b4 (0x78e9 bytes) 2024-09-17T21:30:22.053 INFO:teuthology.orchestra.run.smithi129.stdout:2560 write 0x19c175f thru 0x19c726c (0x5b0e bytes) 2024-09-17T21:30:22.055 INFO:teuthology.orchestra.run.smithi129.stdout:2562 write 0x3f4bdec thru 0x3f553c8 (0x95dd bytes) 2024-09-17T21:30:22.062 INFO:teuthology.orchestra.run.smithi129.stdout:2566 trunc from 0xe0e5682 to 0x9c90730 2024-09-17T21:30:22.076 INFO:teuthology.orchestra.run.smithi129.stdout:2567 read 0x9c74fc6 thru 0x9c751f3 (0x22e bytes) 2024-09-17T21:30:22.076 INFO:teuthology.orchestra.run.smithi129.stdout:2569 read 0x5b09618 thru 0x5b0ee31 (0x581a bytes) 2024-09-17T21:30:22.077 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:22.077 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:22.077 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:22.077 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:22.086 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:30:22.087 INFO:teuthology.orchestra.run.smithi129.stdout:2570 writesame 0xbc701ee thru 0xbc7e3bc data_size 0x9(0xe1cf bytes) 2024-09-17T21:30:22.099 INFO:teuthology.orchestra.run.smithi129.stdout:2571 trunc from 0xbc7e3bd to 0x153af05 2024-09-17T21:30:22.124 INFO:teuthology.orchestra.run.smithi129.stdout:2572 trunc from 0x153af05 to 0x96249bc 2024-09-17T21:30:22.130 INFO:teuthology.orchestra.run.smithi129.stdout:2575 read 0x1b5f71a thru 0x1b6bbe6 (0xc4cd bytes) 2024-09-17T21:30:22.130 INFO:teuthology.orchestra.run.smithi129.stdout:2576 write 0x7b5ae1b thru 0x7b6a4c4 (0xf6aa bytes) 2024-09-17T21:30:22.130 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:22.128+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:22.143 INFO:teuthology.orchestra.run.smithi129.stdout:2579 write 0xbf66846 thru 0xbf6fd4d (0x9508 bytes) 2024-09-17T21:30:22.148 INFO:teuthology.orchestra.run.smithi129.stdout:2581 punch from 0x4fc5de2 to 0x4fcc486, (0x66a4 bytes) 2024-09-17T21:30:22.148 INFO:teuthology.orchestra.run.smithi129.stdout:2582 writesame 0x553f758 thru 0x554c683 data_size 0x2(0xcf2c bytes) 2024-09-17T21:30:22.161 INFO:teuthology.orchestra.run.smithi129.stdout:2583 read 0x5a0c9b3 thru 0x5a15146 (0x8794 bytes) 2024-09-17T21:30:22.162 INFO:teuthology.orchestra.run.smithi129.stdout:2584 read 0x9ec4829 thru 0x9ed0aac (0xc284 bytes) 2024-09-17T21:30:22.162 INFO:teuthology.orchestra.run.smithi129.stdout:2585 read 0xb537cb9 thru 0xb546df3 (0xf13b bytes) 2024-09-17T21:30:22.162 INFO:teuthology.orchestra.run.smithi129.stdout:2586 punch from 0x4f98c7b to 0x4f98ce6, (0x6b bytes) 2024-09-17T21:30:22.162 INFO:teuthology.orchestra.run.smithi129.stdout:2587 read 0x183e80c thru 0x1840a7c (0x2271 bytes) 2024-09-17T21:30:22.162 INFO:teuthology.orchestra.run.smithi129.stdout:2590 trunc from 0xbf6fd4e to 0xb5448f5 2024-09-17T21:30:22.174 INFO:teuthology.orchestra.run.smithi129.stdout:2591 read 0x1ab9a9f thru 0x1ac889c (0xedfe bytes) 2024-09-17T21:30:22.180 INFO:teuthology.orchestra.run.smithi129.stdout:2592 trunc from 0xb5448f5 to 0x647fd07 2024-09-17T21:30:22.190 INFO:teuthology.orchestra.run.smithi129.stdout:2593 punch from 0x3f9ff50 to 0x3fa2d6c, (0x2e1c bytes) 2024-09-17T21:30:22.190 INFO:teuthology.orchestra.run.smithi129.stdout:2596 punch from 0x202d490 to 0x203b056, (0xdbc6 bytes) 2024-09-17T21:30:22.190 INFO:teuthology.orchestra.run.smithi129.stdout:2597 read 0x2102093 thru 0x2109b6e (0x7adc bytes) 2024-09-17T21:30:22.190 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:22.198 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:22.198 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:22.198 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:22.198 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:22.198 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:22.200 INFO:teuthology.orchestra.run.smithi129.stdout:2599 writesame 0xa55f3aa thru 0xa56559d data_size 0x2(0x61f4 bytes) 2024-09-17T21:30:22.200 INFO:teuthology.orchestra.run.smithi129.stdout:2600 read 0x1dc372 thru 0x1e021d (0x3eac bytes) 2024-09-17T21:30:22.200 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:22.200 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:22.200 INFO:teuthology.orchestra.run.smithi129.stdout:2604 write 0x9bb9d48 thru 0x9bc0096 (0x634f bytes) 2024-09-17T21:30:22.203 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:22.207 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:30:22.211 INFO:teuthology.orchestra.run.smithi129.stdout:2607 write 0xc65868e thru 0xc6612d5 (0x8c48 bytes) 2024-09-17T21:30:22.214 INFO:teuthology.orchestra.run.smithi129.stdout:2608 writesame 0x3a2c1b8 thru 0x3a38ae5 data_size 0xb(0xc92e bytes) 2024-09-17T21:30:22.218 INFO:teuthology.orchestra.run.smithi129.stdout:2609 write 0x8eaf28b thru 0x8ebce2b (0xdba1 bytes) 2024-09-17T21:30:22.222 INFO:teuthology.orchestra.run.smithi129.stdout:2611 trunc from 0xc6612d6 to 0xbe77de6 2024-09-17T21:30:22.230 INFO:teuthology.orchestra.run.smithi129.stdout:2612 write 0x470e77d thru 0x471899a (0xa21e bytes) 2024-09-17T21:30:22.233 INFO:teuthology.orchestra.run.smithi129.stdout:2613 punch from 0x86414da to 0x8649180, (0x7ca6 bytes) 2024-09-17T21:30:22.233 INFO:teuthology.orchestra.run.smithi129.stdout:2614 writesame 0x54e23cc thru 0x54edf5b data_size 0x8(0xbb90 bytes) 2024-09-17T21:30:22.239 INFO:teuthology.orchestra.run.smithi129.stdout:2615 read 0xa7af816 thru 0xa7ba0d1 (0xa8bc bytes) 2024-09-17T21:30:22.246 INFO:teuthology.orchestra.run.smithi129.stdout:2617 writesame 0xc0217e5 thru 0xc023333 data_size 0x1(0x1b4f bytes) 2024-09-17T21:30:22.246 INFO:teuthology.orchestra.run.smithi129.stdout:2618 punch from 0x4e27dd6 to 0x4e2ff50, (0x817a bytes) 2024-09-17T21:30:22.246 INFO:teuthology.orchestra.run.smithi129.stdout:2619 write 0x46ce2b2 thru 0x46d9a99 (0xb7e8 bytes) 2024-09-17T21:30:22.248 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:22.246+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:22.250 INFO:teuthology.orchestra.run.smithi129.stdout:2620 punch from 0x194f60f to 0x1952c16, (0x3607 bytes) 2024-09-17T21:30:22.250 INFO:teuthology.orchestra.run.smithi129.stdout:2621 read 0x131580a thru 0x13164fb (0xcf2 bytes) 2024-09-17T21:30:22.250 INFO:teuthology.orchestra.run.smithi129.stdout:2623 punch from 0x870f097 to 0x8716c1f, (0x7b88 bytes) 2024-09-17T21:30:22.250 INFO:teuthology.orchestra.run.smithi129.stdout:2624 punch from 0x9f0f51 to 0x9fcb12, (0xbbc1 bytes) 2024-09-17T21:30:22.250 INFO:teuthology.orchestra.run.smithi129.stdout:2625 write 0x4842fa thru 0x491378 (0xd07f bytes) 2024-09-17T21:30:22.255 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:22.255 INFO:teuthology.orchestra.run.smithi129.stdout:2627 read 0x56be04d thru 0x56be0a1 (0x55 bytes) 2024-09-17T21:30:22.255 INFO:teuthology.orchestra.run.smithi129.stdout:2628 write 0x92158a9 thru 0x9224c74 (0xf3cc bytes) 2024-09-17T21:30:22.258 INFO:teuthology.orchestra.run.smithi129.stdout:2629 read 0x912011e thru 0x912fdb3 (0xfc96 bytes) 2024-09-17T21:30:22.259 INFO:teuthology.orchestra.run.smithi129.stdout:2630 write 0x962634b thru 0x962a780 (0x4436 bytes) 2024-09-17T21:30:22.261 INFO:teuthology.orchestra.run.smithi129.stdout:2631 read 0x900f0f9 thru 0x901375e (0x4666 bytes) 2024-09-17T21:30:22.261 INFO:teuthology.orchestra.run.smithi129.stdout:2632 read 0x686090f thru 0x686c15b (0xb84d bytes) 2024-09-17T21:30:22.261 INFO:teuthology.orchestra.run.smithi129.stdout:2633 write 0xa50c40d thru 0xa50ebb6 (0x27aa bytes) 2024-09-17T21:30:22.264 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:22.264 INFO:teuthology.orchestra.run.smithi129.stdout:2635 clone 24 order 23 su 262144 sc 13 2024-09-17T21:30:22.326 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:22.324+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:22.353 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:30:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 54.410051344000003, 2024-09-17T21:30:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61572000699999996, 2024-09-17T21:30:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:30:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:30:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T21:30:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939745+0000", 2024-09-17T21:30:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6899999999999999e-06 2024-09-17T21:30:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939753+0000", 2024-09-17T21:30:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1950000000000003e-06 2024-09-17T21:30:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939767+0000", 2024-09-17T21:30:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3429e-05 2024-09-17T21:30:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939776+0000", 2024-09-17T21:30:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:30:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939858+0000", 2024-09-17T21:30:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2496000000000004e-05 2024-09-17T21:30:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T21:30:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939868+0000", 2024-09-17T21:30:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1910000000000006e-06 2024-09-17T21:30:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940612+0000", 2024-09-17T21:30:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074429599999999998 2024-09-17T21:30:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:30:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940621+0000", 2024-09-17T21:30:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9139999999999997e-06 2024-09-17T21:30:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554006+0000", 2024-09-17T21:30:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61338505099999996 2024-09-17T21:30:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554166+0000", 2024-09-17T21:30:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015982600000000001 2024-09-17T21:30:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:30:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554296+0000", 2024-09-17T21:30:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013002399999999999 2024-09-17T21:30:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554936+0000", 2024-09-17T21:30:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064009400000000001 2024-09-17T21:30:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555389+0000", 2024-09-17T21:30:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045355200000000002 2024-09-17T21:30:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555432+0000", 2024-09-17T21:30:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2477e-05 2024-09-17T21:30:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555463+0000", 2024-09-17T21:30:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1612000000000002e-05 2024-09-17T21:30:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17527 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1005568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.944823513000003, 2024-09-17T21:30:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046452090000000001, 2024-09-17T21:30:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17527 2024-09-17T21:30:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405496+0000", 2024-09-17T21:30:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052498400000000004 2024-09-17T21:30:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405500+0000", 2024-09-17T21:30:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8410000000000002e-06 2024-09-17T21:30:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405511+0000", 2024-09-17T21:30:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0749e-05 2024-09-17T21:30:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405517+0000", 2024-09-17T21:30:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:30:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451275+0000", 2024-09-17T21:30:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045758134999999998 2024-09-17T21:30:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451355+0000", 2024-09-17T21:30:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9453e-05 2024-09-17T21:30:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451423+0000", 2024-09-17T21:30:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8782999999999995e-05 2024-09-17T21:30:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17530 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1046528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.944215862, 2024-09-17T21:30:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046376737000000001, 2024-09-17T21:30:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17530 2024-09-17T21:30:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3300000000000002e-07 2024-09-17T21:30:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405581+0000", 2024-09-17T21:30:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.074e-06 2024-09-17T21:30:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405587+0000", 2024-09-17T21:30:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7559999999999996e-06 2024-09-17T21:30:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405592+0000", 2024-09-17T21:30:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8860000000000003e-06 2024-09-17T21:30:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451813+0000", 2024-09-17T21:30:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046221119999999997 2024-09-17T21:30:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451891+0000", 2024-09-17T21:30:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7449999999999999e-05 2024-09-17T21:30:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451956+0000", 2024-09-17T21:30:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4918000000000004e-05 2024-09-17T21:30:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17615 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.939583024999997, 2024-09-17T21:30:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046741309000000002, 2024-09-17T21:30:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17615 2024-09-17T21:30:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:30:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410213+0000", 2024-09-17T21:30:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:30:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410217+0000", 2024-09-17T21:30:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8199999999999998e-06 2024-09-17T21:30:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410220+0000", 2024-09-17T21:30:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6900000000000001e-06 2024-09-17T21:30:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455264+0000", 2024-09-17T21:30:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045044002999999999 2024-09-17T21:30:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455338+0000", 2024-09-17T21:30:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4182000000000002e-05 2024-09-17T21:30:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.456953+0000", 2024-09-17T21:30:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016150089999999999 2024-09-17T21:30:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17616 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 872448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.939570973000002, 2024-09-17T21:30:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946039000000002, 2024-09-17T21:30:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17616 2024-09-17T21:30:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T21:30:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410225+0000", 2024-09-17T21:30:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3459999999999999e-06 2024-09-17T21:30:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410228+0000", 2024-09-17T21:30:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.083e-06 2024-09-17T21:30:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410232+0000", 2024-09-17T21:30:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4419999999999998e-06 2024-09-17T21:30:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457004+0000", 2024-09-17T21:30:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046771873999999998 2024-09-17T21:30:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457084+0000", 2024-09-17T21:30:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0506000000000005e-05 2024-09-17T21:30:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457170+0000", 2024-09-17T21:30:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5494000000000005e-05 2024-09-17T21:30:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17617 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.939559570999997, 2024-09-17T21:30:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047084168000000003, 2024-09-17T21:30:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17617 2024-09-17T21:30:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-17T21:30:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410237+0000", 2024-09-17T21:30:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1960000000000001e-06 2024-09-17T21:30:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410246+0000", 2024-09-17T21:30:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7880000000000001e-06 2024-09-17T21:30:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410250+0000", 2024-09-17T21:30:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.518e-06 2024-09-17T21:30:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457209+0000", 2024-09-17T21:30:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046959283999999997 2024-09-17T21:30:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457264+0000", 2024-09-17T21:30:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673999999999999e-05 2024-09-17T21:30:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457319+0000", 2024-09-17T21:30:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5497000000000001e-05 2024-09-17T21:30:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17618 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 899072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.939540127000001, 2024-09-17T21:30:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047204208999999997, 2024-09-17T21:30:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17618 2024-09-17T21:30:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8799999999999998e-07 2024-09-17T21:30:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410256+0000", 2024-09-17T21:30:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T21:30:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410260+0000", 2024-09-17T21:30:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6500000000000002e-06 2024-09-17T21:30:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410263+0000", 2024-09-17T21:30:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7089999999999998e-06 2024-09-17T21:30:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457355+0000", 2024-09-17T21:30:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047092756999999999 2024-09-17T21:30:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457407+0000", 2024-09-17T21:30:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1808999999999998e-05 2024-09-17T21:30:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457459+0000", 2024-09-17T21:30:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1684000000000002e-05 2024-09-17T21:30:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17619 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 913408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.939528889000002, 2024-09-17T21:30:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047293443999999997, 2024-09-17T21:30:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17619 2024-09-17T21:30:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:30:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410267+0000", 2024-09-17T21:30:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:30:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410270+0000", 2024-09-17T21:30:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.01e-06 2024-09-17T21:30:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410273+0000", 2024-09-17T21:30:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6410000000000002e-06 2024-09-17T21:30:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457473+0000", 2024-09-17T21:30:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047199829999999998 2024-09-17T21:30:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457517+0000", 2024-09-17T21:30:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4058999999999999e-05 2024-09-17T21:30:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457559+0000", 2024-09-17T21:30:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2404999999999998e-05 2024-09-17T21:30:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17620 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 927744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.939517717000001, 2024-09-17T21:30:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047402409, 2024-09-17T21:30:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17620 2024-09-17T21:30:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:30:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410278+0000", 2024-09-17T21:30:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1790000000000001e-06 2024-09-17T21:30:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410281+0000", 2024-09-17T21:30:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.061e-06 2024-09-17T21:30:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410284+0000", 2024-09-17T21:30:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.65e-06 2024-09-17T21:30:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457587+0000", 2024-09-17T21:30:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047303206 2024-09-17T21:30:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457633+0000", 2024-09-17T21:30:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6063999999999998e-05 2024-09-17T21:30:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457679+0000", 2024-09-17T21:30:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6007000000000002e-05 2024-09-17T21:30:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17621 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 940032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.939507300999999, 2024-09-17T21:30:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047486550000000002, 2024-09-17T21:30:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17621 2024-09-17T21:30:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410288+0000", 2024-09-17T21:30:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:30:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410289+0000", 2024-09-17T21:30:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1179999999999999e-06 2024-09-17T21:30:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410292+0000", 2024-09-17T21:30:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:30:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410294+0000", 2024-09-17T21:30:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T21:30:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457694+0000", 2024-09-17T21:30:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047399416999999999 2024-09-17T21:30:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457734+0000", 2024-09-17T21:30:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0528999999999998e-05 2024-09-17T21:30:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457774+0000", 2024-09-17T21:30:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9529e-05 2024-09-17T21:30:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17622 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.939497119000002, 2024-09-17T21:30:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047596619, 2024-09-17T21:30:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17622 2024-09-17T21:30:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410303+0000", 2024-09-17T21:30:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2979999999999999e-06 2024-09-17T21:30:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410307+0000", 2024-09-17T21:30:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:30:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410310+0000", 2024-09-17T21:30:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.146e-06 2024-09-17T21:30:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457801+0000", 2024-09-17T21:30:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047491322000000002 2024-09-17T21:30:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457850+0000", 2024-09-17T21:30:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8585000000000003e-05 2024-09-17T21:30:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457894+0000", 2024-09-17T21:30:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4455e-05 2024-09-17T21:30:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17623 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 968704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.939480734999997, 2024-09-17T21:30:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047689377999999998, 2024-09-17T21:30:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17623 2024-09-17T21:30:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:30:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410315+0000", 2024-09-17T21:30:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0499999999999999e-06 2024-09-17T21:30:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410319+0000", 2024-09-17T21:30:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.517e-06 2024-09-17T21:30:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410321+0000", 2024-09-17T21:30:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:30:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457922+0000", 2024-09-17T21:30:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047600638000000001 2024-09-17T21:30:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457964+0000", 2024-09-17T21:30:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2126000000000003e-05 2024-09-17T21:30:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458003+0000", 2024-09-17T21:30:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9138999999999999e-05 2024-09-17T21:30:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17624 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.939470151000002, 2024-09-17T21:30:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047798292999999999, 2024-09-17T21:30:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17624 2024-09-17T21:30:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:30:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410326+0000", 2024-09-17T21:30:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0610000000000001e-06 2024-09-17T21:30:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410329+0000", 2024-09-17T21:30:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0319999999999999e-06 2024-09-17T21:30:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410331+0000", 2024-09-17T21:30:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5749999999999999e-06 2024-09-17T21:30:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458033+0000", 2024-09-17T21:30:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047701371999999999 2024-09-17T21:30:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458079+0000", 2024-09-17T21:30:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:30:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458123+0000", 2024-09-17T21:30:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850999999999997e-05 2024-09-17T21:30:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17625 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.939459991, 2024-09-17T21:30:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047883051000000003, 2024-09-17T21:30:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17625 2024-09-17T21:30:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410336+0000", 2024-09-17T21:30:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1310000000000001e-06 2024-09-17T21:30:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410339+0000", 2024-09-17T21:30:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000001e-06 2024-09-17T21:30:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410342+0000", 2024-09-17T21:30:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7460000000000001e-06 2024-09-17T21:30:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458137+0000", 2024-09-17T21:30:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047794931999999998 2024-09-17T21:30:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458178+0000", 2024-09-17T21:30:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1013999999999997e-05 2024-09-17T21:30:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458218+0000", 2024-09-17T21:30:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9675999999999998e-05 2024-09-17T21:30:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17626 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1009664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.939449369999998, 2024-09-17T21:30:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047991601000000002, 2024-09-17T21:30:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17626 2024-09-17T21:30:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410346+0000", 2024-09-17T21:30:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:30:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410347+0000", 2024-09-17T21:30:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.114e-06 2024-09-17T21:30:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410350+0000", 2024-09-17T21:30:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.101e-06 2024-09-17T21:30:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410353+0000", 2024-09-17T21:30:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T21:30:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458245+0000", 2024-09-17T21:30:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0478921 2024-09-17T21:30:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458292+0000", 2024-09-17T21:30:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397999999999999e-05 2024-09-17T21:30:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458337+0000", 2024-09-17T21:30:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5221e-05 2024-09-17T21:30:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17627 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.939412269000002, 2024-09-17T21:30:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048047383999999999, 2024-09-17T21:30:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17627 2024-09-17T21:30:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410383+0000", 2024-09-17T21:30:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:30:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410385+0000", 2024-09-17T21:30:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:30:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410391+0000", 2024-09-17T21:30:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:30:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410395+0000", 2024-09-17T21:30:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6539999999999998e-06 2024-09-17T21:30:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458350+0000", 2024-09-17T21:30:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047954855999999997 2024-09-17T21:30:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458392+0000", 2024-09-17T21:30:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2094000000000001e-05 2024-09-17T21:30:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458430+0000", 2024-09-17T21:30:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7474000000000001e-05 2024-09-17T21:30:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17628 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.939392708, 2024-09-17T21:30:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048142676000000002, 2024-09-17T21:30:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17628 2024-09-17T21:30:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:22.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:30:22.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410404+0000", 2024-09-17T21:30:22.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-06 2024-09-17T21:30:22.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410411+0000", 2024-09-17T21:30:22.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6850000000000004e-06 2024-09-17T21:30:22.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410415+0000", 2024-09-17T21:30:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6750000000000001e-06 2024-09-17T21:30:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458457+0000", 2024-09-17T21:30:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048042045999999998 2024-09-17T21:30:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458501+0000", 2024-09-17T21:30:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3661000000000001e-05 2024-09-17T21:30:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458545+0000", 2024-09-17T21:30:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3529000000000003e-05 2024-09-17T21:30:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17687 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.936947754999998, 2024-09-17T21:30:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050670448, 2024-09-17T21:30:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17687 2024-09-17T21:30:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412848+0000", 2024-09-17T21:30:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9100000000000003e-07 2024-09-17T21:30:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413102+0000", 2024-09-17T21:30:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025415600000000001 2024-09-17T21:30:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413111+0000", 2024-09-17T21:30:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2010000000000005e-06 2024-09-17T21:30:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413116+0000", 2024-09-17T21:30:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:30:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458559+0000", 2024-09-17T21:30:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045442914000000001 2024-09-17T21:30:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458632+0000", 2024-09-17T21:30:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3207000000000006e-05 2024-09-17T21:30:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458785+0000", 2024-09-17T21:30:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015226700000000001 2024-09-17T21:30:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.461648+0000", 2024-09-17T21:30:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028633069999999998 2024-09-17T21:30:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463416+0000", 2024-09-17T21:30:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017680230000000001 2024-09-17T21:30:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463475+0000", 2024-09-17T21:30:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9333000000000003e-05 2024-09-17T21:30:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463517+0000", 2024-09-17T21:30:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2162000000000001e-05 2024-09-17T21:30:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17688 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [write 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.936668887000003, 2024-09-17T21:30:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050625098, 2024-09-17T21:30:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17688 2024-09-17T21:30:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:30:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413128+0000", 2024-09-17T21:30:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.813e-06 2024-09-17T21:30:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413132+0000", 2024-09-17T21:30:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:30:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413135+0000", 2024-09-17T21:30:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:30:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459756+0000", 2024-09-17T21:30:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046621293000000001 2024-09-17T21:30:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459816+0000", 2024-09-17T21:30:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9156000000000001e-05 2024-09-17T21:30:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459912+0000", 2024-09-17T21:30:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6302000000000001e-05 2024-09-17T21:30:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463231+0000", 2024-09-17T21:30:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033188290000000001 2024-09-17T21:30:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463701+0000", 2024-09-17T21:30:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047068199999999999 2024-09-17T21:30:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463728+0000", 2024-09-17T21:30:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6726000000000001e-05 2024-09-17T21:30:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463751+0000", 2024-09-17T21:30:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2824e-05 2024-09-17T21:30:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17691 3.d 3:b3a055f3:::rbd_data.11207744cc2c.0000000000000083:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.933123983999998, 2024-09-17T21:30:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049309370999999998, 2024-09-17T21:30:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17691 2024-09-17T21:30:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:22.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416672+0000", 2024-09-17T21:30:22.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.47e-07 2024-09-17T21:30:22.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:22.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416915+0000", 2024-09-17T21:30:22.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024344500000000001 2024-09-17T21:30:22.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:22.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416924+0000", 2024-09-17T21:30:22.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7739999999999993e-06 2024-09-17T21:30:22.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:22.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416929+0000", 2024-09-17T21:30:22.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6990000000000002e-06 2024-09-17T21:30:22.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:22.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460414+0000", 2024-09-17T21:30:22.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043485466 2024-09-17T21:30:22.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:22.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460540+0000", 2024-09-17T21:30:22.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012608899999999999 2024-09-17T21:30:22.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:22.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460746+0000", 2024-09-17T21:30:22.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000205577 2024-09-17T21:30:22.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:22.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463800+0000", 2024-09-17T21:30:22.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030542030000000001 2024-09-17T21:30:22.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:22.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465867+0000", 2024-09-17T21:30:22.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020665929999999998 2024-09-17T21:30:22.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:22.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465914+0000", 2024-09-17T21:30:22.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497000000000003e-05 2024-09-17T21:30:22.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:22.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:22.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:22.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465980+0000", 2024-09-17T21:30:22.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6081000000000006e-05 2024-09-17T21:30:22.448 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.448 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.448 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.448 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:22.448 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:22.448 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:22.449 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:30:22.450 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:30:22.451 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:22.449+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:22.551 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:22.550+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:22.557 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:22.558 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:22.558 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:22.558 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:22.567 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:30:22.573 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:22.573 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:22.574 INFO:teuthology.orchestra.run.smithi129.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-17T21:30:22.591 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:22.591 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:22.607 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:22.608 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:22.625 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:22.625 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:22.642 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:22.642 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:22.652 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:22.650+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:22.658 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:22.659 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:22.675 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:22.675 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:22.677 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:22.677 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:22.678 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:30:22.678 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:30:22.686 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:30:22.703 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is unset 2024-09-17T21:30:22.753 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:22.751+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:22.765 INFO:teuthology.orchestra.run.smithi129.stdout:leaving image image_client.0-clone23 intact 2024-09-17T21:30:22.815 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:22.815 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:22.826 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:30:22.853 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:22.852+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:22.906 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-17T21:30:22.942 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:22.942 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:22.954 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:30:22.955 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:22.952+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:23.040 INFO:teuthology.orchestra.run.smithi129.stdout:2636 read 0xa422f31 thru 0xa42ca58 (0x9b28 bytes) 2024-09-17T21:30:23.052 INFO:teuthology.orchestra.run.smithi129.stdout:2639 trunc from 0xc023334 to 0x2ec2005 2024-09-17T21:30:23.055 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:23.054+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:23.092 INFO:teuthology.orchestra.run.smithi129.stdout:2640 writesame 0x8f6cc2 thru 0x9062c7 data_size 0x9(0xf606 bytes) 2024-09-17T21:30:23.095 INFO:teuthology.orchestra.run.smithi129.stdout:2641 writesame 0x1693cc0 thru 0x169d5c4 data_size 0x1(0x9905 bytes) 2024-09-17T21:30:23.117 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:23.117 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:23.125 INFO:teuthology.orchestra.run.smithi129.stdout:2642 write 0xc90ad02 thru 0xc912a04 (0x7d03 bytes) 2024-09-17T21:30:23.129 INFO:teuthology.orchestra.run.smithi129.stdout:2643 writesame 0x922d585 thru 0x9232481 data_size 0x1(0x4efd bytes) 2024-09-17T21:30:23.129 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:30:23.137 INFO:teuthology.orchestra.run.smithi129.stdout:2644 write 0x49d30b6 thru 0x49d6b8d (0x3ad8 bytes) 2024-09-17T21:30:23.138 INFO:teuthology.orchestra.run.smithi129.stdout:2645 write 0x459de14 thru 0x45adba3 (0xfd90 bytes) 2024-09-17T21:30:23.140 INFO:teuthology.orchestra.run.smithi129.stdout:2646 write 0x818ccb6 thru 0x8198fb6 (0xc301 bytes) 2024-09-17T21:30:23.142 INFO:teuthology.orchestra.run.smithi129.stdout:2647 writesame 0x12e28fd thru 0x12e3ed2 data_size 0xd(0x15d6 bytes) 2024-09-17T21:30:23.143 INFO:teuthology.orchestra.run.smithi129.stdout:2648 punch from 0x1d4950c to 0x1d5891d, (0xf411 bytes) 2024-09-17T21:30:23.143 INFO:teuthology.orchestra.run.smithi129.stdout:2649 read 0x6900f70 thru 0x690d3ca (0xc45b bytes) 2024-09-17T21:30:23.144 INFO:teuthology.orchestra.run.smithi129.stdout:2650 punch from 0x115321d to 0x1160846, (0xd629 bytes) 2024-09-17T21:30:23.144 INFO:teuthology.orchestra.run.smithi129.stdout:2652 read 0xbd42779 thru 0xbd4543b (0x2cc3 bytes) 2024-09-17T21:30:23.144 INFO:teuthology.orchestra.run.smithi129.stdout:2653 write 0x83a44e4 thru 0x83b0787 (0xc2a4 bytes) 2024-09-17T21:30:23.146 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:23.152 INFO:teuthology.orchestra.run.smithi129.stdout:2657 trunc from 0xc912a05 to 0x8301a78 2024-09-17T21:30:23.156 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:23.154+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:23.158 INFO:teuthology.orchestra.run.smithi129.stdout:2660 read 0x5b882d0 thru 0x5b96f99 (0xecca bytes) 2024-09-17T21:30:23.159 INFO:teuthology.orchestra.run.smithi129.stdout:2661 read 0x6e6b226 thru 0x6e6e0dd (0x2eb8 bytes) 2024-09-17T21:30:23.159 INFO:teuthology.orchestra.run.smithi129.stdout:2663 write 0x12c0c4f thru 0x12cfd83 (0xf135 bytes) 2024-09-17T21:30:23.170 INFO:teuthology.orchestra.run.smithi129.stdout:2664 write 0xb9e4b14 thru 0xb9eea94 (0x9f81 bytes) 2024-09-17T21:30:23.172 INFO:teuthology.orchestra.run.smithi129.stdout:2665 clone 25 order 22 su 16384 sc 10 2024-09-17T21:30:23.243 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:23.243 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:23.252 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:30:23.292 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:23.291+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:23.358 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:23.356+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:23.395 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:23.395 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:23.404 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:30:23.459 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:23.458+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:23.547 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:23.547 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:23.556 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:30:23.600 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:23.599+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:23.660 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:23.660 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:23.661 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:23.659+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:23.669 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:30:23.761 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:23.759+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:23.779 INFO:teuthology.orchestra.run.smithi129.stdout:leaving image image_client.0-clone24 intact 2024-09-17T21:30:23.779 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:23.779 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:23.788 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:30:23.872 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:23.870+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:23.921 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-17T21:30:23.933 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:23.933 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:23.942 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:30:23.963 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:23.962+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:24.065 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:24.062+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:24.086 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:24.086 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:24.095 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:30:24.145 INFO:teuthology.orchestra.run.smithi129.stdout:2666 trunc from 0xb9eea95 to 0x8c683a2 2024-09-17T21:30:24.179 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:24.177+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:24.218 INFO:teuthology.orchestra.run.smithi129.stdout:2667 write 0x95a3da8 thru 0x95acfc5 (0x921e bytes) 2024-09-17T21:30:24.226 INFO:teuthology.orchestra.run.smithi129.stdout:2668 trunc from 0x95acfc6 to 0xdcc75f6 2024-09-17T21:30:24.231 INFO:teuthology.orchestra.run.smithi129.stdout:2669 punch from 0x152c5fb to 0x153b8ba, (0xf2bf bytes) 2024-09-17T21:30:24.240 INFO:teuthology.orchestra.run.smithi129.stdout:2672 trunc from 0xdcc75f6 to 0x6337b21 2024-09-17T21:30:24.267 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:24.265+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:24.268 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:24.269 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:24.283 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:30:24.307 INFO:teuthology.orchestra.run.smithi129.stdout:2674 read 0x32435ab thru 0x324f169 (0xbbbf bytes) 2024-09-17T21:30:24.308 INFO:teuthology.orchestra.run.smithi129.stdout:2675 read 0x5ebd246 thru 0x5ec2eb4 (0x5c6f bytes) 2024-09-17T21:30:24.308 INFO:teuthology.orchestra.run.smithi129.stdout:2677 write 0x2b4d46e thru 0x2b4d5a2 (0x135 bytes) 2024-09-17T21:30:24.320 INFO:teuthology.orchestra.run.smithi129.stdout:2678 read 0x408791c thru 0x408f7b6 (0x7e9b bytes) 2024-09-17T21:30:24.321 INFO:teuthology.orchestra.run.smithi129.stdout:2680 write 0xabfb3b thru 0xacf7ee (0xfcb4 bytes) 2024-09-17T21:30:24.367 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:24.366+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:24.409 INFO:teuthology.orchestra.run.smithi129.stdout:2681 write 0xb485bbe thru 0xb48ac90 (0x50d3 bytes) 2024-09-17T21:30:24.418 INFO:teuthology.orchestra.run.smithi129.stdout:2682 writesame 0x1245994 thru 0x124fa1d data_size 0x1(0xa08a bytes) 2024-09-17T21:30:24.431 INFO:teuthology.orchestra.run.smithi129.stdout:2684 read 0x1d76e92 thru 0x1d80695 (0x9804 bytes) 2024-09-17T21:30:24.431 INFO:teuthology.orchestra.run.smithi129.stdout:2685 write 0x3529892 thru 0x3530fb2 (0x7721 bytes) 2024-09-17T21:30:24.462 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:24.462 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:24.469 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:24.467+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:24.471 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:30:24.474 INFO:teuthology.orchestra.run.smithi129.stdout:2687 writesame 0xeaece5d thru 0xeaf8286 data_size 0x3(0xb42a bytes) 2024-09-17T21:30:24.474 INFO:teuthology.orchestra.run.smithi129.stdout:2688 read 0x805f2c5 thru 0x8060c76 (0x19b2 bytes) 2024-09-17T21:30:24.481 INFO:teuthology.orchestra.run.smithi129.stdout:2689 trunc from 0xeaf8287 to 0x86e7c67 2024-09-17T21:30:24.492 INFO:teuthology.orchestra.run.smithi129.stdout:2690 writesame 0x8638b1d thru 0x863b4b9 data_size 0x35(0x299d bytes) 2024-09-17T21:30:24.495 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:24.495 INFO:teuthology.orchestra.run.smithi129.stdout:2694 read 0x2b8f386 thru 0x2b9f362 (0xffdd bytes) 2024-09-17T21:30:24.495 INFO:teuthology.orchestra.run.smithi129.stdout:2697 punch from 0x2c37b7 to 0x2c47cd, (0x1016 bytes) 2024-09-17T21:30:24.496 INFO:teuthology.orchestra.run.smithi129.stdout:2699 write 0x7571905 thru 0x758078d (0xee89 bytes) 2024-09-17T21:30:24.498 INFO:teuthology.orchestra.run.smithi129.stdout:2700 punch from 0x5731f03 to 0x573da6f, (0xbb6c bytes) 2024-09-17T21:30:24.498 INFO:teuthology.orchestra.run.smithi129.stdout:2701 write 0x5a7fb74 thru 0x5a807a6 (0xc33 bytes) 2024-09-17T21:30:24.499 INFO:teuthology.orchestra.run.smithi129.stdout:2702 write 0xd62162 thru 0xd6e02e (0xbecd bytes) 2024-09-17T21:30:24.527 INFO:teuthology.orchestra.run.smithi129.stdout:2703 read 0x1965d23 thru 0x196901d (0x32fb bytes) 2024-09-17T21:30:24.527 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:24.536 INFO:teuthology.orchestra.run.smithi129.stdout:2705 writesame 0xafe26b1 thru 0xaff0879 data_size 0x1(0xe1c9 bytes) 2024-09-17T21:30:24.536 INFO:teuthology.orchestra.run.smithi129.stdout:2706 punch from 0x583e994 to 0x584222d, (0x3899 bytes) 2024-09-17T21:30:24.536 INFO:teuthology.orchestra.run.smithi129.stdout:2708 write 0x3e66fe1 thru 0x3e72901 (0xb921 bytes) 2024-09-17T21:30:24.580 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:24.571+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:24.580 INFO:teuthology.orchestra.run.smithi129.stdout:2710 writesame 0x697f192 thru 0x698a074 data_size 0x1(0xaee3 bytes) 2024-09-17T21:30:24.589 INFO:teuthology.orchestra.run.smithi129.stdout:2711 writesame 0xad329b9 thru 0xad395db data_size 0x1(0x6c23 bytes) 2024-09-17T21:30:24.597 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:24.597 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:24.600 INFO:teuthology.orchestra.run.smithi129.stdout:2712 write 0x4aa93c0 thru 0x4ab6245 (0xce86 bytes) 2024-09-17T21:30:24.607 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:30:24.647 INFO:teuthology.orchestra.run.smithi129.stdout:2713 write 0x1c127db thru 0x1c1aea2 (0x86c8 bytes) 2024-09-17T21:30:24.648 INFO:teuthology.orchestra.run.smithi129.stdout:2714 write 0x7f82186 thru 0x7f88406 (0x6281 bytes) 2024-09-17T21:30:24.659 INFO:teuthology.orchestra.run.smithi129.stdout:2715 trunc from 0xaff087a to 0x5f02b02 2024-09-17T21:30:24.677 INFO:teuthology.orchestra.run.smithi129.stdout:2716 trunc from 0x5f02b02 to 0xa50a2ad 2024-09-17T21:30:24.679 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:30:24.690 INFO:teuthology.orchestra.run.smithi129.stdout:2718 read 0x6c87b94 thru 0x6c8b6f3 (0x3b60 bytes) 2024-09-17T21:30:24.690 INFO:teuthology.orchestra.run.smithi129.stdout:2719 read 0x76c1ccb thru 0x76c4a3b (0x2d71 bytes) 2024-09-17T21:30:24.690 INFO:teuthology.orchestra.run.smithi129.stdout:2721 write 0xa8c20f0 thru 0xa8c8221 (0x6132 bytes) 2024-09-17T21:30:24.691 INFO:teuthology.orchestra.run.smithi129.stdout:2722 punch from 0x47726f8 to 0x477ee65, (0xc76d bytes) 2024-09-17T21:30:24.692 INFO:teuthology.orchestra.run.smithi129.stdout:2723 trunc from 0xa8c8222 to 0xa264e30 2024-09-17T21:30:24.697 INFO:teuthology.orchestra.run.smithi129.stdout:2724 punch from 0x6c4b389 to 0x6c54df7, (0x9a6e bytes) 2024-09-17T21:30:24.704 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:24.702+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:24.707 INFO:teuthology.orchestra.run.smithi129.stdout:2725 writesame 0xda89be2 thru 0xda9837a data_size 0x1(0xe799 bytes) 2024-09-17T21:30:24.707 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:24.719 INFO:teuthology.orchestra.run.smithi129.stdout:2727 trunc from 0xda9837b to 0x3dc0e37 2024-09-17T21:30:24.721 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:30:24.760 INFO:teuthology.orchestra.run.smithi129.stdout:2728 write 0xcd1bde4 thru 0xcd1c69a (0x8b7 bytes) 2024-09-17T21:30:24.765 INFO:teuthology.orchestra.run.smithi129.stdout:2730 punch from 0x266858d to 0x2674773, (0xc1e6 bytes) 2024-09-17T21:30:24.765 INFO:teuthology.orchestra.run.smithi129.stdout:2731 write 0x1f970aa thru 0x1f9babb (0x4a12 bytes) 2024-09-17T21:30:24.772 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:24.770+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:24.776 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:24.776 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:24.786 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:30:24.794 INFO:teuthology.orchestra.run.smithi129.stdout:2733 writesame 0xe26194 thru 0xe303d3 data_size 0x58(0xa240 bytes) 2024-09-17T21:30:24.818 INFO:teuthology.orchestra.run.smithi129.stdout:2734 punch from 0x2f96336 to 0x2f9fec0, (0x9b8a bytes) 2024-09-17T21:30:24.819 INFO:teuthology.orchestra.run.smithi129.stdout:2735 write 0x42b9eeb thru 0x42c9031 (0xf147 bytes) 2024-09-17T21:30:24.820 INFO:teuthology.orchestra.run.smithi129.stdout:2736 writesame 0xa78088b thru 0xa78cc25 data_size 0x5(0xc39b bytes) 2024-09-17T21:30:24.837 INFO:teuthology.orchestra.run.smithi129.stdout:2737 trunc from 0xcd1c69b to 0x7da4b62 2024-09-17T21:30:24.846 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:24.846 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:24.846 INFO:teuthology.orchestra.run.smithi129.stdout:2740 write 0x23fb408 thru 0x23fe647 (0x3240 bytes) 2024-09-17T21:30:24.852 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:24.853 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:24.854 INFO:teuthology.orchestra.run.smithi129.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-17T21:30:24.868 INFO:teuthology.orchestra.run.smithi129.stdout:2742 write 0x7aa83a0 thru 0x7aae3ad (0x600e bytes) 2024-09-17T21:30:24.872 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:24.871+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:24.875 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:24.875 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:24.879 INFO:teuthology.orchestra.run.smithi129.stdout:2743 write 0xaa93494 thru 0xaa9e4e6 (0xb053 bytes) 2024-09-17T21:30:24.884 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:24.894 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:24.894 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:24.894 INFO:teuthology.orchestra.run.smithi129.stdout:2745 trunc from 0xaa9e4e7 to 0x210fd83 2024-09-17T21:30:24.920 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:24.921 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:24.940 INFO:teuthology.orchestra.run.smithi129.stdout:2747 write 0x58b1047 thru 0x58b1b90 (0xb4a bytes) 2024-09-17T21:30:24.941 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:24.941 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:24.946 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:24.946 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:24.955 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:30:24.958 INFO:teuthology.orchestra.run.smithi129.stdout:2748 write 0xdc48728 thru 0xdc53530 (0xae09 bytes) 2024-09-17T21:30:24.958 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:24.959 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:24.963 INFO:teuthology.orchestra.run.smithi129.stdout:2750 punch from 0x974fe93 to 0x9754d92, (0x4eff bytes) 2024-09-17T21:30:24.963 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:24.963 INFO:teuthology.orchestra.run.smithi129.stdout:2752 punch from 0x3fff257 to 0x4006177, (0x6f20 bytes) 2024-09-17T21:30:24.963 INFO:teuthology.orchestra.run.smithi129.stdout:2753 trunc from 0xdc53531 to 0xcf3fcbc 2024-09-17T21:30:24.970 INFO:teuthology.orchestra.run.smithi129.stdout:2754 punch from 0x9d077e0 to 0x9d0ae75, (0x3695 bytes) 2024-09-17T21:30:24.970 INFO:teuthology.orchestra.run.smithi129.stdout:2756 write 0x736bdec thru 0x736da2e (0x1c43 bytes) 2024-09-17T21:30:24.973 INFO:teuthology.orchestra.run.smithi129.stdout:2757 read 0x3c9dfa2 thru 0x3cad5e1 (0xf640 bytes) 2024-09-17T21:30:24.973 INFO:teuthology.orchestra.run.smithi129.stdout:2759 punch from 0x7a03851 to 0x7a105f3, (0xcda2 bytes) 2024-09-17T21:30:24.973 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:24.972+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:24.976 INFO:teuthology.orchestra.run.smithi129.stdout:2764 trunc from 0xcf3fcbc to 0xa0b731d 2024-09-17T21:30:24.977 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:24.977 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:24.980 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:30:24.981 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:30:24.985 INFO:teuthology.orchestra.run.smithi129.stdout:2765 write 0x6782099 thru 0x678de8c (0xbdf4 bytes) 2024-09-17T21:30:24.999 INFO:teuthology.orchestra.run.smithi129.stdout:2766 write 0xd423dea thru 0xd42e5aa (0xa7c1 bytes) 2024-09-17T21:30:25.009 INFO:teuthology.orchestra.run.smithi129.stdout:2768 writesame 0x2519330 thru 0x25284b6 data_size 0x4d(0xf187 bytes) 2024-09-17T21:30:25.018 INFO:teuthology.orchestra.run.smithi129.stdout:2770 write 0xe68a73a thru 0xe68dcce (0x3595 bytes) 2024-09-17T21:30:25.021 INFO:teuthology.orchestra.run.smithi129.stdout:2771 punch from 0x80b654d to 0x80bc623, (0x60d6 bytes) 2024-09-17T21:30:25.021 INFO:teuthology.orchestra.run.smithi129.stdout:2772 read 0x979874b thru 0x979bd0f (0x35c5 bytes) 2024-09-17T21:30:25.021 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.035 INFO:teuthology.orchestra.run.smithi129.stdout:2774 trunc from 0xe68dccf to 0x253a829 2024-09-17T21:30:25.062 INFO:teuthology.orchestra.run.smithi129.stdout:2775 write 0xb1d4b9f thru 0xb1d6695 (0x1af7 bytes) 2024-09-17T21:30:25.063 INFO:teuthology.orchestra.run.smithi129.stdout:2777 writesame 0xe8b245 thru 0xe952f6 data_size 0x43(0xa0b2 bytes) 2024-09-17T21:30:25.066 INFO:teuthology.orchestra.run.smithi129.stdout:2782 read 0x764ed8f thru 0x7654e62 (0x60d4 bytes) 2024-09-17T21:30:25.066 INFO:teuthology.orchestra.run.smithi129.stdout:2783 read 0x3d042a7 thru 0x3d05024 (0xd7e bytes) 2024-09-17T21:30:25.067 INFO:teuthology.orchestra.run.smithi129.stdout:2785 trunc from 0xb1d6696 to 0xc8893b0 2024-09-17T21:30:25.068 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:25.069 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:25.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:25.069 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:25.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.069 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:36935 3.a 3:54c8759b:::rbd_data.1120e62eea89.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-17T21:30:25.069 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:25.069 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.106674043999998, 2024-09-17T21:30:25.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020540151999999999, 2024-09-17T21:30:25.069 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.069 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.069 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:25.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.070 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.070 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.070 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 36935 2024-09-17T21:30:25.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.070 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:25.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:25.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958910+0000", 2024-09-17T21:30:25.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:30:25.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958918+0000", 2024-09-17T21:30:25.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6469999999999998e-06 2024-09-17T21:30:25.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958933+0000", 2024-09-17T21:30:25.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4791e-05 2024-09-17T21:30:25.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958940+0000", 2024-09-17T21:30:25.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-17T21:30:25.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958990+0000", 2024-09-17T21:30:25.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9759000000000003e-05 2024-09-17T21:30:25.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.959039+0000", 2024-09-17T21:30:25.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9487000000000002e-05 2024-09-17T21:30:25.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:25.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.962098+0000", 2024-09-17T21:30:25.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003059258 2024-09-17T21:30:25.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:25.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979325+0000", 2024-09-17T21:30:25.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017226841 2024-09-17T21:30:25.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:25.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979343+0000", 2024-09-17T21:30:25.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7486000000000002e-05 2024-09-17T21:30:25.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:25.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979401+0000", 2024-09-17T21:30:25.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7980999999999999e-05 2024-09-17T21:30:25.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979449+0000", 2024-09-17T21:30:25.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8639e-05 2024-09-17T21:30:25.075 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.075 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.075 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.075 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39447 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 909312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.075 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:25.075 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097736143999999, 2024-09-17T21:30:25.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017419153999999999, 2024-09-17T21:30:25.075 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.076 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.076 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.076 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.076 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.076 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.076 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39447 2024-09-17T21:30:25.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.076 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:25.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:25.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:25.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:30:25.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967849+0000", 2024-09-17T21:30:25.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4339999999999999e-06 2024-09-17T21:30:25.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967853+0000", 2024-09-17T21:30:25.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:30:25.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967857+0000", 2024-09-17T21:30:25.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:30:25.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985193+0000", 2024-09-17T21:30:25.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017335973000000001 2024-09-17T21:30:25.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985228+0000", 2024-09-17T21:30:25.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5154999999999998e-05 2024-09-17T21:30:25.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985266+0000", 2024-09-17T21:30:25.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8448000000000002e-05 2024-09-17T21:30:25.079 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.080 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.080 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.080 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39448 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 974848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.080 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:25.080 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097723328000001, 2024-09-17T21:30:25.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017517066000000001, 2024-09-17T21:30:25.080 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.080 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.080 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.080 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.080 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.081 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.081 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39448 2024-09-17T21:30:25.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.081 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:25.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:25.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:25.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.72e-07 2024-09-17T21:30:25.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967861+0000", 2024-09-17T21:30:25.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.096e-06 2024-09-17T21:30:25.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967865+0000", 2024-09-17T21:30:25.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9339999999999999e-06 2024-09-17T21:30:25.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967868+0000", 2024-09-17T21:30:25.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.508e-06 2024-09-17T21:30:25.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985285+0000", 2024-09-17T21:30:25.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017417208 2024-09-17T21:30:25.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985339+0000", 2024-09-17T21:30:25.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3931000000000003e-05 2024-09-17T21:30:25.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985377+0000", 2024-09-17T21:30:25.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8016999999999999e-05 2024-09-17T21:30:25.084 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.084 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.084 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.084 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39449 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.085 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:25.085 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097711218000001, 2024-09-17T21:30:25.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017595137, 2024-09-17T21:30:25.085 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.085 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.085 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.085 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.085 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.085 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.085 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39449 2024-09-17T21:30:25.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.085 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:25.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:25.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:25.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2700000000000001e-07 2024-09-17T21:30:25.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967873+0000", 2024-09-17T21:30:25.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.164e-06 2024-09-17T21:30:25.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967877+0000", 2024-09-17T21:30:25.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.23e-06 2024-09-17T21:30:25.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967880+0000", 2024-09-17T21:30:25.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.286e-06 2024-09-17T21:30:25.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985397+0000", 2024-09-17T21:30:25.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017516982 2024-09-17T21:30:25.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985431+0000", 2024-09-17T21:30:25.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3899999999999997e-05 2024-09-17T21:30:25.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985467+0000", 2024-09-17T21:30:25.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6347999999999998e-05 2024-09-17T21:30:25.089 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.089 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.089 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.089 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39450 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1105920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.089 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:25.089 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097690961000001, 2024-09-17T21:30:25.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017677824000000002, 2024-09-17T21:30:25.090 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.090 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.090 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.090 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.090 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.090 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.090 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39450 2024-09-17T21:30:25.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.090 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:25.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:25.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967893+0000", 2024-09-17T21:30:25.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9100000000000004e-07 2024-09-17T21:30:25.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967895+0000", 2024-09-17T21:30:25.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:30:25.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967901+0000", 2024-09-17T21:30:25.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.083e-06 2024-09-17T21:30:25.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967906+0000", 2024-09-17T21:30:25.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:30:25.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985493+0000", 2024-09-17T21:30:25.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017587565999999999 2024-09-17T21:30:25.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985531+0000", 2024-09-17T21:30:25.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7490000000000002e-05 2024-09-17T21:30:25.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985570+0000", 2024-09-17T21:30:25.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9205000000000002e-05 2024-09-17T21:30:25.094 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.094 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.094 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.094 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39451 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1171456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.094 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:25.094 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097670580999999, 2024-09-17T21:30:25.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017752337999999999, 2024-09-17T21:30:25.094 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.094 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.094 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.095 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.095 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.095 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.095 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39451 2024-09-17T21:30:25.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.095 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:25.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:25.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:25.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:30:25.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967915+0000", 2024-09-17T21:30:25.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780000000000001e-06 2024-09-17T21:30:25.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967922+0000", 2024-09-17T21:30:25.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7020000000000004e-06 2024-09-17T21:30:25.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967926+0000", 2024-09-17T21:30:25.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.809e-06 2024-09-17T21:30:25.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985590+0000", 2024-09-17T21:30:25.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017663913999999999 2024-09-17T21:30:25.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985627+0000", 2024-09-17T21:30:25.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6896e-05 2024-09-17T21:30:25.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985665+0000", 2024-09-17T21:30:25.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7793999999999998e-05 2024-09-17T21:30:25.098 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.098 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.098 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.099 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39452 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1236992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.099 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:25.099 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097650639000001, 2024-09-17T21:30:25.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017829602999999999, 2024-09-17T21:30:25.099 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.099 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.099 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.099 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.099 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.099 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.099 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39452 2024-09-17T21:30:25.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.099 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:25.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:25.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:25.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:30:25.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967947+0000", 2024-09-17T21:30:25.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4280999999999999e-05 2024-09-17T21:30:25.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967954+0000", 2024-09-17T21:30:25.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3249999999999996e-06 2024-09-17T21:30:25.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967959+0000", 2024-09-17T21:30:25.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2299999999999999e-06 2024-09-17T21:30:25.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985687+0000", 2024-09-17T21:30:25.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017727844999999999 2024-09-17T21:30:25.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985725+0000", 2024-09-17T21:30:25.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8243e-05 2024-09-17T21:30:25.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985762+0000", 2024-09-17T21:30:25.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7023000000000001e-05 2024-09-17T21:30:25.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.103 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.103 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39453 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.103 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:25.103 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097614150999998, 2024-09-17T21:30:25.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017893793000000002, 2024-09-17T21:30:25.103 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.104 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.104 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.104 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39453 2024-09-17T21:30:25.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.104 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:25.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:25.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967970+0000", 2024-09-17T21:30:25.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:30:25.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967972+0000", 2024-09-17T21:30:25.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0229999999999999e-06 2024-09-17T21:30:25.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967978+0000", 2024-09-17T21:30:25.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0430000000000004e-06 2024-09-17T21:30:25.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967982+0000", 2024-09-17T21:30:25.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2570000000000001e-06 2024-09-17T21:30:25.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985782+0000", 2024-09-17T21:30:25.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017799683 2024-09-17T21:30:25.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985818+0000", 2024-09-17T21:30:25.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6304999999999999e-05 2024-09-17T21:30:25.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985863+0000", 2024-09-17T21:30:25.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4953999999999997e-05 2024-09-17T21:30:25.107 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.107 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.107 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.108 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39454 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1368064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.108 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:25.108 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097594823000001, 2024-09-17T21:30:25.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017968886, 2024-09-17T21:30:25.108 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.108 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.108 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.108 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39454 2024-09-17T21:30:25.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.108 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:25.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:25.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967989+0000", 2024-09-17T21:30:25.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.15e-07 2024-09-17T21:30:25.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967991+0000", 2024-09-17T21:30:25.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.945e-06 2024-09-17T21:30:25.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967997+0000", 2024-09-17T21:30:25.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0190000000000003e-06 2024-09-17T21:30:25.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968001+0000", 2024-09-17T21:30:25.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6650000000000002e-06 2024-09-17T21:30:25.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985883+0000", 2024-09-17T21:30:25.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017881971999999999 2024-09-17T21:30:25.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985920+0000", 2024-09-17T21:30:25.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-05 2024-09-17T21:30:25.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985957+0000", 2024-09-17T21:30:25.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7370000000000003e-05 2024-09-17T21:30:25.112 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.112 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.112 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.112 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39455 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.112 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:25.112 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097575648999999, 2024-09-17T21:30:25.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018044437999999999, 2024-09-17T21:30:25.112 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.112 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.112 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.112 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.113 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.113 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.113 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39455 2024-09-17T21:30:25.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.113 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:25.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:25.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:25.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:30:25.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968010+0000", 2024-09-17T21:30:25.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.886e-06 2024-09-17T21:30:25.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968017+0000", 2024-09-17T21:30:25.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:30:25.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968021+0000", 2024-09-17T21:30:25.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850000000000004e-06 2024-09-17T21:30:25.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985977+0000", 2024-09-17T21:30:25.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017956368 2024-09-17T21:30:25.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986015+0000", 2024-09-17T21:30:25.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7610000000000001e-05 2024-09-17T21:30:25.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986052+0000", 2024-09-17T21:30:25.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7109e-05 2024-09-17T21:30:25.116 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.116 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.116 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.117 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39456 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1499136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.117 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:25.117 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097555349, 2024-09-17T21:30:25.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018114514000000002, 2024-09-17T21:30:25.117 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.117 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.117 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.117 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.117 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.117 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.117 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39456 2024-09-17T21:30:25.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.118 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:25.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:25.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:25.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-17T21:30:25.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968030+0000", 2024-09-17T21:30:25.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0049999999999999e-06 2024-09-17T21:30:25.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968036+0000", 2024-09-17T21:30:25.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0129999999999999e-06 2024-09-17T21:30:25.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968041+0000", 2024-09-17T21:30:25.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8300000000000003e-06 2024-09-17T21:30:25.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986071+0000", 2024-09-17T21:30:25.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018029620999999999 2024-09-17T21:30:25.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986106+0000", 2024-09-17T21:30:25.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4867999999999999e-05 2024-09-17T21:30:25.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986142+0000", 2024-09-17T21:30:25.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6566000000000002e-05 2024-09-17T21:30:25.121 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.121 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.121 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.121 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39457 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1564672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.121 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:25.121 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097535532999999, 2024-09-17T21:30:25.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018188644, 2024-09-17T21:30:25.122 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.122 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.122 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.122 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.122 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.122 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.122 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39457 2024-09-17T21:30:25.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.122 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:25.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:25.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:25.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7199999999999999e-07 2024-09-17T21:30:25.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968050+0000", 2024-09-17T21:30:25.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1110000000000002e-06 2024-09-17T21:30:25.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968057+0000", 2024-09-17T21:30:25.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6170000000000003e-06 2024-09-17T21:30:25.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968062+0000", 2024-09-17T21:30:25.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:30:25.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986163+0000", 2024-09-17T21:30:25.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018101435999999999 2024-09-17T21:30:25.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986199+0000", 2024-09-17T21:30:25.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.591e-05 2024-09-17T21:30:25.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986236+0000", 2024-09-17T21:30:25.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7416999999999998e-05 2024-09-17T21:30:25.125 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.125 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.125 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.126 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39458 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1630208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.126 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:25.126 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097508593000001, 2024-09-17T21:30:25.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018255856000000001, 2024-09-17T21:30:25.126 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.126 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.126 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.126 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.126 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.126 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.126 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39458 2024-09-17T21:30:25.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.126 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:25.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:25.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:25.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:30:25.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968077+0000", 2024-09-17T21:30:25.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.779e-06 2024-09-17T21:30:25.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968081+0000", 2024-09-17T21:30:25.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.532e-06 2024-09-17T21:30:25.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968085+0000", 2024-09-17T21:30:25.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999999e-06 2024-09-17T21:30:25.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986256+0000", 2024-09-17T21:30:25.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018171273000000002 2024-09-17T21:30:25.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986293+0000", 2024-09-17T21:30:25.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6534e-05 2024-09-17T21:30:25.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986330+0000", 2024-09-17T21:30:25.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7536000000000001e-05 2024-09-17T21:30:25.130 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.130 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.130 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.130 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39459 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1695744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.130 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:25.130 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097494014999999, 2024-09-17T21:30:25.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334493, 2024-09-17T21:30:25.130 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.130 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.131 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.131 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39459 2024-09-17T21:30:25.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.131 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:25.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:25.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968090+0000", 2024-09-17T21:30:25.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:30:25.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968091+0000", 2024-09-17T21:30:25.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3319999999999999e-06 2024-09-17T21:30:25.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968094+0000", 2024-09-17T21:30:25.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2590000000000001e-06 2024-09-17T21:30:25.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968097+0000", 2024-09-17T21:30:25.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:30:25.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986350+0000", 2024-09-17T21:30:25.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018253364000000001 2024-09-17T21:30:25.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986387+0000", 2024-09-17T21:30:25.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6350999999999998e-05 2024-09-17T21:30:25.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986424+0000", 2024-09-17T21:30:25.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6897999999999998e-05 2024-09-17T21:30:25.134 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.134 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.135 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.135 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39460 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1761280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.135 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:25.135 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097482879000001, 2024-09-17T21:30:25.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018415918999999999, 2024-09-17T21:30:25.135 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.135 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.135 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.135 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.135 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.135 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.135 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39460 2024-09-17T21:30:25.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.136 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:25.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:25.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968101+0000", 2024-09-17T21:30:25.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T21:30:25.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968102+0000", 2024-09-17T21:30:25.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1689999999999999e-06 2024-09-17T21:30:25.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968105+0000", 2024-09-17T21:30:25.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:30:25.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968108+0000", 2024-09-17T21:30:25.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6630000000000002e-06 2024-09-17T21:30:25.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986442+0000", 2024-09-17T21:30:25.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334487 2024-09-17T21:30:25.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986479+0000", 2024-09-17T21:30:25.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6557000000000003e-05 2024-09-17T21:30:25.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986516+0000", 2024-09-17T21:30:25.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7648999999999998e-05 2024-09-17T21:30:25.139 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.139 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.139 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.139 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39461 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1826816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.139 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:25.140 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097472118999999, 2024-09-17T21:30:25.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018499768, 2024-09-17T21:30:25.140 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.140 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.140 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.140 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39461 2024-09-17T21:30:25.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.140 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:25.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:25.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:25.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4699999999999998e-07 2024-09-17T21:30:25.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968112+0000", 2024-09-17T21:30:25.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-06 2024-09-17T21:30:25.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968116+0000", 2024-09-17T21:30:25.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1080000000000001e-06 2024-09-17T21:30:25.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968122+0000", 2024-09-17T21:30:25.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1070000000000001e-06 2024-09-17T21:30:25.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986537+0000", 2024-09-17T21:30:25.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018414834000000001 2024-09-17T21:30:25.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986572+0000", 2024-09-17T21:30:25.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5726000000000003e-05 2024-09-17T21:30:25.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986611+0000", 2024-09-17T21:30:25.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8584000000000002e-05 2024-09-17T21:30:25.143 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.144 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.144 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39462 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1892352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.144 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:25.144 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097457755000001, 2024-09-17T21:30:25.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018583252000000001, 2024-09-17T21:30:25.144 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.144 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.144 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.144 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.144 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.145 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.145 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39462 2024-09-17T21:30:25.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.145 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:25.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:25.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968126+0000", 2024-09-17T21:30:25.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:25.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968127+0000", 2024-09-17T21:30:25.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.229e-06 2024-09-17T21:30:25.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968131+0000", 2024-09-17T21:30:25.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8010000000000001e-06 2024-09-17T21:30:25.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968134+0000", 2024-09-17T21:30:25.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8880000000000001e-06 2024-09-17T21:30:25.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986634+0000", 2024-09-17T21:30:25.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018500501999999999 2024-09-17T21:30:25.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986671+0000", 2024-09-17T21:30:25.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6881000000000001e-05 2024-09-17T21:30:25.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986709+0000", 2024-09-17T21:30:25.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7546999999999998e-05 2024-09-17T21:30:25.148 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.148 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.148 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.148 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39463 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1957888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.148 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:25.149 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097446139999999, 2024-09-17T21:30:25.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018667186999999998, 2024-09-17T21:30:25.149 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.149 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.149 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.149 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.149 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.149 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.149 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39463 2024-09-17T21:30:25.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.149 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:25.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:25.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:25.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:30:25.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968138+0000", 2024-09-17T21:30:25.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:30:25.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968142+0000", 2024-09-17T21:30:25.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.145e-06 2024-09-17T21:30:25.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968144+0000", 2024-09-17T21:30:25.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6869999999999999e-06 2024-09-17T21:30:25.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986728+0000", 2024-09-17T21:30:25.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018584163000000001 2024-09-17T21:30:25.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986766+0000", 2024-09-17T21:30:25.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7987000000000001e-05 2024-09-17T21:30:25.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986804+0000", 2024-09-17T21:30:25.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7892000000000002e-05 2024-09-17T21:30:25.153 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.153 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.153 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.153 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39464 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.153 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:25.153 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097435830999999, 2024-09-17T21:30:25.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018753723999999999, 2024-09-17T21:30:25.154 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.154 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.154 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.154 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.154 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.154 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.154 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39464 2024-09-17T21:30:25.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.154 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:25.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:25.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968148+0000", 2024-09-17T21:30:25.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:30:25.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968152+0000", 2024-09-17T21:30:25.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3749999999999996e-06 2024-09-17T21:30:25.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968155+0000", 2024-09-17T21:30:25.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.371e-06 2024-09-17T21:30:25.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968158+0000", 2024-09-17T21:30:25.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T21:30:25.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986824+0000", 2024-09-17T21:30:25.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018665779 2024-09-17T21:30:25.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986863+0000", 2024-09-17T21:30:25.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9273000000000002e-05 2024-09-17T21:30:25.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986901+0000", 2024-09-17T21:30:25.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7978000000000001e-05 2024-09-17T21:30:25.157 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.157 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.158 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.158 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39465 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:25.158 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:25.158 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.097420432, 2024-09-17T21:30:25.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018825341999999998, 2024-09-17T21:30:25.158 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.158 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.158 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.158 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.158 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.158 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.158 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39465 2024-09-17T21:30:25.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.159 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:25.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:25.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:25.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-17T21:30:25.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968164+0000", 2024-09-17T21:30:25.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1289999999999999e-06 2024-09-17T21:30:25.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968167+0000", 2024-09-17T21:30:25.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1379999999999999e-06 2024-09-17T21:30:25.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968170+0000", 2024-09-17T21:30:25.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:30:25.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986914+0000", 2024-09-17T21:30:25.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018743829 2024-09-17T21:30:25.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986950+0000", 2024-09-17T21:30:25.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5819999999999999e-05 2024-09-17T21:30:25.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986988+0000", 2024-09-17T21:30:25.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8362000000000003e-05 2024-09-17T21:30:25.162 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.162 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.162 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.162 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.162 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.162 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:25.164 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:30:25.166 INFO:teuthology.orchestra.run.smithi129.stdout:2786 writesame 0xaf53fb5 thru 0xaf54888 data_size 0x4(0x8d4 bytes) 2024-09-17T21:30:25.166 INFO:teuthology.orchestra.run.smithi129.stdout:2788 read 0x2d765c2 thru 0x2d85c26 (0xf665 bytes) 2024-09-17T21:30:25.166 INFO:teuthology.orchestra.run.smithi129.stdout:2789 read 0x6e4dd72 thru 0x6e5d0d5 (0xf364 bytes) 2024-09-17T21:30:25.166 INFO:teuthology.orchestra.run.smithi129.stdout:2791 read 0xabef528 thru 0xabf174d (0x2226 bytes) 2024-09-17T21:30:25.166 INFO:teuthology.orchestra.run.smithi129.stdout:2794 writesame 0x3ea3ebd thru 0x3ea73a0 data_size 0xa(0x34e4 bytes) 2024-09-17T21:30:25.166 INFO:teuthology.orchestra.run.smithi129.stdout:2795 trunc from 0xc8893b0 to 0x696b41e 2024-09-17T21:30:25.166 INFO:teuthology.orchestra.run.smithi129.stdout:2796 punch from 0x30c0810 to 0x30ccd35, (0xc525 bytes) 2024-09-17T21:30:25.167 INFO:teuthology.orchestra.run.smithi129.stdout:2797 writesame 0x2667591 thru 0x266fc9e data_size 0x3b(0x870e bytes) 2024-09-17T21:30:25.167 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.167 INFO:teuthology.orchestra.run.smithi129.stdout:2799 trunc from 0x696b41e to 0xe6d002b 2024-09-17T21:30:25.167 INFO:teuthology.orchestra.run.smithi129.stdout:2801 trunc from 0xe6d002b to 0x5e4d1bd 2024-09-17T21:30:25.167 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.167 INFO:teuthology.orchestra.run.smithi129.stdout:2803 punch from 0x5444e5b to 0x54511d8, (0xc37d bytes) 2024-09-17T21:30:25.167 INFO:teuthology.orchestra.run.smithi129.stdout:2804 read 0x14fce43 thru 0x14ffc46 (0x2e04 bytes) 2024-09-17T21:30:25.167 INFO:teuthology.orchestra.run.smithi129.stdout:2805 writesame 0x2ec2c82 thru 0x2ed1ba1 data_size 0x10(0xef20 bytes) 2024-09-17T21:30:25.167 INFO:teuthology.orchestra.run.smithi129.stdout:2806 punch from 0x14fef96 to 0x15092bc, (0xa326 bytes) 2024-09-17T21:30:25.167 INFO:teuthology.orchestra.run.smithi129.stdout:2808 read 0x1f3022f thru 0x1f35d3b (0x5b0d bytes) 2024-09-17T21:30:25.167 INFO:teuthology.orchestra.run.smithi129.stdout:2809 trunc from 0x5e4d1bd to 0x7824e22 2024-09-17T21:30:25.167 INFO:teuthology.orchestra.run.smithi129.stdout:2810 write 0x432c986 thru 0x4334faf (0x862a bytes) 2024-09-17T21:30:25.167 INFO:teuthology.orchestra.run.smithi129.stdout:2811 write 0x9b58be9 thru 0x9b5d15a (0x4572 bytes) 2024-09-17T21:30:25.167 INFO:teuthology.orchestra.run.smithi129.stdout:2812 read 0x86c392f thru 0x86c9fa8 (0x667a bytes) 2024-09-17T21:30:25.167 INFO:teuthology.orchestra.run.smithi129.stdout:2813 punch from 0x76a710c to 0x76a781b, (0x70f bytes) 2024-09-17T21:30:25.168 INFO:teuthology.orchestra.run.smithi129.stdout:2814 write 0xc8313c3 thru 0xc83f020 (0xdc5e bytes) 2024-09-17T21:30:25.168 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:25.162+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:25.168 INFO:teuthology.orchestra.run.smithi129.stdout:2817 read 0x9b0aee thru 0x9bab13 (0xa026 bytes) 2024-09-17T21:30:25.168 INFO:teuthology.orchestra.run.smithi129.stdout:2818 punch from 0x65a8fe2 to 0x65b2811, (0x982f bytes) 2024-09-17T21:30:25.173 INFO:teuthology.orchestra.run.smithi129.stdout:2819 trunc from 0xc83f021 to 0x8fa2811 2024-09-17T21:30:25.183 INFO:teuthology.orchestra.run.smithi129.stdout:2820 read 0x5357203 thru 0x5362f49 (0xbd47 bytes) 2024-09-17T21:30:25.183 INFO:teuthology.orchestra.run.smithi129.stdout:2822 punch from 0x3013239 to 0x301c20b, (0x8fd2 bytes) 2024-09-17T21:30:25.183 INFO:teuthology.orchestra.run.smithi129.stdout:2825 punch from 0x44a8c03 to 0x44ac46c, (0x3869 bytes) 2024-09-17T21:30:25.183 INFO:teuthology.orchestra.run.smithi129.stdout:2826 flatten 2024-09-17T21:30:25.188 INFO:teuthology.orchestra.run.smithi129.stdout:2827 punch from 0xcfac9f to 0xd07fdf, (0xd340 bytes) 2024-09-17T21:30:25.188 INFO:teuthology.orchestra.run.smithi129.stdout:2828 write 0x5766447 thru 0x576abc6 (0x4780 bytes) 2024-09-17T21:30:25.192 INFO:teuthology.orchestra.run.smithi129.stdout:2830 write 0xcbc527 thru 0xcbf62e (0x3108 bytes) 2024-09-17T21:30:25.193 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.193 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.193 INFO:teuthology.orchestra.run.smithi129.stdout:2836 punch from 0x79a56c4 to 0x79a9c91, (0x45cd bytes) 2024-09-17T21:30:25.193 INFO:teuthology.orchestra.run.smithi129.stdout:2837 write 0x3bdacf3 thru 0x3be6ca1 (0xbfaf bytes) 2024-09-17T21:30:25.195 INFO:teuthology.orchestra.run.smithi129.stdout:2839 write 0x3f17008 thru 0x3f23930 (0xc929 bytes) 2024-09-17T21:30:25.197 INFO:teuthology.orchestra.run.smithi129.stdout:2840 write 0x16e92e3 thru 0x16f7ace (0xe7ec bytes) 2024-09-17T21:30:25.199 INFO:teuthology.orchestra.run.smithi129.stdout:2841 punch from 0x472c54c to 0x4739ba9, (0xd65d bytes) 2024-09-17T21:30:25.199 INFO:teuthology.orchestra.run.smithi129.stdout:2842 punch from 0x1b486e1 to 0x1b513de, (0x8cfd bytes) 2024-09-17T21:30:25.208 INFO:teuthology.orchestra.run.smithi129.stdout:2844 write 0xb522599 thru 0xb52feab (0xd913 bytes) 2024-09-17T21:30:25.213 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.213 INFO:teuthology.orchestra.run.smithi129.stdout:2846 write 0x25e03fe thru 0x25e5f34 (0x5b37 bytes) 2024-09-17T21:30:25.215 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.215 INFO:teuthology.orchestra.run.smithi129.stdout:2848 punch from 0x9a36629 to 0x9a37dd2, (0x17a9 bytes) 2024-09-17T21:30:25.215 INFO:teuthology.orchestra.run.smithi129.stdout:2850 writesame 0x3881475 thru 0x3885c3c data_size 0x4(0x47c8 bytes) 2024-09-17T21:30:25.217 INFO:teuthology.orchestra.run.smithi129.stdout:2855 writesame 0x848da8f thru 0x8499a67 data_size 0x11(0xbfd9 bytes) 2024-09-17T21:30:25.222 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.222 INFO:teuthology.orchestra.run.smithi129.stdout:2858 writesame 0x7c2208e thru 0x7c29290 data_size 0x2f(0x7203 bytes) 2024-09-17T21:30:25.224 INFO:teuthology.orchestra.run.smithi129.stdout:2860 punch from 0x848c65e to 0x848e0cc, (0x1a6e bytes) 2024-09-17T21:30:25.224 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.234 INFO:teuthology.orchestra.run.smithi129.stdout:2865 trunc from 0xb52feac to 0x2889503 2024-09-17T21:30:25.259 INFO:teuthology.orchestra.run.smithi129.stdout:2867 trunc from 0x2889503 to 0xd15aac0 2024-09-17T21:30:25.264 INFO:teuthology.orchestra.run.smithi129.stdout:2868 punch from 0x5952e84 to 0x595dbda, (0xad56 bytes) 2024-09-17T21:30:25.264 INFO:teuthology.orchestra.run.smithi129.stdout:2869 write 0x18abb72 thru 0x18b9435 (0xd8c4 bytes) 2024-09-17T21:30:25.265 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:25.262+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:25.267 INFO:teuthology.orchestra.run.smithi129.stdout:2872 trunc from 0xd15aac0 to 0xd669618 2024-09-17T21:30:25.271 INFO:teuthology.orchestra.run.smithi129.stdout:2873 read 0xe2ce16 thru 0xe36c78 (0x9e63 bytes) 2024-09-17T21:30:25.272 INFO:teuthology.orchestra.run.smithi129.stdout:2874 write 0x651424c thru 0x651bb1b (0x78d0 bytes) 2024-09-17T21:30:25.275 INFO:teuthology.orchestra.run.smithi129.stdout:2875 read 0x901f889 thru 0x902ed0b (0xf483 bytes) 2024-09-17T21:30:25.275 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.276 INFO:teuthology.orchestra.run.smithi129.stdout:2878 write 0x6f26859 thru 0x6f2bbc8 (0x5370 bytes) 2024-09-17T21:30:25.278 INFO:teuthology.orchestra.run.smithi129.stdout:2880 read 0x1a9dce9 thru 0x1aad027 (0xf33f bytes) 2024-09-17T21:30:25.279 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:25.279 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:25.280 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:25.280 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:25.280 INFO:teuthology.orchestra.run.smithi129.stdout:2881 writesame 0x83da172 thru 0x83daffe data_size 0x5(0xe8d bytes) 2024-09-17T21:30:25.282 INFO:teuthology.orchestra.run.smithi129.stdout:2882 punch from 0xd3302e7 to 0xd33c2e1, (0xbffa bytes) 2024-09-17T21:30:25.282 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.289 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:30:25.298 INFO:teuthology.orchestra.run.smithi129.stdout:2884 trunc from 0xd669618 to 0x4725d8 2024-09-17T21:30:25.323 INFO:teuthology.orchestra.run.smithi129.stdout:2885 write 0x5f82d8d thru 0x5f8b9b8 (0x8c2c bytes) 2024-09-17T21:30:25.335 INFO:teuthology.orchestra.run.smithi129.stdout:2886 write 0x80e25a4 thru 0x80eb324 (0x8d81 bytes) 2024-09-17T21:30:25.348 INFO:teuthology.orchestra.run.smithi129.stdout:2888 trunc from 0x80eb325 to 0x80f791 2024-09-17T21:30:25.360 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.360 INFO:teuthology.orchestra.run.smithi129.stdout:2891 read 0x3d7cb4 thru 0x3e1965 (0x9cb2 bytes) 2024-09-17T21:30:25.365 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:25.364+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:25.380 INFO:teuthology.orchestra.run.smithi129.stdout:2892 writesame 0xc295e10 thru 0xc2a4188 data_size 0x15(0xe379 bytes) 2024-09-17T21:30:25.381 INFO:teuthology.orchestra.run.smithi129.stdout:2893 write 0x92adb77 thru 0x92b01d3 (0x265d bytes) 2024-09-17T21:30:25.383 INFO:teuthology.orchestra.run.smithi129.stdout:2894 writesame 0x8d72bd2 thru 0x8d7746a data_size 0x9(0x4899 bytes) 2024-09-17T21:30:25.388 INFO:teuthology.orchestra.run.smithi129.stdout:2895 read 0x664b3d2 thru 0x6656ce9 (0xb918 bytes) 2024-09-17T21:30:25.388 INFO:teuthology.orchestra.run.smithi129.stdout:2896 read 0xb38e2a9 thru 0xb39b4eb (0xd243 bytes) 2024-09-17T21:30:25.388 INFO:teuthology.orchestra.run.smithi129.stdout:2897 write 0xbd9a005 thru 0xbda45bb (0xa5b7 bytes) 2024-09-17T21:30:25.396 INFO:teuthology.orchestra.run.smithi129.stdout:2898 read 0x38bc5f thru 0x3925e9 (0x698b bytes) 2024-09-17T21:30:25.398 INFO:teuthology.orchestra.run.smithi129.stdout:2899 read 0x43be06b thru 0x43caef4 (0xce8a bytes) 2024-09-17T21:30:25.398 INFO:teuthology.orchestra.run.smithi129.stdout:2900 writesame 0xb084bc1 thru 0xb085ab6 data_size 0x5(0xef6 bytes) 2024-09-17T21:30:25.399 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.399 INFO:teuthology.orchestra.run.smithi129.stdout:2904 write 0x21b4c1e thru 0x21c49ee (0xfdd1 bytes) 2024-09-17T21:30:25.408 INFO:teuthology.orchestra.run.smithi129.stdout:2905 write 0xd42ad9f thru 0xd433ff5 (0x9257 bytes) 2024-09-17T21:30:25.412 INFO:teuthology.orchestra.run.smithi129.stdout:2906 writesame 0xc0db64a thru 0xc0df73f data_size 0x5(0x40f6 bytes) 2024-09-17T21:30:25.417 INFO:teuthology.orchestra.run.smithi129.stdout:2907 writesame 0xa20abaa thru 0xa210e84 data_size 0x1(0x62db bytes) 2024-09-17T21:30:25.425 INFO:teuthology.orchestra.run.smithi129.stdout:2908 writesame 0x76ea0b3 thru 0x76f01f4 data_size 0x3b(0x6142 bytes) 2024-09-17T21:30:25.428 INFO:teuthology.orchestra.run.smithi129.stdout:2910 punch from 0xa23e683 to 0xa24b24d, (0xcbca bytes) 2024-09-17T21:30:25.428 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.429 INFO:teuthology.orchestra.run.smithi129.stdout:2912 punch from 0x84ac7b2 to 0x84b62fa, (0x9b48 bytes) 2024-09-17T21:30:25.433 INFO:teuthology.orchestra.run.smithi129.stdout:2914 writesame 0xdc262d2 thru 0xdc2948a data_size 0x1(0x31b9 bytes) 2024-09-17T21:30:25.435 INFO:teuthology.orchestra.run.smithi129.stdout:2915 read 0x580d1c1 thru 0x5811115 (0x3f55 bytes) 2024-09-17T21:30:25.435 INFO:teuthology.orchestra.run.smithi129.stdout:2917 writesame 0x11ce635 thru 0x11d6cf9 data_size 0x1(0x86c5 bytes) 2024-09-17T21:30:25.443 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:25.443 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:25.447 INFO:teuthology.orchestra.run.smithi129.stdout:2918 write 0x15efe48 thru 0x15f1c3d (0x1df6 bytes) 2024-09-17T21:30:25.449 INFO:teuthology.orchestra.run.smithi129.stdout:2922 punch from 0xb5f37c to 0xb64f79, (0x5bfd bytes) 2024-09-17T21:30:25.449 INFO:teuthology.orchestra.run.smithi129.stdout:2924 read 0x5ce0e20 thru 0x5cec196 (0xb377 bytes) 2024-09-17T21:30:25.449 INFO:teuthology.orchestra.run.smithi129.stdout:2925 read 0xa5f385b thru 0xa5fff7b (0xc721 bytes) 2024-09-17T21:30:25.449 INFO:teuthology.orchestra.run.smithi129.stdout:2926 write 0xb5aaea4 thru 0xb5b0ad5 (0x5c32 bytes) 2024-09-17T21:30:25.452 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:30:25.453 INFO:teuthology.orchestra.run.smithi129.stdout:2928 read 0x2730eb9 thru 0x273ca11 (0xbb59 bytes) 2024-09-17T21:30:25.453 INFO:teuthology.orchestra.run.smithi129.stdout:2929 write 0x85b6152 thru 0x85c3556 (0xd405 bytes) 2024-09-17T21:30:25.462 INFO:teuthology.orchestra.run.smithi129.stdout:2930 writesame 0xdf13b5c thru 0xdf13ee9 data_size 0xe(0x38e bytes) 2024-09-17T21:30:25.462 INFO:teuthology.orchestra.run.smithi129.stdout:2931 read 0x2608b27 thru 0x2611a3a (0x8f14 bytes) 2024-09-17T21:30:25.462 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.462 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.462 INFO:teuthology.orchestra.run.smithi129.stdout:2935 write 0x533c514 thru 0x5348b63 (0xc650 bytes) 2024-09-17T21:30:25.465 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:25.464+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:25.466 INFO:teuthology.orchestra.run.smithi129.stdout:2936 writesame 0x58492a2 thru 0x584f7c0 data_size 0x1(0x651f bytes) 2024-09-17T21:30:25.469 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:30:25.475 INFO:teuthology.orchestra.run.smithi129.stdout:2937 writesame 0xa4c0ba4 thru 0xa4c4fbf data_size 0x6(0x441c bytes) 2024-09-17T21:30:25.477 INFO:teuthology.orchestra.run.smithi129.stdout:2938 punch from 0x14c88e to 0x15b837, (0xefa9 bytes) 2024-09-17T21:30:25.477 INFO:teuthology.orchestra.run.smithi129.stdout:2940 write 0x297e7c4 thru 0x29867e1 (0x801e bytes) 2024-09-17T21:30:25.481 INFO:teuthology.orchestra.run.smithi129.stdout:2941 read 0xa3dd7ad thru 0xa3e63e5 (0x8c39 bytes) 2024-09-17T21:30:25.482 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.482 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.482 INFO:teuthology.orchestra.run.smithi129.stdout:2944 write 0x59f327 thru 0x5a462d (0x5307 bytes) 2024-09-17T21:30:25.484 INFO:teuthology.orchestra.run.smithi129.stdout:2945 punch from 0x4c8562f to 0x4c882fb, (0x2ccc bytes) 2024-09-17T21:30:25.484 INFO:teuthology.orchestra.run.smithi129.stdout:2947 read 0x14bd371 thru 0x14bfe9c (0x2b2c bytes) 2024-09-17T21:30:25.484 INFO:teuthology.orchestra.run.smithi129.stdout:2949 write 0xd03ef36 thru 0xd04b958 (0xca23 bytes) 2024-09-17T21:30:25.488 INFO:teuthology.orchestra.run.smithi129.stdout:2950 read 0x141d36 thru 0x142590 (0x85b bytes) 2024-09-17T21:30:25.503 INFO:teuthology.orchestra.run.smithi129.stdout:2951 trunc from 0xdf13eea to 0x1f30dbe 2024-09-17T21:30:25.519 INFO:teuthology.orchestra.run.smithi129.stdout:2953 punch from 0xc8d301 to 0xc9a70e, (0xd40d bytes) 2024-09-17T21:30:25.525 INFO:teuthology.orchestra.run.smithi129.stdout:2954 write 0x32d71cd thru 0x32e134f (0xa183 bytes) 2024-09-17T21:30:25.534 INFO:teuthology.orchestra.run.smithi129.stdout:2955 punch from 0xf5c265 to 0xf6a8a4, (0xe63f bytes) 2024-09-17T21:30:25.534 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.534 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.540 INFO:teuthology.orchestra.run.smithi129.stdout:2960 write 0x4375faf thru 0x437c4e9 (0x653b bytes) 2024-09-17T21:30:25.548 INFO:teuthology.orchestra.run.smithi129.stdout:2961 read 0x9780e6 thru 0x983104 (0xb01f bytes) 2024-09-17T21:30:25.548 INFO:teuthology.orchestra.run.smithi129.stdout:2962 punch from 0x2d77315 to 0x2d77852, (0x53d bytes) 2024-09-17T21:30:25.556 INFO:teuthology.orchestra.run.smithi129.stdout:2963 trunc from 0x437c4ea to 0xa48f5e3 2024-09-17T21:30:25.560 INFO:teuthology.orchestra.run.smithi129.stdout:2964 read 0x11826cb thru 0x118785c (0x5192 bytes) 2024-09-17T21:30:25.561 INFO:teuthology.orchestra.run.smithi129.stdout:2967 read 0x3e44cbe thru 0x3e48d4f (0x4092 bytes) 2024-09-17T21:30:25.561 INFO:teuthology.orchestra.run.smithi129.stdout:2968 read 0x45f2dac thru 0x45fd7b6 (0xaa0b bytes) 2024-09-17T21:30:25.566 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:25.566 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:25.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:25.566 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:25.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.566 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:30:25.566 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:25.566 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.623320341000003, 2024-09-17T21:30:25.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61572000699999996, 2024-09-17T21:30:25.566 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.566 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.567 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:25.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:30:25.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:30:25.567 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T21:30:25.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.567 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:25.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:25.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939745+0000", 2024-09-17T21:30:25.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6899999999999999e-06 2024-09-17T21:30:25.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939753+0000", 2024-09-17T21:30:25.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1950000000000003e-06 2024-09-17T21:30:25.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939767+0000", 2024-09-17T21:30:25.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3429e-05 2024-09-17T21:30:25.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939776+0000", 2024-09-17T21:30:25.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:30:25.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939858+0000", 2024-09-17T21:30:25.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2496000000000004e-05 2024-09-17T21:30:25.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T21:30:25.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939868+0000", 2024-09-17T21:30:25.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1910000000000006e-06 2024-09-17T21:30:25.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940612+0000", 2024-09-17T21:30:25.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074429599999999998 2024-09-17T21:30:25.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:30:25.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940621+0000", 2024-09-17T21:30:25.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9139999999999997e-06 2024-09-17T21:30:25.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554006+0000", 2024-09-17T21:30:25.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61338505099999996 2024-09-17T21:30:25.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554166+0000", 2024-09-17T21:30:25.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015982600000000001 2024-09-17T21:30:25.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:30:25.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554296+0000", 2024-09-17T21:30:25.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013002399999999999 2024-09-17T21:30:25.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:25.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554936+0000", 2024-09-17T21:30:25.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064009400000000001 2024-09-17T21:30:25.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:25.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555389+0000", 2024-09-17T21:30:25.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045355200000000002 2024-09-17T21:30:25.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:25.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555432+0000", 2024-09-17T21:30:25.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2477e-05 2024-09-17T21:30:25.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555463+0000", 2024-09-17T21:30:25.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1612000000000002e-05 2024-09-17T21:30:25.573 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.573 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.573 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.574 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17527 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1005568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.574 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:25.574 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.158092510000003, 2024-09-17T21:30:25.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046452090000000001, 2024-09-17T21:30:25.574 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.574 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.574 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.574 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.574 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.574 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.574 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17527 2024-09-17T21:30:25.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.575 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:25.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:25.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405496+0000", 2024-09-17T21:30:25.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052498400000000004 2024-09-17T21:30:25.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405500+0000", 2024-09-17T21:30:25.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8410000000000002e-06 2024-09-17T21:30:25.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405511+0000", 2024-09-17T21:30:25.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0749e-05 2024-09-17T21:30:25.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405517+0000", 2024-09-17T21:30:25.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:30:25.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451275+0000", 2024-09-17T21:30:25.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045758134999999998 2024-09-17T21:30:25.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451355+0000", 2024-09-17T21:30:25.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9453e-05 2024-09-17T21:30:25.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451423+0000", 2024-09-17T21:30:25.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8782999999999995e-05 2024-09-17T21:30:25.578 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.578 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.578 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.578 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17530 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1046528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.578 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:25.578 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.157484859, 2024-09-17T21:30:25.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046376737000000001, 2024-09-17T21:30:25.579 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.579 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.579 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.579 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.579 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.579 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.579 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17530 2024-09-17T21:30:25.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.579 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:25.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:25.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405579+0000", 2024-09-17T21:30:25.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3300000000000002e-07 2024-09-17T21:30:25.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405581+0000", 2024-09-17T21:30:25.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.074e-06 2024-09-17T21:30:25.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405587+0000", 2024-09-17T21:30:25.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7559999999999996e-06 2024-09-17T21:30:25.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405592+0000", 2024-09-17T21:30:25.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8860000000000003e-06 2024-09-17T21:30:25.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451813+0000", 2024-09-17T21:30:25.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046221119999999997 2024-09-17T21:30:25.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451891+0000", 2024-09-17T21:30:25.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7449999999999999e-05 2024-09-17T21:30:25.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451956+0000", 2024-09-17T21:30:25.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4918000000000004e-05 2024-09-17T21:30:25.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.583 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.583 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17615 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.583 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:25.583 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.152852021999998, 2024-09-17T21:30:25.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046741309000000002, 2024-09-17T21:30:25.583 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.583 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.583 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.583 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17615 2024-09-17T21:30:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:30:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410213+0000", 2024-09-17T21:30:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:30:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410217+0000", 2024-09-17T21:30:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8199999999999998e-06 2024-09-17T21:30:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410220+0000", 2024-09-17T21:30:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6900000000000001e-06 2024-09-17T21:30:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455264+0000", 2024-09-17T21:30:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045044002999999999 2024-09-17T21:30:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455338+0000", 2024-09-17T21:30:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4182000000000002e-05 2024-09-17T21:30:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.456953+0000", 2024-09-17T21:30:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016150089999999999 2024-09-17T21:30:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17616 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 872448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.152839970000002, 2024-09-17T21:30:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946039000000002, 2024-09-17T21:30:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17616 2024-09-17T21:30:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T21:30:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410225+0000", 2024-09-17T21:30:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3459999999999999e-06 2024-09-17T21:30:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410228+0000", 2024-09-17T21:30:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.083e-06 2024-09-17T21:30:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410232+0000", 2024-09-17T21:30:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4419999999999998e-06 2024-09-17T21:30:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457004+0000", 2024-09-17T21:30:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046771873999999998 2024-09-17T21:30:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457084+0000", 2024-09-17T21:30:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0506000000000005e-05 2024-09-17T21:30:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457170+0000", 2024-09-17T21:30:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5494000000000005e-05 2024-09-17T21:30:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17617 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.152828567999997, 2024-09-17T21:30:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047084168000000003, 2024-09-17T21:30:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17617 2024-09-17T21:30:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-17T21:30:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410237+0000", 2024-09-17T21:30:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1960000000000001e-06 2024-09-17T21:30:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410246+0000", 2024-09-17T21:30:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7880000000000001e-06 2024-09-17T21:30:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410250+0000", 2024-09-17T21:30:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.518e-06 2024-09-17T21:30:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457209+0000", 2024-09-17T21:30:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046959283999999997 2024-09-17T21:30:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457264+0000", 2024-09-17T21:30:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673999999999999e-05 2024-09-17T21:30:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457319+0000", 2024-09-17T21:30:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5497000000000001e-05 2024-09-17T21:30:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17618 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 899072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.152809124000001, 2024-09-17T21:30:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047204208999999997, 2024-09-17T21:30:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17618 2024-09-17T21:30:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:25.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:25.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8799999999999998e-07 2024-09-17T21:30:25.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410256+0000", 2024-09-17T21:30:25.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T21:30:25.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410260+0000", 2024-09-17T21:30:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6500000000000002e-06 2024-09-17T21:30:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410263+0000", 2024-09-17T21:30:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7089999999999998e-06 2024-09-17T21:30:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457355+0000", 2024-09-17T21:30:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047092756999999999 2024-09-17T21:30:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457407+0000", 2024-09-17T21:30:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1808999999999998e-05 2024-09-17T21:30:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457459+0000", 2024-09-17T21:30:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1684000000000002e-05 2024-09-17T21:30:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17619 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 913408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.152797886000002, 2024-09-17T21:30:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047293443999999997, 2024-09-17T21:30:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17619 2024-09-17T21:30:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:30:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410267+0000", 2024-09-17T21:30:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:30:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410270+0000", 2024-09-17T21:30:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.01e-06 2024-09-17T21:30:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410273+0000", 2024-09-17T21:30:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6410000000000002e-06 2024-09-17T21:30:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457473+0000", 2024-09-17T21:30:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047199829999999998 2024-09-17T21:30:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457517+0000", 2024-09-17T21:30:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4058999999999999e-05 2024-09-17T21:30:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457559+0000", 2024-09-17T21:30:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2404999999999998e-05 2024-09-17T21:30:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17620 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 927744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.152786714000001, 2024-09-17T21:30:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047402409, 2024-09-17T21:30:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17620 2024-09-17T21:30:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:30:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410278+0000", 2024-09-17T21:30:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1790000000000001e-06 2024-09-17T21:30:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410281+0000", 2024-09-17T21:30:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.061e-06 2024-09-17T21:30:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410284+0000", 2024-09-17T21:30:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.65e-06 2024-09-17T21:30:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457587+0000", 2024-09-17T21:30:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047303206 2024-09-17T21:30:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457633+0000", 2024-09-17T21:30:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6063999999999998e-05 2024-09-17T21:30:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457679+0000", 2024-09-17T21:30:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6007000000000002e-05 2024-09-17T21:30:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17621 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 940032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.152776297999999, 2024-09-17T21:30:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047486550000000002, 2024-09-17T21:30:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17621 2024-09-17T21:30:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410288+0000", 2024-09-17T21:30:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:30:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410289+0000", 2024-09-17T21:30:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1179999999999999e-06 2024-09-17T21:30:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410292+0000", 2024-09-17T21:30:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:30:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410294+0000", 2024-09-17T21:30:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T21:30:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457694+0000", 2024-09-17T21:30:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047399416999999999 2024-09-17T21:30:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457734+0000", 2024-09-17T21:30:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0528999999999998e-05 2024-09-17T21:30:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457774+0000", 2024-09-17T21:30:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9529e-05 2024-09-17T21:30:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17622 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.152766116000002, 2024-09-17T21:30:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047596619, 2024-09-17T21:30:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17622 2024-09-17T21:30:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410303+0000", 2024-09-17T21:30:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2979999999999999e-06 2024-09-17T21:30:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410307+0000", 2024-09-17T21:30:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:30:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410310+0000", 2024-09-17T21:30:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.146e-06 2024-09-17T21:30:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457801+0000", 2024-09-17T21:30:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047491322000000002 2024-09-17T21:30:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457850+0000", 2024-09-17T21:30:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8585000000000003e-05 2024-09-17T21:30:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457894+0000", 2024-09-17T21:30:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4455e-05 2024-09-17T21:30:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17623 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 968704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.152749731999997, 2024-09-17T21:30:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047689377999999998, 2024-09-17T21:30:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17623 2024-09-17T21:30:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:30:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410315+0000", 2024-09-17T21:30:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0499999999999999e-06 2024-09-17T21:30:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410319+0000", 2024-09-17T21:30:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.517e-06 2024-09-17T21:30:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410321+0000", 2024-09-17T21:30:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:30:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457922+0000", 2024-09-17T21:30:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047600638000000001 2024-09-17T21:30:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457964+0000", 2024-09-17T21:30:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2126000000000003e-05 2024-09-17T21:30:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458003+0000", 2024-09-17T21:30:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9138999999999999e-05 2024-09-17T21:30:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17624 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.152739148000002, 2024-09-17T21:30:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047798292999999999, 2024-09-17T21:30:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17624 2024-09-17T21:30:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:30:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410326+0000", 2024-09-17T21:30:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0610000000000001e-06 2024-09-17T21:30:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410329+0000", 2024-09-17T21:30:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0319999999999999e-06 2024-09-17T21:30:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410331+0000", 2024-09-17T21:30:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5749999999999999e-06 2024-09-17T21:30:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458033+0000", 2024-09-17T21:30:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047701371999999999 2024-09-17T21:30:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458079+0000", 2024-09-17T21:30:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:30:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458123+0000", 2024-09-17T21:30:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850999999999997e-05 2024-09-17T21:30:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17625 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.152728988, 2024-09-17T21:30:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047883051000000003, 2024-09-17T21:30:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17625 2024-09-17T21:30:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410336+0000", 2024-09-17T21:30:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1310000000000001e-06 2024-09-17T21:30:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410339+0000", 2024-09-17T21:30:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000001e-06 2024-09-17T21:30:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410342+0000", 2024-09-17T21:30:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7460000000000001e-06 2024-09-17T21:30:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458137+0000", 2024-09-17T21:30:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047794931999999998 2024-09-17T21:30:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458178+0000", 2024-09-17T21:30:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1013999999999997e-05 2024-09-17T21:30:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458218+0000", 2024-09-17T21:30:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9675999999999998e-05 2024-09-17T21:30:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17626 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1009664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.152718366999999, 2024-09-17T21:30:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047991601000000002, 2024-09-17T21:30:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17626 2024-09-17T21:30:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410346+0000", 2024-09-17T21:30:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:30:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410347+0000", 2024-09-17T21:30:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.114e-06 2024-09-17T21:30:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410350+0000", 2024-09-17T21:30:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.101e-06 2024-09-17T21:30:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410353+0000", 2024-09-17T21:30:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T21:30:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458245+0000", 2024-09-17T21:30:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0478921 2024-09-17T21:30:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458292+0000", 2024-09-17T21:30:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397999999999999e-05 2024-09-17T21:30:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458337+0000", 2024-09-17T21:30:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5221e-05 2024-09-17T21:30:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17627 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.152681266000002, 2024-09-17T21:30:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048047383999999999, 2024-09-17T21:30:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17627 2024-09-17T21:30:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410383+0000", 2024-09-17T21:30:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:30:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410385+0000", 2024-09-17T21:30:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:30:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410391+0000", 2024-09-17T21:30:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:30:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410395+0000", 2024-09-17T21:30:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6539999999999998e-06 2024-09-17T21:30:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458350+0000", 2024-09-17T21:30:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047954855999999997 2024-09-17T21:30:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458392+0000", 2024-09-17T21:30:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2094000000000001e-05 2024-09-17T21:30:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458430+0000", 2024-09-17T21:30:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7474000000000001e-05 2024-09-17T21:30:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17628 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.152661705, 2024-09-17T21:30:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048142676000000002, 2024-09-17T21:30:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17628 2024-09-17T21:30:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:30:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410404+0000", 2024-09-17T21:30:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-06 2024-09-17T21:30:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410411+0000", 2024-09-17T21:30:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6850000000000004e-06 2024-09-17T21:30:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410415+0000", 2024-09-17T21:30:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6750000000000001e-06 2024-09-17T21:30:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458457+0000", 2024-09-17T21:30:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048042045999999998 2024-09-17T21:30:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458501+0000", 2024-09-17T21:30:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3661000000000001e-05 2024-09-17T21:30:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458545+0000", 2024-09-17T21:30:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3529000000000003e-05 2024-09-17T21:30:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17687 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.150216751999999, 2024-09-17T21:30:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050670448, 2024-09-17T21:30:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17687 2024-09-17T21:30:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412848+0000", 2024-09-17T21:30:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9100000000000003e-07 2024-09-17T21:30:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413102+0000", 2024-09-17T21:30:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025415600000000001 2024-09-17T21:30:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413111+0000", 2024-09-17T21:30:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2010000000000005e-06 2024-09-17T21:30:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413116+0000", 2024-09-17T21:30:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:30:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458559+0000", 2024-09-17T21:30:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045442914000000001 2024-09-17T21:30:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458632+0000", 2024-09-17T21:30:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3207000000000006e-05 2024-09-17T21:30:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458785+0000", 2024-09-17T21:30:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015226700000000001 2024-09-17T21:30:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.461648+0000", 2024-09-17T21:30:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028633069999999998 2024-09-17T21:30:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463416+0000", 2024-09-17T21:30:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017680230000000001 2024-09-17T21:30:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463475+0000", 2024-09-17T21:30:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9333000000000003e-05 2024-09-17T21:30:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463517+0000", 2024-09-17T21:30:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2162000000000001e-05 2024-09-17T21:30:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17688 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [write 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.149937884000003, 2024-09-17T21:30:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050625098, 2024-09-17T21:30:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17688 2024-09-17T21:30:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:30:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413128+0000", 2024-09-17T21:30:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.813e-06 2024-09-17T21:30:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413132+0000", 2024-09-17T21:30:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:30:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413135+0000", 2024-09-17T21:30:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:30:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459756+0000", 2024-09-17T21:30:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046621293000000001 2024-09-17T21:30:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459816+0000", 2024-09-17T21:30:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9156000000000001e-05 2024-09-17T21:30:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459912+0000", 2024-09-17T21:30:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6302000000000001e-05 2024-09-17T21:30:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463231+0000", 2024-09-17T21:30:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033188290000000001 2024-09-17T21:30:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463701+0000", 2024-09-17T21:30:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047068199999999999 2024-09-17T21:30:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463728+0000", 2024-09-17T21:30:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6726000000000001e-05 2024-09-17T21:30:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463751+0000", 2024-09-17T21:30:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2824e-05 2024-09-17T21:30:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17691 3.d 3:b3a055f3:::rbd_data.11207744cc2c.0000000000000083:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.146392980999998, 2024-09-17T21:30:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049309370999999998, 2024-09-17T21:30:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17691 2024-09-17T21:30:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416672+0000", 2024-09-17T21:30:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.47e-07 2024-09-17T21:30:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416915+0000", 2024-09-17T21:30:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024344500000000001 2024-09-17T21:30:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416924+0000", 2024-09-17T21:30:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7739999999999993e-06 2024-09-17T21:30:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416929+0000", 2024-09-17T21:30:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6990000000000002e-06 2024-09-17T21:30:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460414+0000", 2024-09-17T21:30:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043485466 2024-09-17T21:30:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460540+0000", 2024-09-17T21:30:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012608899999999999 2024-09-17T21:30:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460746+0000", 2024-09-17T21:30:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000205577 2024-09-17T21:30:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463800+0000", 2024-09-17T21:30:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030542030000000001 2024-09-17T21:30:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465867+0000", 2024-09-17T21:30:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020665929999999998 2024-09-17T21:30:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465914+0000", 2024-09-17T21:30:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497000000000003e-05 2024-09-17T21:30:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465980+0000", 2024-09-17T21:30:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6081000000000006e-05 2024-09-17T21:30:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:25.666 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:25.668 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:25.665+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:25.668 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:30:25.669 INFO:teuthology.orchestra.run.smithi129.stdout:2969 trunc from 0xa48f5e3 to 0x7a1da55 2024-09-17T21:30:25.669 INFO:teuthology.orchestra.run.smithi129.stdout:2970 trunc from 0x7a1da55 to 0xed6e109 2024-09-17T21:30:25.669 INFO:teuthology.orchestra.run.smithi129.stdout:2972 write 0xb99d389 thru 0xb9a37f9 (0x6471 bytes) 2024-09-17T21:30:25.669 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.669 INFO:teuthology.orchestra.run.smithi129.stdout:2975 trunc from 0xed6e109 to 0xa337872 2024-09-17T21:30:25.669 INFO:teuthology.orchestra.run.smithi129.stdout:2977 read 0x5d65a21 thru 0x5d6866e (0x2c4e bytes) 2024-09-17T21:30:25.669 INFO:teuthology.orchestra.run.smithi129.stdout:2978 read 0x3b06b6b thru 0x3b1582a (0xecc0 bytes) 2024-09-17T21:30:25.669 INFO:teuthology.orchestra.run.smithi129.stdout:2979 read 0x29b8509 thru 0x29bc555 (0x404d bytes) 2024-09-17T21:30:25.669 INFO:teuthology.orchestra.run.smithi129.stdout:2981 write 0x55ef8d6 thru 0x55fcda3 (0xd4ce bytes) 2024-09-17T21:30:25.670 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:30:25.670 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-17T21:30:25.671 INFO:teuthology.orchestra.run.smithi129.stderr:0 2024-09-17T21:30:25.671 INFO:teuthology.orchestra.run.smithi129.stderr:1 2024-09-17T21:30:25.671 INFO:teuthology.orchestra.run.smithi129.stderr:2 2024-09-17T21:30:25.671 INFO:teuthology.orchestra.run.smithi129.stderr:abort 2024-09-17T21:30:25.671 INFO:teuthology.orchestra.run.smithi129.stderr:assert 2024-09-17T21:30:25.671 INFO:teuthology.orchestra.run.smithi129.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:30:25.671 INFO:teuthology.orchestra.run.smithi129.stderr:bluefs files list 2024-09-17T21:30:25.671 INFO:teuthology.orchestra.run.smithi129.stderr:bluefs stats 2024-09-17T21:30:25.671 INFO:teuthology.orchestra.run.smithi129.stderr:bluestore allocator dump block 2024-09-17T21:30:25.671 INFO:teuthology.orchestra.run.smithi129.stderr:bluestore allocator fragmentation block 2024-09-17T21:30:25.671 INFO:teuthology.orchestra.run.smithi129.stderr:admin_socket: invalid command 2024-09-17T21:30:25.672 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-17T21:30:25.672 INFO:teuthology.orchestra.run.smithi129.stdout:2982 trunc from 0xa337872 to 0x2a9cfe9 2024-09-17T21:30:25.672 INFO:teuthology.orchestra.run.smithi129.stdout:2985 writesame 0x4994d63 thru 0x49a2a77 data_size 0x1(0xdd15 bytes) 2024-09-17T21:30:25.672 INFO:teuthology.orchestra.run.smithi129.stdout:2986 read 0x1122d03 thru 0x1125133 (0x2431 bytes) 2024-09-17T21:30:25.673 INFO:teuthology.orchestra.run.smithi129.stdout:2987 write 0xb47a9a3 thru 0xb48334f (0x89ad bytes) 2024-09-17T21:30:25.673 INFO:teuthology.orchestra.run.smithi129.stdout:2988 trunc from 0xb483350 to 0x54201de 2024-09-17T21:30:25.678 INFO:teuthology.orchestra.run.smithi129.stdout:2989 write 0x49686ec thru 0x497686e (0xe183 bytes) 2024-09-17T21:30:25.688 INFO:teuthology.orchestra.run.smithi129.stdout:2990 write 0x62aec03 thru 0x62b0e1e (0x221c bytes) 2024-09-17T21:30:25.704 INFO:teuthology.orchestra.run.smithi129.stdout:2991 write 0xc0e3566 thru 0xc0e477f (0x121a bytes) 2024-09-17T21:30:25.706 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.707 INFO:teuthology.orchestra.run.smithi129.stdout:2993 read 0xb936ab9 thru 0xb9415c4 (0xab0c bytes) 2024-09-17T21:30:25.707 INFO:teuthology.orchestra.run.smithi129.stdout:2994 read 0x1a78dd9 thru 0x1a87f7d (0xf1a5 bytes) 2024-09-17T21:30:25.707 INFO:teuthology.orchestra.run.smithi129.stdout:2996 read 0x51f6a6c thru 0x51ff7ea (0x8d7f bytes) 2024-09-17T21:30:25.708 INFO:teuthology.orchestra.run.smithi129.stdout:2997 read 0x9096eb2 thru 0x909b7c1 (0x4910 bytes) 2024-09-17T21:30:25.708 INFO:teuthology.orchestra.run.smithi129.stdout:2998 read 0x6c8196b thru 0x6c90cad (0xf343 bytes) 2024-09-17T21:30:25.714 INFO:teuthology.orchestra.run.smithi129.stdout:2999 trunc from 0xc0e4780 to 0x6fc213d 2024-09-17T21:30:25.716 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is unset 2024-09-17T21:30:25.734 INFO:teuthology.orchestra.run.smithi129.stdout:3000 read 0x1fc9be thru 0x1ffe07 (0x344a bytes) 2024-09-17T21:30:25.734 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.745 INFO:teuthology.orchestra.run.smithi129.stdout:3002 write 0xb8d6f7c thru 0xb8e5265 (0xe2ea bytes) 2024-09-17T21:30:25.756 INFO:teuthology.orchestra.run.smithi129.stdout:3003 punch from 0x40a1e2e to 0x40adc02, (0xbdd4 bytes) 2024-09-17T21:30:25.756 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.756 INFO:teuthology.orchestra.run.smithi129.stdout:3005 read 0x552da61 thru 0x55334f4 (0x5a94 bytes) 2024-09-17T21:30:25.756 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.757 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.757 INFO:teuthology.orchestra.run.smithi129.stdout:3009 trunc from 0xb8e5266 to 0xc40bdf1 2024-09-17T21:30:25.762 INFO:teuthology.orchestra.run.smithi129.stdout:3010 write 0x2110d84 thru 0x211f6e8 (0xe965 bytes) 2024-09-17T21:30:25.764 INFO:teuthology.orchestra.run.smithi129.stdout:3011 write 0xe936e8 thru 0xe96a24 (0x333d bytes) 2024-09-17T21:30:25.767 INFO:teuthology.orchestra.run.smithi129.stdout:3012 write 0x2767f54 thru 0x2771d54 (0x9e01 bytes) 2024-09-17T21:30:25.767 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:25.766+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:25.768 INFO:teuthology.orchestra.run.smithi129.stdout:3013 read 0x153dd01 thru 0x154948b (0xb78b bytes) 2024-09-17T21:30:25.782 INFO:teuthology.orchestra.run.smithi129.stdout:3015 trunc from 0xc40bdf1 to 0x127c9dc 2024-09-17T21:30:25.802 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:25.802 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:25.802 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:25.802 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:25.802 INFO:teuthology.orchestra.run.smithi129.stdout:3017 trunc from 0x127c9dc to 0x70701ea 2024-09-17T21:30:25.808 INFO:teuthology.orchestra.run.smithi129.stdout:3018 punch from 0x41d6320 to 0x41e1072, (0xad52 bytes) 2024-09-17T21:30:25.808 INFO:teuthology.orchestra.run.smithi129.stdout:3019 punch from 0x6cef55 to 0x6d11b3, (0x225e bytes) 2024-09-17T21:30:25.808 INFO:teuthology.orchestra.run.smithi129.stdout:3020 read 0x412f605 thru 0x4130dc4 (0x17c0 bytes) 2024-09-17T21:30:25.808 INFO:teuthology.orchestra.run.smithi129.stdout:3023 read 0x61683cb thru 0x6176d13 (0xe949 bytes) 2024-09-17T21:30:25.808 INFO:teuthology.orchestra.run.smithi129.stdout:3024 read 0xfbbbe9 thru 0xfbdb3c (0x1f54 bytes) 2024-09-17T21:30:25.808 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.808 INFO:teuthology.orchestra.run.smithi129.stdout:3028 read 0x247a3bd thru 0x24856e3 (0xb327 bytes) 2024-09-17T21:30:25.809 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.809 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.809 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.809 INFO:teuthology.orchestra.run.smithi129.stdout:3034 read 0x6e3cd57 thru 0x6e3e689 (0x1933 bytes) 2024-09-17T21:30:25.809 INFO:teuthology.orchestra.run.smithi129.stdout:3035 writesame 0x5bb2f2c thru 0x5bbe114 data_size 0x1(0xb1e9 bytes) 2024-09-17T21:30:25.811 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:30:25.836 INFO:teuthology.orchestra.run.smithi129.stdout:3036 write 0xdd10b14 thru 0xdd13e9d (0x338a bytes) 2024-09-17T21:30:25.838 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.838 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.839 INFO:teuthology.orchestra.run.smithi129.stdout:3041 read 0xbf28936 thru 0xbf2d827 (0x4ef2 bytes) 2024-09-17T21:30:25.839 INFO:teuthology.orchestra.run.smithi129.stdout:3044 read 0x2753a2a thru 0x275c0e6 (0x86bd bytes) 2024-09-17T21:30:25.839 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.839 INFO:teuthology.orchestra.run.smithi129.stdout:3046 punch from 0x140d57d to 0x140f4ad, (0x1f30 bytes) 2024-09-17T21:30:25.839 INFO:teuthology.orchestra.run.smithi129.stdout:3047 write 0x216b979 thru 0x2174acc (0x9154 bytes) 2024-09-17T21:30:25.841 INFO:teuthology.orchestra.run.smithi129.stdout:3050 punch from 0xdb294b4 to 0xdb370bd, (0xdc09 bytes) 2024-09-17T21:30:25.841 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.841 INFO:teuthology.orchestra.run.smithi129.stdout:3052 write 0x8dfd256 thru 0x8dfec03 (0x19ae bytes) 2024-09-17T21:30:25.843 INFO:teuthology.orchestra.run.smithi129.stdout:3053 write 0xc353b6f thru 0xc35c1b8 (0x864a bytes) 2024-09-17T21:30:25.847 INFO:teuthology.orchestra.run.smithi129.stdout:3054 write 0x4912120 thru 0x4913f7a (0x1e5b bytes) 2024-09-17T21:30:25.849 INFO:teuthology.orchestra.run.smithi129.stdout:3055 read 0xaec587 thru 0xaf5a38 (0x94b2 bytes) 2024-09-17T21:30:25.850 INFO:teuthology.orchestra.run.smithi129.stdout:3056 write 0xd0915f5 thru 0xd09280f (0x121b bytes) 2024-09-17T21:30:25.866 INFO:teuthology.orchestra.run.smithi129.stdout:3057 trunc from 0xdd13e9e to 0x1b0aca1 2024-09-17T21:30:25.868 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:25.867+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:25.882 INFO:teuthology.orchestra.run.smithi129.stdout:3058 trunc from 0x1b0aca1 to 0xc30436 2024-09-17T21:30:25.888 INFO:teuthology.orchestra.run.smithi129.stdout:3059 read 0x74c9fd thru 0x755eba (0x94be bytes) 2024-09-17T21:30:25.889 INFO:teuthology.orchestra.run.smithi129.stdout:3060 read 0x6ff79 thru 0x7ceba (0xcf42 bytes) 2024-09-17T21:30:25.896 INFO:teuthology.orchestra.run.smithi129.stdout:3062 writesame 0x1ba4c2a thru 0x1baaf42 data_size 0x1(0x6319 bytes) 2024-09-17T21:30:25.897 INFO:teuthology.orchestra.run.smithi129.stdout:3063 read 0x5c3b72 thru 0x5cf5c9 (0xba58 bytes) 2024-09-17T21:30:25.898 INFO:teuthology.orchestra.run.smithi129.stdout:3064 read 0xbaf5de thru 0xbb0329 (0xd4c bytes) 2024-09-17T21:30:25.898 INFO:teuthology.orchestra.run.smithi129.stdout:3067 read 0x1402422 thru 0x140fcf4 (0xd8d3 bytes) 2024-09-17T21:30:25.898 INFO:teuthology.orchestra.run.smithi129.stdout:3068 punch from 0x144787f to 0x145653a, (0xecbb bytes) 2024-09-17T21:30:25.901 INFO:teuthology.orchestra.run.smithi129.stdout:3069 trunc from 0x1baaf43 to 0x3bd46ae 2024-09-17T21:30:25.906 INFO:teuthology.orchestra.run.smithi129.stdout:3071 write 0x1eb6d4b thru 0x1ebf659 (0x890f bytes) 2024-09-17T21:30:25.908 INFO:teuthology.orchestra.run.smithi129.stdout:3073 read 0x1e4848d thru 0x1e57e28 (0xf99c bytes) 2024-09-17T21:30:25.909 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.909 INFO:teuthology.orchestra.run.smithi129.stdout:3077 trunc from 0x3bd46ae to 0x3e2fcde 2024-09-17T21:30:25.913 INFO:teuthology.orchestra.run.smithi129.stdout:3078 read 0x3de7f11 thru 0x3de91b5 (0x12a5 bytes) 2024-09-17T21:30:25.913 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.927 INFO:teuthology.orchestra.run.smithi129.stdout:3080 write 0xc25d10b thru 0xc261de6 (0x4cdc bytes) 2024-09-17T21:30:25.932 INFO:teuthology.orchestra.run.smithi129.stdout:3082 trunc from 0xc261de7 to 0xae44161 2024-09-17T21:30:25.937 INFO:teuthology.orchestra.run.smithi129.stdout:3084 read 0x24ba16f thru 0x24c01c6 (0x6058 bytes) 2024-09-17T21:30:25.938 INFO:teuthology.orchestra.run.smithi129.stdout:3085 read 0x5848815 thru 0x5852d8f (0xa57b bytes) 2024-09-17T21:30:25.938 INFO:teuthology.orchestra.run.smithi129.stdout:3086 write 0x474301f thru 0x4743b19 (0xafb bytes) 2024-09-17T21:30:25.940 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.940 INFO:teuthology.orchestra.run.smithi129.stdout:3088 punch from 0x39d0456 to 0x39df03b, (0xebe5 bytes) 2024-09-17T21:30:25.951 INFO:teuthology.orchestra.run.smithi129.stdout:3090 trunc from 0xae44161 to 0x1ab8451 2024-09-17T21:30:25.961 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:25.961 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:25.961 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:25.961 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:25.961 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:25.969 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:25.967+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:25.973 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:30:25.984 INFO:teuthology.orchestra.run.smithi129.stdout:3091 writesame 0xee63320 thru 0xee6b27f data_size 0x1(0x7f60 bytes) 2024-09-17T21:30:25.985 INFO:teuthology.orchestra.run.smithi129.stdout:3092 writesame 0x86e8920 thru 0x86ede84 data_size 0x15(0x5565 bytes) 2024-09-17T21:30:25.987 INFO:teuthology.orchestra.run.smithi129.stdout:3093 write 0x98e0a6b thru 0x98ef6ca (0xec60 bytes) 2024-09-17T21:30:25.991 INFO:teuthology.orchestra.run.smithi129.stdout:3094 read 0x8d30d8e thru 0x8d38483 (0x76f6 bytes) 2024-09-17T21:30:25.991 INFO:teuthology.orchestra.run.smithi129.stdout:3095 write 0x45713c9 thru 0x4573e09 (0x2a41 bytes) 2024-09-17T21:30:25.993 INFO:teuthology.orchestra.run.smithi129.stdout:3096 punch from 0x7ee3b58 to 0x7ee9db4, (0x625c bytes) 2024-09-17T21:30:25.994 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.994 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.994 INFO:teuthology.orchestra.run.smithi129.stdout:3101 read 0x3fed0a9 thru 0x3ffa932 (0xd88a bytes) 2024-09-17T21:30:25.994 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:25.994 INFO:teuthology.orchestra.run.smithi129.stdout:3103 read 0xe8d86a3 thru 0xe8ddab9 (0x5417 bytes) 2024-09-17T21:30:25.994 INFO:teuthology.orchestra.run.smithi129.stdout:3104 punch from 0x75c0282 to 0x75ce3f5, (0xe173 bytes) 2024-09-17T21:30:25.995 INFO:teuthology.orchestra.run.smithi129.stdout:3105 write 0x5a6fd04 thru 0x5a75f31 (0x622e bytes) 2024-09-17T21:30:25.998 INFO:teuthology.orchestra.run.smithi129.stdout:3107 read 0xa7b3d7a thru 0xa7b5d92 (0x2019 bytes) 2024-09-17T21:30:25.998 INFO:teuthology.orchestra.run.smithi129.stdout:3108 read 0xaa91e1e thru 0xaaa0779 (0xe95c bytes) 2024-09-17T21:30:25.998 INFO:teuthology.orchestra.run.smithi129.stdout:3109 read 0xe6f0eaf thru 0xe6f3cb1 (0x2e03 bytes) 2024-09-17T21:30:25.998 INFO:teuthology.orchestra.run.smithi129.stdout:3111 punch from 0x9f540e1 to 0x9f5d950, (0x986f bytes) 2024-09-17T21:30:25.999 INFO:teuthology.orchestra.run.smithi129.stdout:3112 write 0x2d2494 thru 0x2de53b (0xc0a8 bytes) 2024-09-17T21:30:26.000 INFO:teuthology.orchestra.run.smithi129.stdout:3116 read 0x4e738f5 thru 0x4e81191 (0xd89d bytes) 2024-09-17T21:30:26.001 INFO:teuthology.orchestra.run.smithi129.stdout:3117 punch from 0x7f32080 to 0x7f33d65, (0x1ce5 bytes) 2024-09-17T21:30:26.001 INFO:teuthology.orchestra.run.smithi129.stdout:3118 write 0xcd39bb5 thru 0xcd40b6e (0x6fba bytes) 2024-09-17T21:30:26.012 INFO:teuthology.orchestra.run.smithi129.stdout:3119 trunc from 0xee6b280 to 0xa6169de 2024-09-17T21:30:26.024 INFO:teuthology.orchestra.run.smithi129.stdout:3120 trunc from 0xa6169de to 0x77b4e95 2024-09-17T21:30:26.036 INFO:teuthology.orchestra.run.smithi129.stdout:3121 read 0x53b9293 thru 0x53c1d96 (0x8b04 bytes) 2024-09-17T21:30:26.038 INFO:teuthology.orchestra.run.smithi129.stdout:3122 trunc from 0x77b4e95 to 0x863c2b7 2024-09-17T21:30:26.043 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.043 INFO:teuthology.orchestra.run.smithi129.stdout:3125 write 0x2abefa thru 0x2b4b04 (0x8c0b bytes) 2024-09-17T21:30:26.058 INFO:teuthology.orchestra.run.smithi129.stdout:3126 write 0xec66693 thru 0xec76426 (0xfd94 bytes) 2024-09-17T21:30:26.070 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:26.068+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:26.071 INFO:teuthology.orchestra.run.smithi129.stdout:3127 read 0x2f84e0a thru 0x2f8ae3c (0x6033 bytes) 2024-09-17T21:30:26.071 INFO:teuthology.orchestra.run.smithi129.stdout:3128 write 0x1944df6 thru 0x194aa37 (0x5c42 bytes) 2024-09-17T21:30:26.072 INFO:teuthology.orchestra.run.smithi129.stdout:3129 read 0x9529b73 thru 0x952b5fa (0x1a88 bytes) 2024-09-17T21:30:26.072 INFO:teuthology.orchestra.run.smithi129.stdout:3130 write 0x840b854 thru 0x840efb0 (0x375d bytes) 2024-09-17T21:30:26.078 INFO:teuthology.orchestra.run.smithi129.stdout:3131 punch from 0x1040545 to 0x1043958, (0x3413 bytes) 2024-09-17T21:30:26.078 INFO:teuthology.orchestra.run.smithi129.stdout:3132 write 0xe175cea thru 0xe177726 (0x1a3d bytes) 2024-09-17T21:30:26.079 INFO:teuthology.orchestra.run.smithi129.stdout:3133 write 0xc5904c9 thru 0xc59218f (0x1cc7 bytes) 2024-09-17T21:30:26.082 INFO:teuthology.orchestra.run.smithi129.stdout:3134 read 0x6c5824e thru 0x6c66f6c (0xed1f bytes) 2024-09-17T21:30:26.082 INFO:teuthology.orchestra.run.smithi129.stdout:3135 punch from 0x8b8a378 to 0x8b8d7b0, (0x3438 bytes) 2024-09-17T21:30:26.082 INFO:teuthology.orchestra.run.smithi129.stdout:3137 write 0x547862 thru 0x54f767 (0x7f06 bytes) 2024-09-17T21:30:26.084 INFO:teuthology.orchestra.run.smithi129.stdout:3138 punch from 0xb913828 to 0xb91af5b, (0x7733 bytes) 2024-09-17T21:30:26.084 INFO:teuthology.orchestra.run.smithi129.stdout:3139 writesame 0xc9e4724 thru 0xc9e481d data_size 0x5(0xfa bytes) 2024-09-17T21:30:26.085 INFO:teuthology.orchestra.run.smithi129.stdout:3140 writesame 0x627655e thru 0x6279877 data_size 0x1f(0x331a bytes) 2024-09-17T21:30:26.091 INFO:teuthology.orchestra.run.smithi129.stdout:3142 trunc from 0xec76427 to 0xc103ace 2024-09-17T21:30:26.101 INFO:teuthology.orchestra.run.smithi129.stdout:3143 trunc from 0xc103ace to 0xcb0d1b9 2024-09-17T21:30:26.102 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:26.103 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:26.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:26.103 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:26.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.103 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:37817 3.17 3:eab85a8d:::rbd_header.1120e62eea89:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:30:26.103 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:26.103 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 24.403972659000001, 2024-09-17T21:30:26.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86635970399999995, 2024-09-17T21:30:26.103 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.103 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.103 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.103 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.103 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.103 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.104 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37817 2024-09-17T21:30:26.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.104 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:26.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:26.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695457+0000", 2024-09-17T21:30:26.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5799999999999998e-07 2024-09-17T21:30:26.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695461+0000", 2024-09-17T21:30:26.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1110000000000001e-06 2024-09-17T21:30:26.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695471+0000", 2024-09-17T21:30:26.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0441e-05 2024-09-17T21:30:26.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695477+0000", 2024-09-17T21:30:26.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7599999999999999e-06 2024-09-17T21:30:26.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695533+0000", 2024-09-17T21:30:26.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6570999999999998e-05 2024-09-17T21:30:26.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:30:26.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695543+0000", 2024-09-17T21:30:26.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9979999999999998e-06 2024-09-17T21:30:26.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561472+0000", 2024-09-17T21:30:26.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86592841899999995 2024-09-17T21:30:26.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561610+0000", 2024-09-17T21:30:26.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013854699999999999 2024-09-17T21:30:26.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561816+0000", 2024-09-17T21:30:26.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020509900000000001 2024-09-17T21:30:26.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.108 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.108 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.108 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50202 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.108 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:26.108 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.991801869, 2024-09-17T21:30:26.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.076620261999999995, 2024-09-17T21:30:26.108 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.108 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.108 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:26.108 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.108 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.108 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.109 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50202 2024-09-17T21:30:26.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.109 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:26.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:26.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107629+0000", 2024-09-17T21:30:26.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3709999999999998e-06 2024-09-17T21:30:26.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119765+0000", 2024-09-17T21:30:26.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012135683 2024-09-17T21:30:26.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119787+0000", 2024-09-17T21:30:26.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2748999999999999e-05 2024-09-17T21:30:26.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119797+0000", 2024-09-17T21:30:26.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8169999999999994e-06 2024-09-17T21:30:26.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119872+0000", 2024-09-17T21:30:26.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270999999999994e-05 2024-09-17T21:30:26.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119987+0000", 2024-09-17T21:30:26.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011557699999999999 2024-09-17T21:30:26.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:26.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.121766+0000", 2024-09-17T21:30:26.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00177853 2024-09-17T21:30:26.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:26.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178307+0000", 2024-09-17T21:30:26.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056541559999999998 2024-09-17T21:30:26.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:26.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184097+0000", 2024-09-17T21:30:26.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057899029999999999 2024-09-17T21:30:26.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:26.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184172+0000", 2024-09-17T21:30:26.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5226000000000001e-05 2024-09-17T21:30:26.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184247+0000", 2024-09-17T21:30:26.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4574999999999997e-05 2024-09-17T21:30:26.113 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.113 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.113 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.114 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50203 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [truncate 20971520] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.114 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:26.114 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.979601964, 2024-09-17T21:30:26.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065009779000000004, 2024-09-17T21:30:26.114 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.114 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.114 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:26.114 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.114 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.114 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.114 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50203 2024-09-17T21:30:26.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.114 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:26.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:26.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119828+0000", 2024-09-17T21:30:26.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6390000000000001e-06 2024-09-17T21:30:26.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119832+0000", 2024-09-17T21:30:26.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7699999999999999e-06 2024-09-17T21:30:26.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119845+0000", 2024-09-17T21:30:26.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3093e-05 2024-09-17T21:30:26.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119852+0000", 2024-09-17T21:30:26.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6950000000000003e-06 2024-09-17T21:30:26.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.160945+0000", 2024-09-17T21:30:26.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041093076999999999 2024-09-17T21:30:26.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161071+0000", 2024-09-17T21:30:26.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012569799999999999 2024-09-17T21:30:26.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:26.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161283+0000", 2024-09-17T21:30:26.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021201999999999999 2024-09-17T21:30:26.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:26.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.179360+0000", 2024-09-17T21:30:26.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018077784999999999 2024-09-17T21:30:26.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:26.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184726+0000", 2024-09-17T21:30:26.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053652960000000003 2024-09-17T21:30:26.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:26.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184787+0000", 2024-09-17T21:30:26.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1402000000000006e-05 2024-09-17T21:30:26.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184836+0000", 2024-09-17T21:30:26.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9304000000000001e-05 2024-09-17T21:30:26.119 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.119 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.119 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.119 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50206 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.119 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:26.119 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.942520424, 2024-09-17T21:30:26.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.090361968000000001, 2024-09-17T21:30:26.119 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.119 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.119 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:26.120 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.120 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.120 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.120 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50206 2024-09-17T21:30:26.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.120 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:26.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:26.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156911+0000", 2024-09-17T21:30:26.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-17T21:30:26.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173670+0000", 2024-09-17T21:30:26.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016758907 2024-09-17T21:30:26.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173696+0000", 2024-09-17T21:30:26.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6329000000000002e-05 2024-09-17T21:30:26.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173708+0000", 2024-09-17T21:30:26.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2512e-05 2024-09-17T21:30:26.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173788+0000", 2024-09-17T21:30:26.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9332999999999994e-05 2024-09-17T21:30:26.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173918+0000", 2024-09-17T21:30:26.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013063300000000001 2024-09-17T21:30:26.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:26.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.176228+0000", 2024-09-17T21:30:26.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002309757 2024-09-17T21:30:26.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:26.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.245164+0000", 2024-09-17T21:30:26.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068936301000000005 2024-09-17T21:30:26.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:26.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247176+0000", 2024-09-17T21:30:26.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020117590000000001 2024-09-17T21:30:26.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:26.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247215+0000", 2024-09-17T21:30:26.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9094000000000002e-05 2024-09-17T21:30:26.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247270+0000", 2024-09-17T21:30:26.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4793999999999997e-05 2024-09-17T21:30:26.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.125 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.125 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50207 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [truncate 21556327] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.125 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:26.125 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.925671444000001, 2024-09-17T21:30:26.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.074055363999999999, 2024-09-17T21:30:26.125 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.125 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.125 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:26.125 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.125 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.126 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.126 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50207 2024-09-17T21:30:26.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.126 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:26.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:26.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173759+0000", 2024-09-17T21:30:26.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8160000000000001e-06 2024-09-17T21:30:26.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173763+0000", 2024-09-17T21:30:26.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5360000000000003e-06 2024-09-17T21:30:26.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173771+0000", 2024-09-17T21:30:26.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8849999999999999e-06 2024-09-17T21:30:26.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173777+0000", 2024-09-17T21:30:26.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7799999999999997e-06 2024-09-17T21:30:26.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221363+0000", 2024-09-17T21:30:26.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047585493 2024-09-17T21:30:26.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221430+0000", 2024-09-17T21:30:26.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7129000000000001e-05 2024-09-17T21:30:26.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:26.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221550+0000", 2024-09-17T21:30:26.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120658 2024-09-17T21:30:26.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:26.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246113+0000", 2024-09-17T21:30:26.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02456262 2024-09-17T21:30:26.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:26.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247744+0000", 2024-09-17T21:30:26.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001630692 2024-09-17T21:30:26.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:26.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247777+0000", 2024-09-17T21:30:26.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3015000000000002e-05 2024-09-17T21:30:26.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247812+0000", 2024-09-17T21:30:26.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.574e-05 2024-09-17T21:30:26.130 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.130 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.130 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.130 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50828 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.130 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:26.130 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.617559512, 2024-09-17T21:30:26.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047324185999999997, 2024-09-17T21:30:26.131 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.131 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.131 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.131 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.131 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.131 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.131 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50828 2024-09-17T21:30:26.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.131 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:26.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:26.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:26.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.46e-07 2024-09-17T21:30:26.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481871+0000", 2024-09-17T21:30:26.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T21:30:26.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481878+0000", 2024-09-17T21:30:26.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:30:26.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481882+0000", 2024-09-17T21:30:26.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:30:26.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529038+0000", 2024-09-17T21:30:26.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047156034999999999 2024-09-17T21:30:26.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529099+0000", 2024-09-17T21:30:26.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0316000000000003e-05 2024-09-17T21:30:26.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529193+0000", 2024-09-17T21:30:26.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4369000000000004e-05 2024-09-17T21:30:26.134 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.134 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.134 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.135 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50829 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 12845056~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.135 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:26.135 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.617540718000001, 2024-09-17T21:30:26.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047887596999999997, 2024-09-17T21:30:26.135 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.135 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.135 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.135 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.135 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.135 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.135 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50829 2024-09-17T21:30:26.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.135 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:26.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:26.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:26.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:30:26.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481890+0000", 2024-09-17T21:30:26.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.877e-06 2024-09-17T21:30:26.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481895+0000", 2024-09-17T21:30:26.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:30:26.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481900+0000", 2024-09-17T21:30:26.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4440000000000001e-06 2024-09-17T21:30:26.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529225+0000", 2024-09-17T21:30:26.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047325117999999999 2024-09-17T21:30:26.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529292+0000", 2024-09-17T21:30:26.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7100000000000005e-05 2024-09-17T21:30:26.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529775+0000", 2024-09-17T21:30:26.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048322799999999999 2024-09-17T21:30:26.138 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.138 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.138 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.139 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50830 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.139 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:26.139 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.617522316000001, 2024-09-17T21:30:26.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048062123999999998, 2024-09-17T21:30:26.139 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.139 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.139 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.139 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.139 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.139 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.139 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50830 2024-09-17T21:30:26.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.139 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:26.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:26.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481907+0000", 2024-09-17T21:30:26.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:30:26.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481909+0000", 2024-09-17T21:30:26.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T21:30:26.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481915+0000", 2024-09-17T21:30:26.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.02e-06 2024-09-17T21:30:26.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481920+0000", 2024-09-17T21:30:26.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6799999999999998e-06 2024-09-17T21:30:26.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529793+0000", 2024-09-17T21:30:26.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047873042999999997 2024-09-17T21:30:26.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529870+0000", 2024-09-17T21:30:26.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6967999999999999e-05 2024-09-17T21:30:26.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529968+0000", 2024-09-17T21:30:26.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8085999999999996e-05 2024-09-17T21:30:26.143 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.143 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.143 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.143 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50831 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 18087936~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.143 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:26.143 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.617501969999999, 2024-09-17T21:30:26.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048663304999999997, 2024-09-17T21:30:26.143 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.143 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.143 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.144 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50831 2024-09-17T21:30:26.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.144 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:26.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:26.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:26.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:30:26.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481929+0000", 2024-09-17T21:30:26.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2189999999999998e-06 2024-09-17T21:30:26.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481935+0000", 2024-09-17T21:30:26.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.383e-06 2024-09-17T21:30:26.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481939+0000", 2024-09-17T21:30:26.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2479999999999998e-06 2024-09-17T21:30:26.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530095+0000", 2024-09-17T21:30:26.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048156617999999998 2024-09-17T21:30:26.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530159+0000", 2024-09-17T21:30:26.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3394999999999999e-05 2024-09-17T21:30:26.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530590+0000", 2024-09-17T21:30:26.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043101199999999999 2024-09-17T21:30:26.147 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.147 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.147 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.147 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50832 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 19660800~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.147 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:26.148 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.617483323, 2024-09-17T21:30:26.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049339466999999998, 2024-09-17T21:30:26.148 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.148 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.148 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.148 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.148 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.148 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.148 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50832 2024-09-17T21:30:26.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.148 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:26.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:26.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481946+0000", 2024-09-17T21:30:26.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:26.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481951+0000", 2024-09-17T21:30:26.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1479999999999997e-06 2024-09-17T21:30:26.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481956+0000", 2024-09-17T21:30:26.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7339999999999996e-06 2024-09-17T21:30:26.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481961+0000", 2024-09-17T21:30:26.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4889999999999997e-06 2024-09-17T21:30:26.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530630+0000", 2024-09-17T21:30:26.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048668596000000001 2024-09-17T21:30:26.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530720+0000", 2024-09-17T21:30:26.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0163999999999999e-05 2024-09-17T21:30:26.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531285+0000", 2024-09-17T21:30:26.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056493199999999998 2024-09-17T21:30:26.151 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.151 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.151 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.152 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50833 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.152 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:26.152 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.617461178999999, 2024-09-17T21:30:26.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049908069999999999, 2024-09-17T21:30:26.152 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.152 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.152 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.152 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.152 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.152 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.152 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50833 2024-09-17T21:30:26.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.153 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:26.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:26.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481968+0000", 2024-09-17T21:30:26.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5600000000000001e-07 2024-09-17T21:30:26.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481970+0000", 2024-09-17T21:30:26.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:30:26.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481975+0000", 2024-09-17T21:30:26.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8499999999999999e-06 2024-09-17T21:30:26.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481980+0000", 2024-09-17T21:30:26.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-17T21:30:26.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531337+0000", 2024-09-17T21:30:26.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049356874000000002 2024-09-17T21:30:26.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531392+0000", 2024-09-17T21:30:26.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4826e-05 2024-09-17T21:30:26.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531875+0000", 2024-09-17T21:30:26.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048345099999999998 2024-09-17T21:30:26.156 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.156 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.156 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.156 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50834 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 24903680~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.156 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:26.156 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.617436411, 2024-09-17T21:30:26.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050534015000000002, 2024-09-17T21:30:26.156 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.156 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.156 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.157 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.157 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.157 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.157 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50834 2024-09-17T21:30:26.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.157 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:26.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:26.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481993+0000", 2024-09-17T21:30:26.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:30:26.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481995+0000", 2024-09-17T21:30:26.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.199e-06 2024-09-17T21:30:26.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482001+0000", 2024-09-17T21:30:26.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.772e-06 2024-09-17T21:30:26.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482006+0000", 2024-09-17T21:30:26.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0660000000000003e-06 2024-09-17T21:30:26.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531923+0000", 2024-09-17T21:30:26.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049917200000000002 2024-09-17T21:30:26.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532004+0000", 2024-09-17T21:30:26.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0758000000000002e-05 2024-09-17T21:30:26.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532526+0000", 2024-09-17T21:30:26.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00052258899999999995 2024-09-17T21:30:26.160 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.160 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.160 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.160 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50835 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 28573696~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.161 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:26.161 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.617416428, 2024-09-17T21:30:26.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051204701999999998, 2024-09-17T21:30:26.161 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.161 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.161 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.161 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.161 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.161 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.161 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50835 2024-09-17T21:30:26.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.161 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:26.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:26.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482013+0000", 2024-09-17T21:30:26.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:30:26.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482015+0000", 2024-09-17T21:30:26.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.272e-06 2024-09-17T21:30:26.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482020+0000", 2024-09-17T21:30:26.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3040000000000004e-06 2024-09-17T21:30:26.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482025+0000", 2024-09-17T21:30:26.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5650000000000003e-06 2024-09-17T21:30:26.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532569+0000", 2024-09-17T21:30:26.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050544661999999997 2024-09-17T21:30:26.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532649+0000", 2024-09-17T21:30:26.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9640000000000003e-05 2024-09-17T21:30:26.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533217+0000", 2024-09-17T21:30:26.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056784100000000003 2024-09-17T21:30:26.164 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.164 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.165 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.165 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50836 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 30146560~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.165 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:26.165 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.617396811000001, 2024-09-17T21:30:26.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051875589, 2024-09-17T21:30:26.165 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.165 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.165 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.165 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.165 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.165 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.166 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50836 2024-09-17T21:30:26.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.166 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:26.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:26.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:26.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3700000000000001e-07 2024-09-17T21:30:26.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482034+0000", 2024-09-17T21:30:26.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0059999999999999e-06 2024-09-17T21:30:26.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482040+0000", 2024-09-17T21:30:26.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:30:26.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482044+0000", 2024-09-17T21:30:26.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:30:26.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533260+0000", 2024-09-17T21:30:26.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051215416 2024-09-17T21:30:26.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533348+0000", 2024-09-17T21:30:26.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8695000000000004e-05 2024-09-17T21:30:26.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533907+0000", 2024-09-17T21:30:26.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055902199999999995 2024-09-17T21:30:26.169 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.169 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.169 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.169 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50844 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.169 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:26.170 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.617233385, 2024-09-17T21:30:26.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0475644, 2024-09-17T21:30:26.170 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.170 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.170 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.170 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.170 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.170 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.170 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50844 2024-09-17T21:30:26.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.170 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:26.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:26.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482196+0000", 2024-09-17T21:30:26.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.39e-07 2024-09-17T21:30:26.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482198+0000", 2024-09-17T21:30:26.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0099999999999998e-06 2024-09-17T21:30:26.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482204+0000", 2024-09-17T21:30:26.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.93e-06 2024-09-17T21:30:26.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482208+0000", 2024-09-17T21:30:26.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7439999999999998e-06 2024-09-17T21:30:26.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529318+0000", 2024-09-17T21:30:26.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047110051999999999 2024-09-17T21:30:26.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529396+0000", 2024-09-17T21:30:26.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7712000000000004e-05 2024-09-17T21:30:26.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529760+0000", 2024-09-17T21:30:26.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00036351299999999999 2024-09-17T21:30:26.173 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.173 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.173 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.174 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50845 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 21757952~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.174 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:26.174 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.617214281999999, 2024-09-17T21:30:26.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048084625999999998, 2024-09-17T21:30:26.174 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.174 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.174 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.174 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.174 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.174 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.174 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50845 2024-09-17T21:30:26.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.175 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:26.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:26.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482215+0000", 2024-09-17T21:30:26.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:30:26.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482216+0000", 2024-09-17T21:30:26.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.823e-06 2024-09-17T21:30:26.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482222+0000", 2024-09-17T21:30:26.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.507e-06 2024-09-17T21:30:26.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482227+0000", 2024-09-17T21:30:26.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6609999999999999e-06 2024-09-17T21:30:26.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529792+0000", 2024-09-17T21:30:26.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047565075999999998 2024-09-17T21:30:26.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529871+0000", 2024-09-17T21:30:26.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8855000000000003e-05 2024-09-17T21:30:26.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530299+0000", 2024-09-17T21:30:26.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000428307 2024-09-17T21:30:26.178 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.178 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.178 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.178 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50846 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 25427968~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.178 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:26.178 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.617191894999999, 2024-09-17T21:30:26.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048735142000000002, 2024-09-17T21:30:26.178 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.178 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.178 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.179 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.179 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.179 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.179 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50846 2024-09-17T21:30:26.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.179 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:26.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:26.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:26.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:30:26.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482239+0000", 2024-09-17T21:30:26.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.871e-06 2024-09-17T21:30:26.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482244+0000", 2024-09-17T21:30:26.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.49e-06 2024-09-17T21:30:26.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482249+0000", 2024-09-17T21:30:26.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6269999999999999e-06 2024-09-17T21:30:26.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530347+0000", 2024-09-17T21:30:26.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048097721000000003 2024-09-17T21:30:26.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530437+0000", 2024-09-17T21:30:26.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0604000000000002e-05 2024-09-17T21:30:26.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530972+0000", 2024-09-17T21:30:26.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053445000000000001 2024-09-17T21:30:26.182 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.182 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.182 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.182 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50847 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 27000832~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.182 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:26.183 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.617173999, 2024-09-17T21:30:26.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049308668, 2024-09-17T21:30:26.183 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.183 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.183 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.183 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.183 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.183 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.183 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50847 2024-09-17T21:30:26.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.183 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:26.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:26.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:26.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:30:26.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482257+0000", 2024-09-17T21:30:26.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.846e-06 2024-09-17T21:30:26.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482264+0000", 2024-09-17T21:30:26.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8170000000000001e-06 2024-09-17T21:30:26.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482283+0000", 2024-09-17T21:30:26.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9718e-05 2024-09-17T21:30:26.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531018+0000", 2024-09-17T21:30:26.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048734497000000002 2024-09-17T21:30:26.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531080+0000", 2024-09-17T21:30:26.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1795e-05 2024-09-17T21:30:26.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531563+0000", 2024-09-17T21:30:26.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048366499999999998 2024-09-17T21:30:26.186 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.187 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.187 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.187 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50848 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 30670848~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.187 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:26.187 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.617137457, 2024-09-17T21:30:26.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049893337000000003, 2024-09-17T21:30:26.187 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.187 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.187 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.187 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50848 2024-09-17T21:30:26.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.188 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:26.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:26.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482292+0000", 2024-09-17T21:30:26.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T21:30:26.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482294+0000", 2024-09-17T21:30:26.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3209999999999999e-06 2024-09-17T21:30:26.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482300+0000", 2024-09-17T21:30:26.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7189999999999998e-06 2024-09-17T21:30:26.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482305+0000", 2024-09-17T21:30:26.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T21:30:26.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531606+0000", 2024-09-17T21:30:26.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049301273999999999 2024-09-17T21:30:26.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531696+0000", 2024-09-17T21:30:26.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0161999999999995e-05 2024-09-17T21:30:26.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532184+0000", 2024-09-17T21:30:26.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048806400000000002 2024-09-17T21:30:26.191 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.191 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.191 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.191 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50849 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 32243712~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:26.191 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:26.191 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.617118015999999, 2024-09-17T21:30:26.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050444607000000002, 2024-09-17T21:30:26.191 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.191 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.192 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.192 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.192 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.192 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.192 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50849 2024-09-17T21:30:26.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.192 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:26.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:26.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482311+0000", 2024-09-17T21:30:26.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:30:26.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482316+0000", 2024-09-17T21:30:26.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1209999999999998e-06 2024-09-17T21:30:26.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482321+0000", 2024-09-17T21:30:26.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4779999999999999e-06 2024-09-17T21:30:26.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482326+0000", 2024-09-17T21:30:26.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7099999999999998e-06 2024-09-17T21:30:26.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532221+0000", 2024-09-17T21:30:26.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049894936000000001 2024-09-17T21:30:26.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532287+0000", 2024-09-17T21:30:26.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5914000000000006e-05 2024-09-17T21:30:26.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532755+0000", 2024-09-17T21:30:26.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000468107 2024-09-17T21:30:26.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.195 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.195 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.195 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:26.197 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:30:26.199 INFO:teuthology.orchestra.run.smithi129.stdout:3145 write 0xdb231fe thru 0xdb2cf3b (0x9d3e bytes) 2024-09-17T21:30:26.199 INFO:teuthology.orchestra.run.smithi129.stdout:3146 write 0xbf19b83 thru 0xbf2340f (0x988d bytes) 2024-09-17T21:30:26.199 INFO:teuthology.orchestra.run.smithi129.stdout:3147 write 0xc5d6426 thru 0xc5dc3ef (0x5fca bytes) 2024-09-17T21:30:26.199 INFO:teuthology.orchestra.run.smithi129.stdout:3148 write 0x79861a8 thru 0x798db4c (0x79a5 bytes) 2024-09-17T21:30:26.199 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.199 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.199 INFO:teuthology.orchestra.run.smithi129.stdout:3151 write 0xa7cebcb thru 0xa7d8f2a (0xa360 bytes) 2024-09-17T21:30:26.199 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.199 INFO:teuthology.orchestra.run.smithi129.stdout:3154 write 0xb219605 thru 0xb228db3 (0xf7af bytes) 2024-09-17T21:30:26.199 INFO:teuthology.orchestra.run.smithi129.stdout:3155 punch from 0xc4d197c to 0xc4d62b3, (0x4937 bytes) 2024-09-17T21:30:26.199 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.200 INFO:teuthology.orchestra.run.smithi129.stdout:3158 write 0x3fe240d thru 0x3fe8586 (0x617a bytes) 2024-09-17T21:30:26.200 INFO:teuthology.orchestra.run.smithi129.stdout:3160 punch from 0xa3863cd to 0xa38ccbf, (0x68f2 bytes) 2024-09-17T21:30:26.200 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.200 INFO:teuthology.orchestra.run.smithi129.stdout:3163 trunc from 0xdb2cf3c to 0x43e847 2024-09-17T21:30:26.200 INFO:teuthology.orchestra.run.smithi129.stdout:3164 writesame 0x413594c thru 0x41376a4 data_size 0x1(0x1d59 bytes) 2024-09-17T21:30:26.200 INFO:teuthology.orchestra.run.smithi129.stdout:3165 write 0x18510bb thru 0x185381c (0x2762 bytes) 2024-09-17T21:30:26.200 INFO:teuthology.orchestra.run.smithi129.stdout:3167 trunc from 0x41376a5 to 0xe80069b 2024-09-17T21:30:26.200 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:26.195+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:26.202 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.202 INFO:teuthology.orchestra.run.smithi129.stdout:3169 read 0xdf404aa thru 0xdf4799c (0x74f3 bytes) 2024-09-17T21:30:26.202 INFO:teuthology.orchestra.run.smithi129.stdout:3170 punch from 0x566a412 to 0x5676c14, (0xc802 bytes) 2024-09-17T21:30:26.202 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.202 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.202 INFO:teuthology.orchestra.run.smithi129.stdout:3175 write 0x6581506 thru 0x65881d2 (0x6ccd bytes) 2024-09-17T21:30:26.205 INFO:teuthology.orchestra.run.smithi129.stdout:3176 read 0x39450c0 thru 0x394cc62 (0x7ba3 bytes) 2024-09-17T21:30:26.205 INFO:teuthology.orchestra.run.smithi129.stdout:3177 read 0x191a544 thru 0x1924f0b (0xa9c8 bytes) 2024-09-17T21:30:26.206 INFO:teuthology.orchestra.run.smithi129.stdout:3178 trunc from 0xe80069b to 0xeddd439 2024-09-17T21:30:26.213 INFO:teuthology.orchestra.run.smithi129.stdout:3180 punch from 0x70471a4 to 0x70554cd, (0xe329 bytes) 2024-09-17T21:30:26.213 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.213 INFO:teuthology.orchestra.run.smithi129.stdout:3182 write 0xe50cd49 thru 0xe513985 (0x6c3d bytes) 2024-09-17T21:30:26.232 INFO:teuthology.orchestra.run.smithi129.stdout:3183 trunc from 0xeddd439 to 0x39eed0e 2024-09-17T21:30:26.240 INFO:teuthology.orchestra.run.smithi129.stdout:3186 punch from 0x3665abc to 0x366cb59, (0x709d bytes) 2024-09-17T21:30:26.244 INFO:teuthology.orchestra.run.smithi129.stdout:3187 trunc from 0x39eed0e to 0x6ad3485 2024-09-17T21:30:26.258 INFO:teuthology.orchestra.run.smithi129.stdout:3188 writesame 0xa7cd503 thru 0xa7dca58 data_size 0x1f(0xf556 bytes) 2024-09-17T21:30:26.259 INFO:teuthology.orchestra.run.smithi129.stdout:3189 read 0x487ae88 thru 0x4889fb7 (0xf130 bytes) 2024-09-17T21:30:26.259 INFO:teuthology.orchestra.run.smithi129.stdout:3192 punch from 0xa466510 to 0xa46d821, (0x7311 bytes) 2024-09-17T21:30:26.259 INFO:teuthology.orchestra.run.smithi129.stdout:3193 write 0x3ef03d thru 0x3f4814 (0x57d8 bytes) 2024-09-17T21:30:26.260 INFO:teuthology.orchestra.run.smithi129.stdout:3194 read 0x17e0bb7 thru 0x17ea398 (0x97e2 bytes) 2024-09-17T21:30:26.261 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.261 INFO:teuthology.orchestra.run.smithi129.stdout:3197 write 0x14d598b thru 0x14dcf73 (0x75e9 bytes) 2024-09-17T21:30:26.262 INFO:teuthology.orchestra.run.smithi129.stdout:3198 writesame 0x623103e thru 0x623e975 data_size 0x6(0xd938 bytes) 2024-09-17T21:30:26.268 INFO:teuthology.orchestra.run.smithi129.stdout:3200 read 0x47448b8 thru 0x4744f9d (0x6e6 bytes) 2024-09-17T21:30:26.268 INFO:teuthology.orchestra.run.smithi129.stdout:3201 writesame 0x126df97 thru 0x126f2df data_size 0x1(0x1349 bytes) 2024-09-17T21:30:26.276 INFO:teuthology.orchestra.run.smithi129.stdout:3202 trunc from 0xa7dca59 to 0x6d9809b 2024-09-17T21:30:26.289 INFO:teuthology.orchestra.run.smithi129.stdout:3205 write 0x7da6ac4 thru 0x7da9d64 (0x32a1 bytes) 2024-09-17T21:30:26.292 INFO:teuthology.orchestra.run.smithi129.stdout:3209 write 0x13e454d thru 0x13ec1ec (0x7ca0 bytes) 2024-09-17T21:30:26.294 INFO:teuthology.orchestra.run.smithi129.stdout:3211 write 0x356ddc3 thru 0x35775bc (0x97fa bytes) 2024-09-17T21:30:26.298 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.298 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:26.296+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:26.304 INFO:teuthology.orchestra.run.smithi129.stdout:3213 write 0x8b7c02d thru 0x8b80104 (0x40d8 bytes) 2024-09-17T21:30:26.317 INFO:teuthology.orchestra.run.smithi129.stdout:3216 trunc from 0x8b80105 to 0x8cce15 2024-09-17T21:30:26.321 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:26.321 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:26.321 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:26.321 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:26.330 INFO:teuthology.orchestra.run.smithi129.stdout:3217 punch from 0x7a2a2c to 0x7ab59f, (0x8b73 bytes) 2024-09-17T21:30:26.332 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:30:26.347 INFO:teuthology.orchestra.run.smithi129.stdout:3221 writesame 0x9dc6e42 thru 0x9dd0043 data_size 0xb(0x9202 bytes) 2024-09-17T21:30:26.347 INFO:teuthology.orchestra.run.smithi129.stdout:3222 write 0x92991eb thru 0x929da36 (0x484c bytes) 2024-09-17T21:30:26.353 INFO:teuthology.orchestra.run.smithi129.stdout:3223 read 0x4246c77 thru 0x4248b36 (0x1ec0 bytes) 2024-09-17T21:30:26.353 INFO:teuthology.orchestra.run.smithi129.stdout:3225 writesame 0x8bc5f8a thru 0x8bccdc3 data_size 0x3(0x6e3a bytes) 2024-09-17T21:30:26.371 INFO:teuthology.orchestra.run.smithi129.stdout:3226 write 0xd056a0b thru 0xd063395 (0xc98b bytes) 2024-09-17T21:30:26.377 INFO:teuthology.orchestra.run.smithi129.stdout:3227 writesame 0xaead652 thru 0xaebc352 data_size 0x2b(0xed01 bytes) 2024-09-17T21:30:26.383 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.383 INFO:teuthology.orchestra.run.smithi129.stdout:3229 write 0x50533f4 thru 0x505c0ec (0x8cf9 bytes) 2024-09-17T21:30:26.389 INFO:teuthology.orchestra.run.smithi129.stdout:3232 read 0x3b972b7 thru 0x3b9a902 (0x364c bytes) 2024-09-17T21:30:26.389 INFO:teuthology.orchestra.run.smithi129.stdout:3233 read 0x41ce96 thru 0x4250fe (0x8269 bytes) 2024-09-17T21:30:26.389 INFO:teuthology.orchestra.run.smithi129.stdout:3234 read 0xa7ed626 thru 0xa7f2fbc (0x5997 bytes) 2024-09-17T21:30:26.389 INFO:teuthology.orchestra.run.smithi129.stdout:3235 read 0x5c4af54 thru 0x5c4f2b7 (0x4364 bytes) 2024-09-17T21:30:26.399 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:26.396+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:26.399 INFO:teuthology.orchestra.run.smithi129.stdout:3236 trunc from 0xd063396 to 0x52672d4 2024-09-17T21:30:26.424 INFO:teuthology.orchestra.run.smithi129.stdout:3238 write 0xcb5d12e thru 0xcb66657 (0x952a bytes) 2024-09-17T21:30:26.428 INFO:teuthology.orchestra.run.smithi129.stdout:3240 write 0x9ab5f86 thru 0x9abcf99 (0x7014 bytes) 2024-09-17T21:30:26.432 INFO:teuthology.orchestra.run.smithi129.stdout:3241 read 0x4c31aa thru 0x4c7b2a (0x4981 bytes) 2024-09-17T21:30:26.432 INFO:teuthology.orchestra.run.smithi129.stdout:3242 writesame 0x2433164 thru 0x24360a8 data_size 0x1(0x2f45 bytes) 2024-09-17T21:30:26.437 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.437 INFO:teuthology.orchestra.run.smithi129.stdout:3245 punch from 0x5671554 to 0x5677ffb, (0x6aa7 bytes) 2024-09-17T21:30:26.437 INFO:teuthology.orchestra.run.smithi129.stdout:3246 write 0x3b8e10 thru 0x3bc148 (0x3339 bytes) 2024-09-17T21:30:26.438 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.438 INFO:teuthology.orchestra.run.smithi129.stdout:3248 read 0x1784d47 thru 0x1784fe1 (0x29b bytes) 2024-09-17T21:30:26.439 INFO:teuthology.orchestra.run.smithi129.stdout:3249 read 0x267da47 thru 0x268058f (0x2b49 bytes) 2024-09-17T21:30:26.439 INFO:teuthology.orchestra.run.smithi129.stdout:3250 punch from 0x1cf055b to 0x1cf4c91, (0x4736 bytes) 2024-09-17T21:30:26.441 INFO:teuthology.orchestra.run.smithi129.stdout:3252 trunc from 0xcb66658 to 0xaf1db94 2024-09-17T21:30:26.450 INFO:teuthology.orchestra.run.smithi129.stdout:3253 punch from 0x8884c62 to 0x88891de, (0x457c bytes) 2024-09-17T21:30:26.451 INFO:teuthology.orchestra.run.smithi129.stdout:3254 punch from 0x134c268 to 0x1355be1, (0x9979 bytes) 2024-09-17T21:30:26.451 INFO:teuthology.orchestra.run.smithi129.stdout:3255 punch from 0x43a80a0 to 0x43b026a, (0x81ca bytes) 2024-09-17T21:30:26.451 INFO:teuthology.orchestra.run.smithi129.stdout:3256 read 0x42bc5d3 thru 0x42cb7a6 (0xf1d4 bytes) 2024-09-17T21:30:26.451 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.451 INFO:teuthology.orchestra.run.smithi129.stdout:3258 punch from 0x2dd2a49 to 0x2ddb532, (0x8ae9 bytes) 2024-09-17T21:30:26.451 INFO:teuthology.orchestra.run.smithi129.stdout:3259 read 0x4e64054 thru 0x4e67775 (0x3722 bytes) 2024-09-17T21:30:26.451 INFO:teuthology.orchestra.run.smithi129.stdout:3261 writesame 0x3fcdc88 thru 0x3fd797d data_size 0x6f(0x9cf6 bytes) 2024-09-17T21:30:26.456 INFO:teuthology.orchestra.run.smithi129.stdout:3262 trunc from 0xaf1db94 to 0x978cd76 2024-09-17T21:30:26.461 INFO:teuthology.orchestra.run.smithi129.stdout:3263 write 0x7977eda thru 0x797cbd9 (0x4d00 bytes) 2024-09-17T21:30:26.464 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:26.464 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:26.464 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:26.465 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:26.465 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:26.465 INFO:teuthology.orchestra.run.smithi129.stdout:3264 write 0x7985e11 thru 0x798c166 (0x6356 bytes) 2024-09-17T21:30:26.469 INFO:teuthology.orchestra.run.smithi129.stdout:3265 write 0x841c694 thru 0x84230d7 (0x6a44 bytes) 2024-09-17T21:30:26.471 INFO:teuthology.orchestra.run.smithi129.stdout:3267 read 0xd1a4f5 thru 0xd24ea2 (0xa9ae bytes) 2024-09-17T21:30:26.472 INFO:teuthology.orchestra.run.smithi129.stdout:3268 read 0x7c34ed4 thru 0x7c3c3a0 (0x74cd bytes) 2024-09-17T21:30:26.472 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.472 INFO:teuthology.orchestra.run.smithi129.stdout:3271 punch from 0x3e441ba to 0x3e4f809, (0xb64f bytes) 2024-09-17T21:30:26.473 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.473 INFO:teuthology.orchestra.run.smithi129.stdout:3273 write 0x1ad655 thru 0x1b95d6 (0xbf82 bytes) 2024-09-17T21:30:26.474 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:30:26.487 INFO:teuthology.orchestra.run.smithi129.stdout:3276 writesame 0xee59e50 thru 0xee5a04e data_size 0x1(0x1ff bytes) 2024-09-17T21:30:26.487 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.504 INFO:teuthology.orchestra.run.smithi129.stdout:3278 trunc from 0xee5a04f to 0x2c9326 2024-09-17T21:30:26.505 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:26.503+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:26.530 INFO:teuthology.orchestra.run.smithi129.stdout:3279 write 0x5f195e8 thru 0x5f2697a (0xd393 bytes) 2024-09-17T21:30:26.549 INFO:teuthology.orchestra.run.smithi129.stdout:3280 write 0x8d55755 thru 0x8d5cd82 (0x762e bytes) 2024-09-17T21:30:26.558 INFO:teuthology.orchestra.run.smithi129.stdout:3281 read 0x4690112 thru 0x4695dc0 (0x5caf bytes) 2024-09-17T21:30:26.558 INFO:teuthology.orchestra.run.smithi129.stdout:3282 read 0xc7298a thru 0xc78ae4 (0x615b bytes) 2024-09-17T21:30:26.558 INFO:teuthology.orchestra.run.smithi129.stdout:3283 read 0x6f742c9 thru 0x6f82645 (0xe37d bytes) 2024-09-17T21:30:26.559 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.559 INFO:teuthology.orchestra.run.smithi129.stdout:3290 read 0x6f1810 thru 0x6f4730 (0x2f21 bytes) 2024-09-17T21:30:26.559 INFO:teuthology.orchestra.run.smithi129.stdout:3291 writesame 0x57a133a thru 0x57aeda4 data_size 0x35(0xda6b bytes) 2024-09-17T21:30:26.564 INFO:teuthology.orchestra.run.smithi129.stdout:3294 writesame 0x42b1711 thru 0x42b78e8 data_size 0x3e(0x61d8 bytes) 2024-09-17T21:30:26.568 INFO:teuthology.orchestra.run.smithi129.stdout:3295 read 0x5a62787 thru 0x5a64bba (0x2434 bytes) 2024-09-17T21:30:26.568 INFO:teuthology.orchestra.run.smithi129.stdout:3297 read 0x765fed0 thru 0x7666d25 (0x6e56 bytes) 2024-09-17T21:30:26.568 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.568 INFO:teuthology.orchestra.run.smithi129.stdout:3301 write 0x7d30375 thru 0x7d3e80f (0xe49b bytes) 2024-09-17T21:30:26.586 INFO:teuthology.orchestra.run.smithi129.stdout:3302 write 0xe2307eb thru 0xe235679 (0x4e8f bytes) 2024-09-17T21:30:26.590 INFO:teuthology.orchestra.run.smithi129.stdout:3303 writesame 0x20531c7 thru 0x2058398 data_size 0x3(0x51d2 bytes) 2024-09-17T21:30:26.594 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.594 INFO:teuthology.orchestra.run.smithi129.stdout:3305 read 0xe043d90 thru 0xe04d02d (0x929e bytes) 2024-09-17T21:30:26.594 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.594 INFO:teuthology.orchestra.run.smithi129.stdout:3308 writesame 0x8256a25 thru 0x825eb0e data_size 0x1d(0x80ea bytes) 2024-09-17T21:30:26.597 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.597 INFO:teuthology.orchestra.run.smithi129.stdout:3311 punch from 0xd0ce2b5 to 0xd0d25dd, (0x4328 bytes) 2024-09-17T21:30:26.597 INFO:teuthology.orchestra.run.smithi129.stdout:3312 read 0x976fa04 thru 0x9774983 (0x4f80 bytes) 2024-09-17T21:30:26.597 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.597 INFO:teuthology.orchestra.run.smithi129.stdout:3314 read 0xd4f1609 thru 0xd4fd282 (0xbc7a bytes) 2024-09-17T21:30:26.597 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.600 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:26.598+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:26.603 INFO:teuthology.orchestra.run.smithi129.stdout:3322 write 0xecb5880 thru 0xecc1202 (0xb983 bytes) 2024-09-17T21:30:26.610 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:26.610 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:26.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:26.610 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:26.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.611 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "pg_update_log_missing(3.5 epoch 45/37 rep_tid 844 entries 45'73 (0'0) error 3:a65c40ae:::rbd_data.1120b21c3309.0000000000000126:head by client.4384.0:48900 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 45'70)", 2024-09-17T21:30:26.611 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:26.611 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.56040338, 2024-09-17T21:30:26.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019554365000000001, 2024-09-17T21:30:26.611 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.611 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.611 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T21:30:26.611 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:26.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:26.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047112+0000", 2024-09-17T21:30:26.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:30:26.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047121+0000", 2024-09-17T21:30:26.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4379999999999995e-06 2024-09-17T21:30:26.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047139+0000", 2024-09-17T21:30:26.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8235000000000001e-05 2024-09-17T21:30:26.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047303+0000", 2024-09-17T21:30:26.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016389400000000001 2024-09-17T21:30:26.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047396+0000", 2024-09-17T21:30:26.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3355000000000003e-05 2024-09-17T21:30:26.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066666+0000", 2024-09-17T21:30:26.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019269649999999999 2024-09-17T21:30:26.615 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.615 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.615 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.615 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48906 3.4 e45/25 3:2503fe27:::rbd_data.1120b21c3309.0000000000000127:head v 45'148, mlcod=45'146)", 2024-09-17T21:30:26.615 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:26.615 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.558277200999999, 2024-09-17T21:30:26.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018169739000000001, 2024-09-17T21:30:26.615 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.616 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.616 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:26.616 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:26.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:26.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:26.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:30:26.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049244+0000", 2024-09-17T21:30:26.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8390000000000003e-06 2024-09-17T21:30:26.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066411+0000", 2024-09-17T21:30:26.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017166818 2024-09-17T21:30:26.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066426+0000", 2024-09-17T21:30:26.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4888999999999999e-05 2024-09-17T21:30:26.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066611+0000", 2024-09-17T21:30:26.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018464499999999999 2024-09-17T21:30:26.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066630+0000", 2024-09-17T21:30:26.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9201000000000001e-05 2024-09-17T21:30:26.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:26.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067373+0000", 2024-09-17T21:30:26.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00074322600000000004 2024-09-17T21:30:26.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067407+0000", 2024-09-17T21:30:26.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4357999999999999e-05 2024-09-17T21:30:26.620 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.620 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.620 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.620 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50206 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'301, mlcod=45'299)", 2024-09-17T21:30:26.620 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:26.621 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.428800357, 2024-09-17T21:30:26.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068285810000000002, 2024-09-17T21:30:26.621 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.621 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.621 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:26.621 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:26.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:26.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178716+0000", 2024-09-17T21:30:26.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.438e-06 2024-09-17T21:30:26.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192044+0000", 2024-09-17T21:30:26.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013328370000000001 2024-09-17T21:30:26.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192069+0000", 2024-09-17T21:30:26.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4496000000000001e-05 2024-09-17T21:30:26.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192085+0000", 2024-09-17T21:30:26.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5758999999999999e-05 2024-09-17T21:30:26.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192167+0000", 2024-09-17T21:30:26.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2842000000000004e-05 2024-09-17T21:30:26.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192201+0000", 2024-09-17T21:30:26.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3127999999999999e-05 2024-09-17T21:30:26.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:26.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246938+0000", 2024-09-17T21:30:26.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054736982000000003 2024-09-17T21:30:26.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247000+0000", 2024-09-17T21:30:26.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2794999999999998e-05 2024-09-17T21:30:26.625 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.625 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.625 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.626 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50207 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'303, mlcod=45'299)", 2024-09-17T21:30:26.626 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:26.626 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.385853308, 2024-09-17T21:30:26.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026003433999999999, 2024-09-17T21:30:26.626 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.626 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.626 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:26.626 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:26.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:26.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221663+0000", 2024-09-17T21:30:26.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.271e-06 2024-09-17T21:30:26.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221672+0000", 2024-09-17T21:30:26.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6810000000000004e-06 2024-09-17T21:30:26.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221684+0000", 2024-09-17T21:30:26.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:30:26.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221691+0000", 2024-09-17T21:30:26.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:30:26.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221743+0000", 2024-09-17T21:30:26.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2868999999999998e-05 2024-09-17T21:30:26.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221765+0000", 2024-09-17T21:30:26.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1316999999999999e-05 2024-09-17T21:30:26.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:26.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247616+0000", 2024-09-17T21:30:26.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025851504000000001 2024-09-17T21:30:26.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247665+0000", 2024-09-17T21:30:26.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8739000000000002e-05 2024-09-17T21:30:26.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.631 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.631 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57808 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 196608~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:26.631 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:26.631 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8295542679999999, 2024-09-17T21:30:26.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017751475999999999, 2024-09-17T21:30:26.632 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.632 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.632 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.632 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57808 2024-09-17T21:30:26.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.632 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:26.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:26.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:26.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:30:26.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777968+0000", 2024-09-17T21:30:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8530000000000003e-06 2024-09-17T21:30:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777975+0000", 2024-09-17T21:30:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4410000000000004e-06 2024-09-17T21:30:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777980+0000", 2024-09-17T21:30:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7210000000000002e-06 2024-09-17T21:30:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795323+0000", 2024-09-17T21:30:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017343319999999999 2024-09-17T21:30:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795410+0000", 2024-09-17T21:30:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6567e-05 2024-09-17T21:30:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795712+0000", 2024-09-17T21:30:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030219200000000001 2024-09-17T21:30:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57809 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 458752~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:30:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8295284160000005, 2024-09-17T21:30:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018092869000000001, 2024-09-17T21:30:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57809 2024-09-17T21:30:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:30:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:30:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777987+0000", 2024-09-17T21:30:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T21:30:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777989+0000", 2024-09-17T21:30:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8649999999999999e-06 2024-09-17T21:30:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777994+0000", 2024-09-17T21:30:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4199999999999998e-06 2024-09-17T21:30:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778004+0000", 2024-09-17T21:30:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6900000000000004e-06 2024-09-17T21:30:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795745+0000", 2024-09-17T21:30:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017740859000000001 2024-09-17T21:30:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795815+0000", 2024-09-17T21:30:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0219e-05 2024-09-17T21:30:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796079+0000", 2024-09-17T21:30:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000264446 2024-09-17T21:30:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57810 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 720896~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:30:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8295046890000002, 2024-09-17T21:30:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018361253000000001, 2024-09-17T21:30:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57810 2024-09-17T21:30:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:30:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:30:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778011+0000", 2024-09-17T21:30:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778012+0000", 2024-09-17T21:30:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.872e-06 2024-09-17T21:30:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778018+0000", 2024-09-17T21:30:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8150000000000002e-06 2024-09-17T21:30:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778023+0000", 2024-09-17T21:30:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3939999999999998e-06 2024-09-17T21:30:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796106+0000", 2024-09-17T21:30:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018083446999999999 2024-09-17T21:30:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796159+0000", 2024-09-17T21:30:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3029999999999999e-05 2024-09-17T21:30:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796371+0000", 2024-09-17T21:30:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021231700000000001 2024-09-17T21:30:26.647 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.647 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.647 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.647 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57811 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:26.647 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:26.647 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8294862690000002, 2024-09-17T21:30:26.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018666035000000001, 2024-09-17T21:30:26.647 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.648 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.648 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.648 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.648 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.648 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.648 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57811 2024-09-17T21:30:26.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.648 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:26.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:26.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:26.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:30:26.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778031+0000", 2024-09-17T21:30:26.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.796e-06 2024-09-17T21:30:26.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778036+0000", 2024-09-17T21:30:26.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3290000000000001e-06 2024-09-17T21:30:26.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778041+0000", 2024-09-17T21:30:26.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5839999999999996e-06 2024-09-17T21:30:26.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796409+0000", 2024-09-17T21:30:26.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018368703 2024-09-17T21:30:26.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796483+0000", 2024-09-17T21:30:26.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3751999999999995e-05 2024-09-17T21:30:26.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796695+0000", 2024-09-17T21:30:26.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021153 2024-09-17T21:30:26.652 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.652 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.652 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.652 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57812 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1245184~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:26.652 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:26.652 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8294673719999999, 2024-09-17T21:30:26.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019000064000000001, 2024-09-17T21:30:26.652 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.652 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.653 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.653 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.653 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.653 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.653 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57812 2024-09-17T21:30:26.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.653 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:26.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:26.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:26.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:30:26.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778050+0000", 2024-09-17T21:30:26.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7e-06 2024-09-17T21:30:26.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778055+0000", 2024-09-17T21:30:26.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4049999999999999e-06 2024-09-17T21:30:26.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778059+0000", 2024-09-17T21:30:26.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1479999999999999e-06 2024-09-17T21:30:26.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796734+0000", 2024-09-17T21:30:26.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018674399000000001 2024-09-17T21:30:26.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796786+0000", 2024-09-17T21:30:26.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2879e-05 2024-09-17T21:30:26.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797048+0000", 2024-09-17T21:30:26.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026118400000000003 2024-09-17T21:30:26.657 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.657 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.657 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.657 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57813 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1507328~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:26.657 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:30:26.657 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8294494800000001, 2024-09-17T21:30:26.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019279392999999999, 2024-09-17T21:30:26.657 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.658 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.658 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.658 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.658 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.658 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.658 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57813 2024-09-17T21:30:26.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.658 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:30:26.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:30:26.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778066+0000", 2024-09-17T21:30:26.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6199999999999999e-07 2024-09-17T21:30:26.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778068+0000", 2024-09-17T21:30:26.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7990000000000001e-06 2024-09-17T21:30:26.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778074+0000", 2024-09-17T21:30:26.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:30:26.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778078+0000", 2024-09-17T21:30:26.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2579999999999997e-06 2024-09-17T21:30:26.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797077+0000", 2024-09-17T21:30:26.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018999180000000001 2024-09-17T21:30:26.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797123+0000", 2024-09-17T21:30:26.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5834999999999997e-05 2024-09-17T21:30:26.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797345+0000", 2024-09-17T21:30:26.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022191100000000001 2024-09-17T21:30:26.662 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.662 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.662 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.662 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57814 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:26.663 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:30:26.663 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.829430608, 2024-09-17T21:30:26.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019604947000000001, 2024-09-17T21:30:26.663 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.663 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.663 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.663 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.663 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.663 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.663 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57814 2024-09-17T21:30:26.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.663 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:30:26.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:30:26.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778085+0000", 2024-09-17T21:30:26.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:26.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778086+0000", 2024-09-17T21:30:26.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7850000000000001e-06 2024-09-17T21:30:26.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778093+0000", 2024-09-17T21:30:26.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6449999999999999e-06 2024-09-17T21:30:26.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778098+0000", 2024-09-17T21:30:26.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:30:26.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797383+0000", 2024-09-17T21:30:26.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019284879000000001 2024-09-17T21:30:26.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797431+0000", 2024-09-17T21:30:26.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8856000000000001e-05 2024-09-17T21:30:26.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797689+0000", 2024-09-17T21:30:26.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025785800000000002 2024-09-17T21:30:26.667 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.667 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.667 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.667 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57815 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2031616~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:26.667 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:26.668 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8294099399999997, 2024-09-17T21:30:26.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019910231, 2024-09-17T21:30:26.668 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.668 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.668 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.668 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.668 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.668 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.668 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57815 2024-09-17T21:30:26.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.668 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:26.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:26.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:26.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:30:26.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778118+0000", 2024-09-17T21:30:26.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2967e-05 2024-09-17T21:30:26.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778125+0000", 2024-09-17T21:30:26.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.601e-06 2024-09-17T21:30:26.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778130+0000", 2024-09-17T21:30:26.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1839999999999998e-06 2024-09-17T21:30:26.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797720+0000", 2024-09-17T21:30:26.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019589766000000002 2024-09-17T21:30:26.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797765+0000", 2024-09-17T21:30:26.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5281999999999998e-05 2024-09-17T21:30:26.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798015+0000", 2024-09-17T21:30:26.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025006200000000001 2024-09-17T21:30:26.672 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.672 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.672 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.673 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57816 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2293760~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:26.673 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:26.673 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8293773460000002, 2024-09-17T21:30:26.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020175322999999998, 2024-09-17T21:30:26.673 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.673 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.673 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.673 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.673 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.673 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.673 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57816 2024-09-17T21:30:26.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.673 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:26.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:26.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:26.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:30:26.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778140+0000", 2024-09-17T21:30:26.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T21:30:26.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778145+0000", 2024-09-17T21:30:26.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7320000000000003e-06 2024-09-17T21:30:26.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778154+0000", 2024-09-17T21:30:26.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6969999999999999e-06 2024-09-17T21:30:26.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798049+0000", 2024-09-17T21:30:26.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019894603 2024-09-17T21:30:26.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798106+0000", 2024-09-17T21:30:26.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7207000000000003e-05 2024-09-17T21:30:26.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798313+0000", 2024-09-17T21:30:26.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206861 2024-09-17T21:30:26.677 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.677 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.677 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.677 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57817 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2555904~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:26.678 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:26.678 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8293540999999998, 2024-09-17T21:30:26.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020466314999999999, 2024-09-17T21:30:26.678 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.678 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.678 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.678 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.678 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.678 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.678 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57817 2024-09-17T21:30:26.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.678 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:26.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:26.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:26.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2399999999999999e-07 2024-09-17T21:30:26.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778163+0000", 2024-09-17T21:30:26.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.204e-06 2024-09-17T21:30:26.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778169+0000", 2024-09-17T21:30:26.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:30:26.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778173+0000", 2024-09-17T21:30:26.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:30:26.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798342+0000", 2024-09-17T21:30:26.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020168416000000002 2024-09-17T21:30:26.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798388+0000", 2024-09-17T21:30:26.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6171000000000002e-05 2024-09-17T21:30:26.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798627+0000", 2024-09-17T21:30:26.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000239058 2024-09-17T21:30:26.682 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.682 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.682 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.683 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57818 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2818048~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:26.683 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:30:26.683 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.829335403, 2024-09-17T21:30:26.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020767240999999999, 2024-09-17T21:30:26.683 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.683 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.683 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.683 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57818 2024-09-17T21:30:26.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.684 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:30:26.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:30:26.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778180+0000", 2024-09-17T21:30:26.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:30:26.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778182+0000", 2024-09-17T21:30:26.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7269999999999999e-06 2024-09-17T21:30:26.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778188+0000", 2024-09-17T21:30:26.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6080000000000001e-06 2024-09-17T21:30:26.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778202+0000", 2024-09-17T21:30:26.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3369e-05 2024-09-17T21:30:26.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798660+0000", 2024-09-17T21:30:26.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020458171000000001 2024-09-17T21:30:26.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798704+0000", 2024-09-17T21:30:26.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4273000000000001e-05 2024-09-17T21:30:26.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798947+0000", 2024-09-17T21:30:26.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242651 2024-09-17T21:30:26.688 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.688 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.688 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.688 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57819 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:26.688 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:30:26.688 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8293045340000003, 2024-09-17T21:30:26.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020892340999999998, 2024-09-17T21:30:26.689 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.689 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.689 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.689 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.689 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.689 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.689 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57819 2024-09-17T21:30:26.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.689 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:30:26.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:30:26.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778211+0000", 2024-09-17T21:30:26.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:30:26.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778217+0000", 2024-09-17T21:30:26.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T21:30:26.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778224+0000", 2024-09-17T21:30:26.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3520000000000003e-06 2024-09-17T21:30:26.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778228+0000", 2024-09-17T21:30:26.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7480000000000001e-06 2024-09-17T21:30:26.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798977+0000", 2024-09-17T21:30:26.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020748571 2024-09-17T21:30:26.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799035+0000", 2024-09-17T21:30:26.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7908999999999997e-05 2024-09-17T21:30:26.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799103+0000", 2024-09-17T21:30:26.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7794999999999997e-05 2024-09-17T21:30:26.693 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.693 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.693 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.693 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57820 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:26.693 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:26.694 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8292793419999995, 2024-09-17T21:30:26.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021016658000000001, 2024-09-17T21:30:26.694 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.694 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.694 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.694 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57820 2024-09-17T21:30:26.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.694 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:26.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:26.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:26.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:30:26.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778238+0000", 2024-09-17T21:30:26.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.945e-06 2024-09-17T21:30:26.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778244+0000", 2024-09-17T21:30:26.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6520000000000003e-06 2024-09-17T21:30:26.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778248+0000", 2024-09-17T21:30:26.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5920000000000002e-06 2024-09-17T21:30:26.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799131+0000", 2024-09-17T21:30:26.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020883288999999999 2024-09-17T21:30:26.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799185+0000", 2024-09-17T21:30:26.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3223000000000002e-05 2024-09-17T21:30:26.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799252+0000", 2024-09-17T21:30:26.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7576999999999994e-05 2024-09-17T21:30:26.698 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.698 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.698 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.699 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57821 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:26.699 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:26.699 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8292609970000004, 2024-09-17T21:30:26.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021153528000000001, 2024-09-17T21:30:26.699 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.699 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.699 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.699 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57821 2024-09-17T21:30:26.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.700 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:26.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:26.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:26.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-07 2024-09-17T21:30:26.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778256+0000", 2024-09-17T21:30:26.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.604e-06 2024-09-17T21:30:26.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778262+0000", 2024-09-17T21:30:26.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7629999999999997e-06 2024-09-17T21:30:26.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778266+0000", 2024-09-17T21:30:26.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:30:26.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799284+0000", 2024-09-17T21:30:26.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021017312 2024-09-17T21:30:26.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799326+0000", 2024-09-17T21:30:26.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2639000000000003e-05 2024-09-17T21:30:26.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799407+0000", 2024-09-17T21:30:26.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1077999999999999e-05 2024-09-17T21:30:26.703 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.703 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.703 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.704 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57822 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:26.704 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:30:26.704 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8292426869999998, 2024-09-17T21:30:26.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021268966, 2024-09-17T21:30:26.704 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.704 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.704 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.705 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.705 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.705 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57822 2024-09-17T21:30:26.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.705 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:30:26.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:30:26.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778273+0000", 2024-09-17T21:30:26.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:30:26.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778274+0000", 2024-09-17T21:30:26.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.598e-06 2024-09-17T21:30:26.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778283+0000", 2024-09-17T21:30:26.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8279999999999992e-06 2024-09-17T21:30:26.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778289+0000", 2024-09-17T21:30:26.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5609999999999998e-06 2024-09-17T21:30:26.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799432+0000", 2024-09-17T21:30:26.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021143862999999999 2024-09-17T21:30:26.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799473+0000", 2024-09-17T21:30:26.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1075999999999998e-05 2024-09-17T21:30:26.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799541+0000", 2024-09-17T21:30:26.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7657999999999995e-05 2024-09-17T21:30:26.709 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.709 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.709 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.709 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57823 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:26.709 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:26.709 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8292197259999998, 2024-09-17T21:30:26.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021379770999999999, 2024-09-17T21:30:26.709 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:26.709 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:26.710 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:26.710 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:26.710 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:26.710 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:26.710 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57823 2024-09-17T21:30:26.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.710 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:26.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:26.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:26.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:26.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:26.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:26.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:26.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:26.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:30:26.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:26.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778297+0000", 2024-09-17T21:30:26.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:30:26.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:26.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778303+0000", 2024-09-17T21:30:26.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4559999999999999e-06 2024-09-17T21:30:26.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:26.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778307+0000", 2024-09-17T21:30:26.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6229999999999996e-06 2024-09-17T21:30:26.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:26.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799560+0000", 2024-09-17T21:30:26.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021252931999999999 2024-09-17T21:30:26.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:26.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799609+0000", 2024-09-17T21:30:26.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8301999999999999e-05 2024-09-17T21:30:26.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:26.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:26.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:26.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799675+0000", 2024-09-17T21:30:26.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6205999999999996e-05 2024-09-17T21:30:26.714 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.714 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.714 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.714 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:26.714 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:26.714 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:26.715 INFO:teuthology.orchestra.run.smithi129.stdout:3323 read 0x5e675ea thru 0x5e76573 (0xef8a bytes) 2024-09-17T21:30:26.715 INFO:teuthology.orchestra.run.smithi129.stdout:3324 writesame 0x47f93aa thru 0x48062c3 data_size 0x31(0xcf1a bytes) 2024-09-17T21:30:26.715 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.716 INFO:teuthology.orchestra.run.smithi129.stdout:3327 read 0x77e9f7d thru 0x77efb82 (0x5c06 bytes) 2024-09-17T21:30:26.716 INFO:teuthology.orchestra.run.smithi129.stdout:3328 read 0x5b4ccc2 thru 0x5b593ea (0xc729 bytes) 2024-09-17T21:30:26.716 INFO:teuthology.orchestra.run.smithi129.stdout:3329 punch from 0x909fb6 to 0x90f4b9, (0x5503 bytes) 2024-09-17T21:30:26.716 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.716 INFO:teuthology.orchestra.run.smithi129.stdout:3334 write 0x5aaaa76 thru 0x5aae08c (0x3617 bytes) 2024-09-17T21:30:26.717 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:30:26.718 INFO:teuthology.orchestra.run.smithi129.stdout:3335 read 0x4b3f4ad thru 0x4b4e6b3 (0xf207 bytes) 2024-09-17T21:30:26.718 INFO:teuthology.orchestra.run.smithi129.stdout:3339 punch from 0xe506911 to 0xe50821d, (0x190c bytes) 2024-09-17T21:30:26.718 INFO:teuthology.orchestra.run.smithi129.stdout:3340 write 0x1f2c4ea thru 0x1f391c7 (0xccde bytes) 2024-09-17T21:30:26.719 INFO:teuthology.orchestra.run.smithi129.stdout:3341 punch from 0x9affc58 to 0x9b00d66, (0x110e bytes) 2024-09-17T21:30:26.719 INFO:teuthology.orchestra.run.smithi129.stdout:3343 read 0xbfa403c thru 0xbfad1d7 (0x919c bytes) 2024-09-17T21:30:26.719 INFO:teuthology.orchestra.run.smithi129.stdout:3344 read 0x7b4e350 thru 0x7b5ad23 (0xc9d4 bytes) 2024-09-17T21:30:26.719 INFO:teuthology.orchestra.run.smithi129.stdout:3345 write 0x4959895 thru 0x495b492 (0x1bfe bytes) 2024-09-17T21:30:26.719 INFO:teuthology.orchestra.run.smithi129.stdout:3346 punch from 0x982ee2d to 0x9831b45, (0x2d18 bytes) 2024-09-17T21:30:26.719 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:26.624+0000 7f7a098ed640 -1 osd.1 47 log_to_monitors true 2024-09-17T21:30:26.722 INFO:teuthology.orchestra.run.smithi129.stdout:3348 write 0x54cfa0f thru 0x54d5688 (0x5c7a bytes) 2024-09-17T21:30:26.722 INFO:teuthology.orchestra.run.smithi129.stdout:3349 write 0xa65e0b thru 0xa6db56 (0x7d4c bytes) 2024-09-17T21:30:26.722 INFO:teuthology.orchestra.run.smithi129.stdout:3351 write 0x548087a thru 0x548869e (0x7e25 bytes) 2024-09-17T21:30:26.722 INFO:teuthology.orchestra.run.smithi129.stdout:3352 writesame 0x1bd74bc thru 0x1be1592 data_size 0xf(0xa0d7 bytes) 2024-09-17T21:30:26.722 INFO:teuthology.orchestra.run.smithi129.stdout:3353 write 0x61d2934 thru 0x61d923f (0x690c bytes) 2024-09-17T21:30:26.722 INFO:teuthology.orchestra.run.smithi129.stdout:3357 read 0x4d823e9 thru 0x4d8719f (0x4db7 bytes) 2024-09-17T21:30:26.722 INFO:teuthology.orchestra.run.smithi129.stdout:3359 write 0x8f4f5f2 thru 0x8f531e9 (0x3bf8 bytes) 2024-09-17T21:30:26.723 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.723 INFO:teuthology.orchestra.run.smithi129.stdout:3361 write 0x20bafb5 thru 0x20bdc17 (0x2c63 bytes) 2024-09-17T21:30:26.723 INFO:teuthology.orchestra.run.smithi129.stdout:3362 read 0x83aaaf0 thru 0x83baa02 (0xff13 bytes) 2024-09-17T21:30:26.723 INFO:teuthology.orchestra.run.smithi129.stdout:3363 read 0x844485e thru 0x845313f (0xe8e2 bytes) 2024-09-17T21:30:26.723 INFO:teuthology.orchestra.run.smithi129.stdout:3364 punch from 0xc3b15d4 to 0xc3b9dc6, (0x87f2 bytes) 2024-09-17T21:30:26.723 INFO:teuthology.orchestra.run.smithi129.stdout:3368 read 0xc02f3ef thru 0xc031bbe (0x27d0 bytes) 2024-09-17T21:30:26.723 INFO:teuthology.orchestra.run.smithi129.stdout:3369 write 0x179a73f thru 0x17a60cf (0xb991 bytes) 2024-09-17T21:30:26.723 INFO:teuthology.orchestra.run.smithi129.stdout:3370 writesame 0x9a6ae32 thru 0x9a71348 data_size 0x1(0x6517 bytes) 2024-09-17T21:30:26.723 INFO:teuthology.orchestra.run.smithi129.stdout:3371 write 0x4804ce0 thru 0x480d1b9 (0x84da bytes) 2024-09-17T21:30:26.723 INFO:teuthology.orchestra.run.smithi129.stdout:3372 writesame 0x4b9649a thru 0x4ba1a5d data_size 0x2(0xb5c4 bytes) 2024-09-17T21:30:26.723 INFO:teuthology.orchestra.run.smithi129.stdout:3373 read 0x3741f66 thru 0x37518dc (0xf977 bytes) 2024-09-17T21:30:26.723 INFO:teuthology.orchestra.run.smithi129.stdout:3375 punch from 0x14e8799 to 0x14f58a7, (0xd10e bytes) 2024-09-17T21:30:26.724 INFO:teuthology.orchestra.run.smithi129.stdout:3376 writesame 0xa507749 thru 0xa50de5e data_size 0x5b(0x6716 bytes) 2024-09-17T21:30:26.724 INFO:teuthology.orchestra.run.smithi129.stdout:3378 trunc from 0xecc1203 to 0xa03c45a 2024-09-17T21:30:26.724 INFO:teuthology.orchestra.run.smithi129.stdout:3380 trunc from 0xa03c45a to 0xd1213f3 2024-09-17T21:30:26.724 INFO:teuthology.orchestra.run.smithi129.stdout:3382 punch from 0x95a7991 to 0x95ac0ec, (0x475b bytes) 2024-09-17T21:30:26.724 INFO:teuthology.orchestra.run.smithi129.stdout:3384 punch from 0x4ca7e47 to 0x4cb7b3e, (0xfcf7 bytes) 2024-09-17T21:30:26.724 INFO:teuthology.orchestra.run.smithi129.stdout:3385 read 0x845dea1 thru 0x846b3d6 (0xd536 bytes) 2024-09-17T21:30:26.724 INFO:teuthology.orchestra.run.smithi129.stdout:3389 trunc from 0xd1213f3 to 0xac24efa 2024-09-17T21:30:26.724 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.724 INFO:teuthology.orchestra.run.smithi129.stdout:3391 punch from 0x5cec55a to 0x5cfaf1e, (0xe9c4 bytes) 2024-09-17T21:30:26.724 INFO:teuthology.orchestra.run.smithi129.stdout:3392 writesame 0xa350802 thru 0xa36016b data_size 0x5(0xf96a bytes) 2024-09-17T21:30:26.724 INFO:teuthology.orchestra.run.smithi129.stdout:3395 trunc from 0xac24efa to 0x4362b7a 2024-09-17T21:30:26.737 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:26.735+0000 7f79fd69b640 -1 osd.1 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:30:26.737 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:26.735+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:26.742 INFO:teuthology.orchestra.run.smithi129.stdout:3396 write 0x5861948 thru 0x5869712 (0x7dcb bytes) 2024-09-17T21:30:26.748 INFO:teuthology.orchestra.run.smithi129.stdout:3397 punch from 0x54432d6 to 0x54461f4, (0x2f1e bytes) 2024-09-17T21:30:26.749 INFO:teuthology.orchestra.run.smithi129.stdout:3400 read 0x1728f3e thru 0x172c150 (0x3213 bytes) 2024-09-17T21:30:26.749 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:26.760 INFO:teuthology.orchestra.run.smithi129.stdout:3405 writesame 0xa6a875d thru 0xa6b0135 data_size 0x1(0x79d9 bytes) 2024-09-17T21:30:26.760 INFO:teuthology.orchestra.run.smithi129.stdout:3406 read 0x2640c2c thru 0x264490d (0x3ce2 bytes) 2024-09-17T21:30:26.760 INFO:teuthology.orchestra.run.smithi129.stdout:3407 punch from 0x6764c9b to 0x6764cdb, (0x40 bytes) 2024-09-17T21:30:26.761 INFO:teuthology.orchestra.run.smithi129.stdout:3408 trunc from 0xa6b0136 to 0xb7806d7 2024-09-17T21:30:26.766 INFO:teuthology.orchestra.run.smithi129.stdout:3409 clone 26 order 23 su 32768 sc 7 2024-09-17T21:30:26.819 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:26.818+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:26.842 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:26.842 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:26.842 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:26.842 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:26.850 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:30:26.934 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:26.933+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:26.982 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:30:26.994 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:26.994 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:26.994 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:26.994 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:26.994 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:27.003 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:30:27.021 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:27.019+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:27.105 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:27.105 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:27.106 INFO:teuthology.orchestra.run.smithi129.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-17T21:30:27.122 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:27.120+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:27.122 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:27.122 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:27.139 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:27.139 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:27.150 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:27.150 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:27.156 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:27.156 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:27.159 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:30:27.172 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:27.173 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:27.189 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:27.190 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:27.206 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:27.206 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:27.209 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:30:27.209 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:30:27.223 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:27.221+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:27.270 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:27.271 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:27.279 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:30:27.324 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:27.322+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:27.423 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:27.424 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:27.425 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:27.423+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:27.432 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:30:27.526 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:27.524+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:27.575 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:27.575 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:27.583 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:30:27.627 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:27.625+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:27.688 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:27.688 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:27.696 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:30:27.731 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:30:27.737 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:27.735+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:27.828 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:27.826+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:27.839 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:27.840 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:27.849 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:30:27.929 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:27.927+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:27.952 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:27.952 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:27.961 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:30:28.045 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:28.043+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:28.104 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:28.104 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:28.113 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:30:28.130 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:28.129+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:28.231 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:28.229+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:28.257 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:28.258 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:28.266 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:30:28.351 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:28.349+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:28.410 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:28.410 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:28.418 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:30:28.459 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:28.457+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:28.534 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:28.532+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:28.562 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:28.562 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:28.571 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:30:28.635 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:28.632+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:28.715 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:28.715 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:28.724 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:30:28.746 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is set 2024-09-17T21:30:28.769 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:28.767+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:28.834 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:28.834 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:28.836 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:28.834+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:28.842 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:30:28.937 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:28.935+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:28.952 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:28.952 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:28.961 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:30:29.039 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:29.036+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:29.064 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:29.064 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:29.073 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:30:29.139 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:29.137+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:29.177 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:29.177 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:29.186 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:30:29.210 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:30:29.269 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:29.267+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:29.333 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:29.333 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:29.341 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:29.339+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:29.342 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:30:29.349 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:29.349 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:29.366 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:30:29.366 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-17T21:30:29.382 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:29.382 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:29.399 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:29.399 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:29.429 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:29.430 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:29.442 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:29.439+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:29.457 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:29.457 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:29.471 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:29.471 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:29.471 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:29.471 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:29.471 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:29.474 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:29.474 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:29.480 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:30:29.491 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:29.491 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:29.543 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:29.541+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:29.594 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "pg_update_log_missing(3.5 epoch 45/37 rep_tid 844 entries 45'73 (0'0) error 3:a65c40ae:::rbd_data.1120b21c3309.0000000000000126:head by client.4384.0:48900 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 45'70)", 2024-09-17T21:30:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.544027533000001, 2024-09-17T21:30:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019554365000000001, 2024-09-17T21:30:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T21:30:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047112+0000", 2024-09-17T21:30:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:30:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047121+0000", 2024-09-17T21:30:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4379999999999995e-06 2024-09-17T21:30:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047139+0000", 2024-09-17T21:30:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8235000000000001e-05 2024-09-17T21:30:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047303+0000", 2024-09-17T21:30:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016389400000000001 2024-09-17T21:30:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047396+0000", 2024-09-17T21:30:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3355000000000003e-05 2024-09-17T21:30:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066666+0000", 2024-09-17T21:30:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019269649999999999 2024-09-17T21:30:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48906 3.4 e45/25 3:2503fe27:::rbd_data.1120b21c3309.0000000000000127:head v 45'148, mlcod=45'146)", 2024-09-17T21:30:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.541901354, 2024-09-17T21:30:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018169739000000001, 2024-09-17T21:30:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:30:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049244+0000", 2024-09-17T21:30:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8390000000000003e-06 2024-09-17T21:30:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066411+0000", 2024-09-17T21:30:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017166818 2024-09-17T21:30:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066426+0000", 2024-09-17T21:30:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4888999999999999e-05 2024-09-17T21:30:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066611+0000", 2024-09-17T21:30:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018464499999999999 2024-09-17T21:30:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066630+0000", 2024-09-17T21:30:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9201000000000001e-05 2024-09-17T21:30:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067373+0000", 2024-09-17T21:30:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00074322600000000004 2024-09-17T21:30:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067407+0000", 2024-09-17T21:30:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4357999999999999e-05 2024-09-17T21:30:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50206 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'301, mlcod=45'299)", 2024-09-17T21:30:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.412424510000001, 2024-09-17T21:30:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068285810000000002, 2024-09-17T21:30:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178716+0000", 2024-09-17T21:30:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.438e-06 2024-09-17T21:30:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192044+0000", 2024-09-17T21:30:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013328370000000001 2024-09-17T21:30:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192069+0000", 2024-09-17T21:30:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4496000000000001e-05 2024-09-17T21:30:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192085+0000", 2024-09-17T21:30:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5758999999999999e-05 2024-09-17T21:30:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192167+0000", 2024-09-17T21:30:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2842000000000004e-05 2024-09-17T21:30:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192201+0000", 2024-09-17T21:30:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3127999999999999e-05 2024-09-17T21:30:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246938+0000", 2024-09-17T21:30:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054736982000000003 2024-09-17T21:30:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247000+0000", 2024-09-17T21:30:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2794999999999998e-05 2024-09-17T21:30:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50207 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'303, mlcod=45'299)", 2024-09-17T21:30:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.369477460999999, 2024-09-17T21:30:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026003433999999999, 2024-09-17T21:30:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221663+0000", 2024-09-17T21:30:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.271e-06 2024-09-17T21:30:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221672+0000", 2024-09-17T21:30:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6810000000000004e-06 2024-09-17T21:30:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221684+0000", 2024-09-17T21:30:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:30:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221691+0000", 2024-09-17T21:30:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:30:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221743+0000", 2024-09-17T21:30:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2868999999999998e-05 2024-09-17T21:30:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221765+0000", 2024-09-17T21:30:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1316999999999999e-05 2024-09-17T21:30:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247616+0000", 2024-09-17T21:30:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025851504000000001 2024-09-17T21:30:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247665+0000", 2024-09-17T21:30:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8739000000000002e-05 2024-09-17T21:30:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57808 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 196608~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.8131784209999999, 2024-09-17T21:30:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017751475999999999, 2024-09-17T21:30:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57808 2024-09-17T21:30:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:30:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777968+0000", 2024-09-17T21:30:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8530000000000003e-06 2024-09-17T21:30:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777975+0000", 2024-09-17T21:30:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4410000000000004e-06 2024-09-17T21:30:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777980+0000", 2024-09-17T21:30:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7210000000000002e-06 2024-09-17T21:30:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795323+0000", 2024-09-17T21:30:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017343319999999999 2024-09-17T21:30:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795410+0000", 2024-09-17T21:30:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6567e-05 2024-09-17T21:30:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795712+0000", 2024-09-17T21:30:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030219200000000001 2024-09-17T21:30:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57809 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 458752~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:30:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.8131525689999997, 2024-09-17T21:30:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018092869000000001, 2024-09-17T21:30:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57809 2024-09-17T21:30:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:30:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:30:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777987+0000", 2024-09-17T21:30:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T21:30:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777989+0000", 2024-09-17T21:30:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8649999999999999e-06 2024-09-17T21:30:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777994+0000", 2024-09-17T21:30:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4199999999999998e-06 2024-09-17T21:30:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778004+0000", 2024-09-17T21:30:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6900000000000004e-06 2024-09-17T21:30:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795745+0000", 2024-09-17T21:30:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017740859000000001 2024-09-17T21:30:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795815+0000", 2024-09-17T21:30:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0219e-05 2024-09-17T21:30:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796079+0000", 2024-09-17T21:30:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000264446 2024-09-17T21:30:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57810 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 720896~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:30:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.8131288420000002, 2024-09-17T21:30:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018361253000000001, 2024-09-17T21:30:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57810 2024-09-17T21:30:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:30:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:30:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778011+0000", 2024-09-17T21:30:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778012+0000", 2024-09-17T21:30:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.872e-06 2024-09-17T21:30:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778018+0000", 2024-09-17T21:30:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8150000000000002e-06 2024-09-17T21:30:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778023+0000", 2024-09-17T21:30:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3939999999999998e-06 2024-09-17T21:30:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796106+0000", 2024-09-17T21:30:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018083446999999999 2024-09-17T21:30:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796159+0000", 2024-09-17T21:30:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3029999999999999e-05 2024-09-17T21:30:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796371+0000", 2024-09-17T21:30:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021231700000000001 2024-09-17T21:30:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57811 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.8131104220000003, 2024-09-17T21:30:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018666035000000001, 2024-09-17T21:30:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57811 2024-09-17T21:30:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:30:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778031+0000", 2024-09-17T21:30:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.796e-06 2024-09-17T21:30:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778036+0000", 2024-09-17T21:30:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3290000000000001e-06 2024-09-17T21:30:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778041+0000", 2024-09-17T21:30:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5839999999999996e-06 2024-09-17T21:30:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796409+0000", 2024-09-17T21:30:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018368703 2024-09-17T21:30:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796483+0000", 2024-09-17T21:30:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3751999999999995e-05 2024-09-17T21:30:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796695+0000", 2024-09-17T21:30:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021153 2024-09-17T21:30:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57812 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1245184~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.8130915249999999, 2024-09-17T21:30:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019000064000000001, 2024-09-17T21:30:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57812 2024-09-17T21:30:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:30:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778050+0000", 2024-09-17T21:30:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7e-06 2024-09-17T21:30:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778055+0000", 2024-09-17T21:30:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4049999999999999e-06 2024-09-17T21:30:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778059+0000", 2024-09-17T21:30:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1479999999999999e-06 2024-09-17T21:30:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796734+0000", 2024-09-17T21:30:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018674399000000001 2024-09-17T21:30:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796786+0000", 2024-09-17T21:30:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2879e-05 2024-09-17T21:30:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797048+0000", 2024-09-17T21:30:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026118400000000003 2024-09-17T21:30:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57813 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1507328~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:30:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.8130736330000001, 2024-09-17T21:30:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019279392999999999, 2024-09-17T21:30:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57813 2024-09-17T21:30:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:30:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:30:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778066+0000", 2024-09-17T21:30:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6199999999999999e-07 2024-09-17T21:30:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778068+0000", 2024-09-17T21:30:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7990000000000001e-06 2024-09-17T21:30:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778074+0000", 2024-09-17T21:30:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:30:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778078+0000", 2024-09-17T21:30:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2579999999999997e-06 2024-09-17T21:30:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797077+0000", 2024-09-17T21:30:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018999180000000001 2024-09-17T21:30:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797123+0000", 2024-09-17T21:30:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5834999999999997e-05 2024-09-17T21:30:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797345+0000", 2024-09-17T21:30:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022191100000000001 2024-09-17T21:30:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57814 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:30:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.8130547610000001, 2024-09-17T21:30:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019604947000000001, 2024-09-17T21:30:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57814 2024-09-17T21:30:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:30:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:30:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778085+0000", 2024-09-17T21:30:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778086+0000", 2024-09-17T21:30:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7850000000000001e-06 2024-09-17T21:30:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778093+0000", 2024-09-17T21:30:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6449999999999999e-06 2024-09-17T21:30:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778098+0000", 2024-09-17T21:30:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:30:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797383+0000", 2024-09-17T21:30:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019284879000000001 2024-09-17T21:30:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797431+0000", 2024-09-17T21:30:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8856000000000001e-05 2024-09-17T21:30:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797689+0000", 2024-09-17T21:30:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025785800000000002 2024-09-17T21:30:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57815 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2031616~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.8130340929999997, 2024-09-17T21:30:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019910231, 2024-09-17T21:30:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57815 2024-09-17T21:30:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:29.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:30:29.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778118+0000", 2024-09-17T21:30:29.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2967e-05 2024-09-17T21:30:29.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778125+0000", 2024-09-17T21:30:29.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.601e-06 2024-09-17T21:30:29.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778130+0000", 2024-09-17T21:30:29.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1839999999999998e-06 2024-09-17T21:30:29.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797720+0000", 2024-09-17T21:30:29.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019589766000000002 2024-09-17T21:30:29.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797765+0000", 2024-09-17T21:30:29.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5281999999999998e-05 2024-09-17T21:30:29.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798015+0000", 2024-09-17T21:30:29.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025006200000000001 2024-09-17T21:30:29.658 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.658 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.658 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.659 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57816 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2293760~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:29.659 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:29.659 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.8130014990000003, 2024-09-17T21:30:29.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020175322999999998, 2024-09-17T21:30:29.659 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.659 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.659 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:29.659 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:29.659 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:29.659 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:29.660 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57816 2024-09-17T21:30:29.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.660 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:29.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:29.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:29.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:30:29.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778140+0000", 2024-09-17T21:30:29.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T21:30:29.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778145+0000", 2024-09-17T21:30:29.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7320000000000003e-06 2024-09-17T21:30:29.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778154+0000", 2024-09-17T21:30:29.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6969999999999999e-06 2024-09-17T21:30:29.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798049+0000", 2024-09-17T21:30:29.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019894603 2024-09-17T21:30:29.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798106+0000", 2024-09-17T21:30:29.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7207000000000003e-05 2024-09-17T21:30:29.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798313+0000", 2024-09-17T21:30:29.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206861 2024-09-17T21:30:29.664 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.664 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.664 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.664 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57817 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2555904~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:29.664 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:29.664 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.8129782529999998, 2024-09-17T21:30:29.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020466314999999999, 2024-09-17T21:30:29.664 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.664 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.665 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:29.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:29.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:29.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:29.665 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57817 2024-09-17T21:30:29.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.665 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:29.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:29.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:29.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2399999999999999e-07 2024-09-17T21:30:29.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778163+0000", 2024-09-17T21:30:29.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.204e-06 2024-09-17T21:30:29.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778169+0000", 2024-09-17T21:30:29.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:30:29.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778173+0000", 2024-09-17T21:30:29.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:30:29.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798342+0000", 2024-09-17T21:30:29.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020168416000000002 2024-09-17T21:30:29.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798388+0000", 2024-09-17T21:30:29.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6171000000000002e-05 2024-09-17T21:30:29.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798627+0000", 2024-09-17T21:30:29.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000239058 2024-09-17T21:30:29.669 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.669 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.669 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.669 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57818 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2818048~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:29.669 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:30:29.670 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.812959556, 2024-09-17T21:30:29.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020767240999999999, 2024-09-17T21:30:29.670 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.670 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.670 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:29.670 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:29.670 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:29.670 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:29.670 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57818 2024-09-17T21:30:29.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.670 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:30:29.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:30:29.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778180+0000", 2024-09-17T21:30:29.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:30:29.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778182+0000", 2024-09-17T21:30:29.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7269999999999999e-06 2024-09-17T21:30:29.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778188+0000", 2024-09-17T21:30:29.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6080000000000001e-06 2024-09-17T21:30:29.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778202+0000", 2024-09-17T21:30:29.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3369e-05 2024-09-17T21:30:29.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798660+0000", 2024-09-17T21:30:29.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020458171000000001 2024-09-17T21:30:29.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798704+0000", 2024-09-17T21:30:29.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4273000000000001e-05 2024-09-17T21:30:29.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798947+0000", 2024-09-17T21:30:29.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242651 2024-09-17T21:30:29.674 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.674 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.674 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.675 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57819 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:29.675 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:30:29.675 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.8129286870000003, 2024-09-17T21:30:29.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020892340999999998, 2024-09-17T21:30:29.675 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.675 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.675 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:29.675 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:29.675 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:29.675 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:29.676 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57819 2024-09-17T21:30:29.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.676 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:30:29.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:30:29.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778211+0000", 2024-09-17T21:30:29.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:30:29.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778217+0000", 2024-09-17T21:30:29.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T21:30:29.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778224+0000", 2024-09-17T21:30:29.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3520000000000003e-06 2024-09-17T21:30:29.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778228+0000", 2024-09-17T21:30:29.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7480000000000001e-06 2024-09-17T21:30:29.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798977+0000", 2024-09-17T21:30:29.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020748571 2024-09-17T21:30:29.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799035+0000", 2024-09-17T21:30:29.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7908999999999997e-05 2024-09-17T21:30:29.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799103+0000", 2024-09-17T21:30:29.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7794999999999997e-05 2024-09-17T21:30:29.679 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.680 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.680 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.680 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57820 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:29.680 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:29.680 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.8129034950000005, 2024-09-17T21:30:29.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021016658000000001, 2024-09-17T21:30:29.680 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.680 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.680 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:29.681 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:29.681 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:29.681 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:29.681 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57820 2024-09-17T21:30:29.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.681 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:29.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:29.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:29.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:30:29.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778238+0000", 2024-09-17T21:30:29.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.945e-06 2024-09-17T21:30:29.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778244+0000", 2024-09-17T21:30:29.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6520000000000003e-06 2024-09-17T21:30:29.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778248+0000", 2024-09-17T21:30:29.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5920000000000002e-06 2024-09-17T21:30:29.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799131+0000", 2024-09-17T21:30:29.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020883288999999999 2024-09-17T21:30:29.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799185+0000", 2024-09-17T21:30:29.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3223000000000002e-05 2024-09-17T21:30:29.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799252+0000", 2024-09-17T21:30:29.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7576999999999994e-05 2024-09-17T21:30:29.685 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.685 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.685 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.685 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57821 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:29.685 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:29.685 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.8128851499999996, 2024-09-17T21:30:29.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021153528000000001, 2024-09-17T21:30:29.686 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.686 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.686 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:29.686 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:29.686 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:29.686 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:29.686 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57821 2024-09-17T21:30:29.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.686 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:29.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:29.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:29.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-07 2024-09-17T21:30:29.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778256+0000", 2024-09-17T21:30:29.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.604e-06 2024-09-17T21:30:29.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778262+0000", 2024-09-17T21:30:29.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7629999999999997e-06 2024-09-17T21:30:29.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778266+0000", 2024-09-17T21:30:29.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:30:29.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799284+0000", 2024-09-17T21:30:29.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021017312 2024-09-17T21:30:29.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799326+0000", 2024-09-17T21:30:29.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2639000000000003e-05 2024-09-17T21:30:29.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799407+0000", 2024-09-17T21:30:29.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1077999999999999e-05 2024-09-17T21:30:29.690 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.690 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.690 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.691 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57822 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:29.691 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:30:29.691 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.8128668399999999, 2024-09-17T21:30:29.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021268966, 2024-09-17T21:30:29.691 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.691 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.691 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:29.691 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:29.691 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:29.691 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:29.691 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57822 2024-09-17T21:30:29.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.692 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:30:29.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:30:29.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778273+0000", 2024-09-17T21:30:29.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:30:29.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778274+0000", 2024-09-17T21:30:29.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.598e-06 2024-09-17T21:30:29.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778283+0000", 2024-09-17T21:30:29.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8279999999999992e-06 2024-09-17T21:30:29.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778289+0000", 2024-09-17T21:30:29.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5609999999999998e-06 2024-09-17T21:30:29.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799432+0000", 2024-09-17T21:30:29.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021143862999999999 2024-09-17T21:30:29.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799473+0000", 2024-09-17T21:30:29.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1075999999999998e-05 2024-09-17T21:30:29.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799541+0000", 2024-09-17T21:30:29.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7657999999999995e-05 2024-09-17T21:30:29.695 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.695 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.696 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.696 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57823 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:29.696 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:29.696 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.8128438789999999, 2024-09-17T21:30:29.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021379770999999999, 2024-09-17T21:30:29.696 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:29.696 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:29.696 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:29.696 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:29.697 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:29.697 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:29.697 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57823 2024-09-17T21:30:29.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.697 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:29.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:29.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:29.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:29.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:29.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:29.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:29.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:29.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:30:29.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:29.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778297+0000", 2024-09-17T21:30:29.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:30:29.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:29.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778303+0000", 2024-09-17T21:30:29.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4559999999999999e-06 2024-09-17T21:30:29.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:29.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778307+0000", 2024-09-17T21:30:29.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6229999999999996e-06 2024-09-17T21:30:29.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:29.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799560+0000", 2024-09-17T21:30:29.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021252931999999999 2024-09-17T21:30:29.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:29.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799609+0000", 2024-09-17T21:30:29.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8301999999999999e-05 2024-09-17T21:30:29.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:29.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:29.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:29.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799675+0000", 2024-09-17T21:30:29.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6205999999999996e-05 2024-09-17T21:30:29.701 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.701 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.701 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.701 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:29.701 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:29.701 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:29.703 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:30:29.703 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:29.699+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:29.804 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:29.801+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:29.810 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:29.810 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:29.810 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:29.810 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:29.818 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:30:29.905 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:29.903+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:29.921 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:29.922 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:29.922 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:29.922 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:29.922 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:29.931 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:30:30.006 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:30.004+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:30.034 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:30.034 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:30.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:30.034 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:30.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.034 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:49361 3.19 e46/25 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:head v 46'135, mlcod=45'133)", 2024-09-17T21:30:30.034 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:30.035 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.277942464999999, 2024-09-17T21:30:30.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043477744999999998, 2024-09-17T21:30:30.035 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.035 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.035 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:30.035 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:30.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:30.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:30.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-07 2024-09-17T21:30:30.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769703+0000", 2024-09-17T21:30:30.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016134381 2024-09-17T21:30:30.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769724+0000", 2024-09-17T21:30:30.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0781999999999999e-05 2024-09-17T21:30:30.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769734+0000", 2024-09-17T21:30:30.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6870000000000006e-06 2024-09-17T21:30:30.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769797+0000", 2024-09-17T21:30:30.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3418999999999997e-05 2024-09-17T21:30:30.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769822+0000", 2024-09-17T21:30:30.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4652999999999999e-05 2024-09-17T21:30:30.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:30.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.796982+0000", 2024-09-17T21:30:30.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027160205 2024-09-17T21:30:30.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.797047+0000", 2024-09-17T21:30:30.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4455999999999994e-05 2024-09-17T21:30:30.039 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.039 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.039 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.039 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50804 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 14155776~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.039 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495110+0000", 2024-09-17T21:30:30.039 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536401660999999, 2024-09-17T21:30:30.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.034879440999999997, 2024-09-17T21:30:30.039 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.040 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.040 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.040 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.040 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.040 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.040 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50804 2024-09-17T21:30:30.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.040 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495110+0000", 2024-09-17T21:30:30.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495110+0000", 2024-09-17T21:30:30.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495110+0000", 2024-09-17T21:30:30.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2e-07 2024-09-17T21:30:30.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495112+0000", 2024-09-17T21:30:30.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7689999999999999e-06 2024-09-17T21:30:30.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495133+0000", 2024-09-17T21:30:30.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1319e-05 2024-09-17T21:30:30.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495138+0000", 2024-09-17T21:30:30.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:30:30.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529422+0000", 2024-09-17T21:30:30.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.034284199000000001 2024-09-17T21:30:30.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529492+0000", 2024-09-17T21:30:30.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9502e-05 2024-09-17T21:30:30.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529989+0000", 2024-09-17T21:30:30.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00049732900000000002 2024-09-17T21:30:30.043 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.043 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.044 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.044 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50805 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 17825792~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.044 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:30:30.044 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536358970999999, 2024-09-17T21:30:30.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035591579999999998, 2024-09-17T21:30:30.044 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.044 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.044 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.044 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.044 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.044 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.045 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50805 2024-09-17T21:30:30.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.045 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:30:30.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:30:30.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495153+0000", 2024-09-17T21:30:30.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:30:30.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495155+0000", 2024-09-17T21:30:30.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1849999999999998e-06 2024-09-17T21:30:30.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495162+0000", 2024-09-17T21:30:30.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.64e-06 2024-09-17T21:30:30.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495166+0000", 2024-09-17T21:30:30.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:30:30.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530020+0000", 2024-09-17T21:30:30.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.034853824999999998 2024-09-17T21:30:30.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530076+0000", 2024-09-17T21:30:30.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5235999999999998e-05 2024-09-17T21:30:30.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530744+0000", 2024-09-17T21:30:30.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066835500000000003 2024-09-17T21:30:30.048 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.048 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.048 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.048 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50806 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19398656~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.048 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:30.049 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536338503, 2024-09-17T21:30:30.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036408459999999997, 2024-09-17T21:30:30.049 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.049 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.049 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.049 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.049 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.049 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.049 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50806 2024-09-17T21:30:30.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.049 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:30.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:30.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:30.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.72e-07 2024-09-17T21:30:30.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495175+0000", 2024-09-17T21:30:30.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:30:30.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495182+0000", 2024-09-17T21:30:30.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8079999999999999e-06 2024-09-17T21:30:30.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495186+0000", 2024-09-17T21:30:30.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5549999999999996e-06 2024-09-17T21:30:30.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530791+0000", 2024-09-17T21:30:30.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035604963000000003 2024-09-17T21:30:30.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530855+0000", 2024-09-17T21:30:30.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3286000000000004e-05 2024-09-17T21:30:30.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531581+0000", 2024-09-17T21:30:30.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00072655699999999996 2024-09-17T21:30:30.053 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.053 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.053 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.053 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50807 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23068672~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.053 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:30.053 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536319618, 2024-09-17T21:30:30.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037183314000000002, 2024-09-17T21:30:30.053 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.053 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.053 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.053 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.053 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.054 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.054 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50807 2024-09-17T21:30:30.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.054 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:30.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:30.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:30.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9999999999999999e-07 2024-09-17T21:30:30.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495194+0000", 2024-09-17T21:30:30.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.779e-06 2024-09-17T21:30:30.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495199+0000", 2024-09-17T21:30:30.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.468e-06 2024-09-17T21:30:30.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495203+0000", 2024-09-17T21:30:30.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0010000000000003e-06 2024-09-17T21:30:30.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531636+0000", 2024-09-17T21:30:30.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036432665000000003 2024-09-17T21:30:30.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531727+0000", 2024-09-17T21:30:30.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1607999999999994e-05 2024-09-17T21:30:30.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532375+0000", 2024-09-17T21:30:30.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00064749299999999996 2024-09-17T21:30:30.057 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.057 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.057 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.057 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50808 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24641536~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.057 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:30:30.057 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536302875000001, 2024-09-17T21:30:30.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037870190999999997, 2024-09-17T21:30:30.058 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.058 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.058 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.058 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.058 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.058 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.058 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50808 2024-09-17T21:30:30.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.058 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:30:30.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:30:30.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495209+0000", 2024-09-17T21:30:30.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:30:30.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495215+0000", 2024-09-17T21:30:30.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1630000000000001e-06 2024-09-17T21:30:30.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495221+0000", 2024-09-17T21:30:30.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-17T21:30:30.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495226+0000", 2024-09-17T21:30:30.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8189999999999999e-06 2024-09-17T21:30:30.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532425+0000", 2024-09-17T21:30:30.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037199135000000001 2024-09-17T21:30:30.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532531+0000", 2024-09-17T21:30:30.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106314 2024-09-17T21:30:30.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533079+0000", 2024-09-17T21:30:30.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00054739700000000003 2024-09-17T21:30:30.061 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.061 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.061 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.061 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50809 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28311552~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.061 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:30:30.062 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536279909000001, 2024-09-17T21:30:30.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038597605, 2024-09-17T21:30:30.062 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.062 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.062 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.062 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50809 2024-09-17T21:30:30.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.062 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:30:30.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:30:30.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495232+0000", 2024-09-17T21:30:30.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.39e-07 2024-09-17T21:30:30.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495234+0000", 2024-09-17T21:30:30.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T21:30:30.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495239+0000", 2024-09-17T21:30:30.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7969999999999997e-06 2024-09-17T21:30:30.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495244+0000", 2024-09-17T21:30:30.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999996e-06 2024-09-17T21:30:30.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533125+0000", 2024-09-17T21:30:30.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037881523 2024-09-17T21:30:30.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533237+0000", 2024-09-17T21:30:30.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000111879 2024-09-17T21:30:30.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533829+0000", 2024-09-17T21:30:30.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000591933 2024-09-17T21:30:30.065 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.066 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.066 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.066 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50810 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 29884416~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.066 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:30:30.066 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536261991, 2024-09-17T21:30:30.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039149809000000001, 2024-09-17T21:30:30.066 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.066 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.066 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.066 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.066 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.066 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.067 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50810 2024-09-17T21:30:30.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.067 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:30:30.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:30:30.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495250+0000", 2024-09-17T21:30:30.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:30:30.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495251+0000", 2024-09-17T21:30:30.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.81e-06 2024-09-17T21:30:30.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495257+0000", 2024-09-17T21:30:30.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1399999999999999e-06 2024-09-17T21:30:30.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495261+0000", 2024-09-17T21:30:30.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4229999999999998e-06 2024-09-17T21:30:30.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533879+0000", 2024-09-17T21:30:30.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038618292999999998 2024-09-17T21:30:30.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533939+0000", 2024-09-17T21:30:30.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.012e-05 2024-09-17T21:30:30.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534399+0000", 2024-09-17T21:30:30.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00045968099999999998 2024-09-17T21:30:30.070 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.070 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.070 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.070 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50850 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 2359296~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.070 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:30.071 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536243616, 2024-09-17T21:30:30.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039731491000000001, 2024-09-17T21:30:30.071 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.071 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.071 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.071 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.071 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.071 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.071 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50850 2024-09-17T21:30:30.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.071 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:30.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:30.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:30.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3200000000000001e-07 2024-09-17T21:30:30.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495270+0000", 2024-09-17T21:30:30.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.694e-06 2024-09-17T21:30:30.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495275+0000", 2024-09-17T21:30:30.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5890000000000002e-06 2024-09-17T21:30:30.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495279+0000", 2024-09-17T21:30:30.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-17T21:30:30.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534440+0000", 2024-09-17T21:30:30.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039160805999999999 2024-09-17T21:30:30.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534526+0000", 2024-09-17T21:30:30.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5327999999999993e-05 2024-09-17T21:30:30.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534999+0000", 2024-09-17T21:30:30.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00047353400000000001 2024-09-17T21:30:30.075 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.075 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.075 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.075 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50851 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 3932160~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.075 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:30:30.075 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536226004, 2024-09-17T21:30:30.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040211991000000002, 2024-09-17T21:30:30.075 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.075 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.076 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.076 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50851 2024-09-17T21:30:30.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.076 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:30:30.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:30:30.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495286+0000", 2024-09-17T21:30:30.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:30.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495287+0000", 2024-09-17T21:30:30.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6640000000000001e-06 2024-09-17T21:30:30.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495293+0000", 2024-09-17T21:30:30.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6710000000000004e-06 2024-09-17T21:30:30.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495298+0000", 2024-09-17T21:30:30.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6700000000000002e-06 2024-09-17T21:30:30.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535017+0000", 2024-09-17T21:30:30.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039719059000000001 2024-09-17T21:30:30.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535067+0000", 2024-09-17T21:30:30.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0046000000000002e-05 2024-09-17T21:30:30.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535497+0000", 2024-09-17T21:30:30.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043050299999999999 2024-09-17T21:30:30.079 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.079 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.079 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.080 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50852 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 7602176~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.080 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:30:30.080 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536207087000001, 2024-09-17T21:30:30.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041166337999999997, 2024-09-17T21:30:30.080 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.080 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.080 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.080 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.080 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.080 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.080 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50852 2024-09-17T21:30:30.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.081 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:30:30.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:30:30.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495305+0000", 2024-09-17T21:30:30.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1600000000000002e-07 2024-09-17T21:30:30.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495307+0000", 2024-09-17T21:30:30.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0980000000000001e-06 2024-09-17T21:30:30.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495312+0000", 2024-09-17T21:30:30.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6210000000000001e-06 2024-09-17T21:30:30.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495317+0000", 2024-09-17T21:30:30.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T21:30:30.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535531+0000", 2024-09-17T21:30:30.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040214580999999999 2024-09-17T21:30:30.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535618+0000", 2024-09-17T21:30:30.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6648000000000001e-05 2024-09-17T21:30:30.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536471+0000", 2024-09-17T21:30:30.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00085251599999999999 2024-09-17T21:30:30.084 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.084 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.084 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.084 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50853 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 9175040~204800] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.084 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:30.084 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536188756000001, 2024-09-17T21:30:30.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041673496999999997, 2024-09-17T21:30:30.084 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.084 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.085 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.085 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.085 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.085 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.085 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50853 2024-09-17T21:30:30.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.085 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:30.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:30.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:30.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:30:30.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495334+0000", 2024-09-17T21:30:30.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.076e-05 2024-09-17T21:30:30.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495340+0000", 2024-09-17T21:30:30.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:30:30.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495345+0000", 2024-09-17T21:30:30.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:30:30.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536514+0000", 2024-09-17T21:30:30.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041168534999999999 2024-09-17T21:30:30.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536572+0000", 2024-09-17T21:30:30.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8742000000000001e-05 2024-09-17T21:30:30.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536996+0000", 2024-09-17T21:30:30.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042368899999999999 2024-09-17T21:30:30.088 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.088 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.088 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.089 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50854 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 12845056~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.089 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:30.089 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536159696999999, 2024-09-17T21:30:30.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042777397000000002, 2024-09-17T21:30:30.089 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.089 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.089 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.089 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.089 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.089 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.089 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50854 2024-09-17T21:30:30.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.089 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:30.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:30.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:30.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.82e-07 2024-09-17T21:30:30.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495354+0000", 2024-09-17T21:30:30.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.091e-06 2024-09-17T21:30:30.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495360+0000", 2024-09-17T21:30:30.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.716e-06 2024-09-17T21:30:30.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495365+0000", 2024-09-17T21:30:30.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2939999999999996e-06 2024-09-17T21:30:30.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537020+0000", 2024-09-17T21:30:30.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041654419999999998 2024-09-17T21:30:30.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537105+0000", 2024-09-17T21:30:30.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5857000000000001e-05 2024-09-17T21:30:30.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538129+0000", 2024-09-17T21:30:30.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010235369999999999 2024-09-17T21:30:30.093 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.093 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.093 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.093 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50855 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 14417920~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.093 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:30.093 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536139731999999, 2024-09-17T21:30:30.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043254325000000003, 2024-09-17T21:30:30.093 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.093 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.093 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.094 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.094 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.094 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.094 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50855 2024-09-17T21:30:30.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.094 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:30.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:30.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:30.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:30:30.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495374+0000", 2024-09-17T21:30:30.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.922e-06 2024-09-17T21:30:30.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495379+0000", 2024-09-17T21:30:30.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-17T21:30:30.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495384+0000", 2024-09-17T21:30:30.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-17T21:30:30.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538167+0000", 2024-09-17T21:30:30.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042782845 2024-09-17T21:30:30.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538218+0000", 2024-09-17T21:30:30.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1335000000000002e-05 2024-09-17T21:30:30.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538626+0000", 2024-09-17T21:30:30.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000407656 2024-09-17T21:30:30.097 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.097 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.098 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.098 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50856 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 18087936~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.098 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:30.098 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536121276999999, 2024-09-17T21:30:30.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043911444000000001, 2024-09-17T21:30:30.098 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.098 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.098 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.098 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.098 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.098 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.099 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50856 2024-09-17T21:30:30.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.099 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:30.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:30.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:30.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0499999999999999e-07 2024-09-17T21:30:30.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495392+0000", 2024-09-17T21:30:30.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7430000000000001e-06 2024-09-17T21:30:30.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495398+0000", 2024-09-17T21:30:30.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3040000000000001e-06 2024-09-17T21:30:30.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495403+0000", 2024-09-17T21:30:30.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5600000000000004e-06 2024-09-17T21:30:30.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538660+0000", 2024-09-17T21:30:30.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043256832000000002 2024-09-17T21:30:30.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538746+0000", 2024-09-17T21:30:30.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6448999999999999e-05 2024-09-17T21:30:30.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539301+0000", 2024-09-17T21:30:30.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055525099999999999 2024-09-17T21:30:30.102 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.102 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.102 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.103 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50857 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19660800~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.103 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:30:30.103 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536101872, 2024-09-17T21:30:30.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044408156999999997, 2024-09-17T21:30:30.103 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.103 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.103 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.103 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.103 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.103 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.103 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50857 2024-09-17T21:30:30.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.103 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:30:30.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:30:30.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495410+0000", 2024-09-17T21:30:30.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:30:30.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495412+0000", 2024-09-17T21:30:30.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9580000000000001e-06 2024-09-17T21:30:30.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495418+0000", 2024-09-17T21:30:30.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1909999999999997e-06 2024-09-17T21:30:30.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495423+0000", 2024-09-17T21:30:30.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2429999999999996e-06 2024-09-17T21:30:30.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539338+0000", 2024-09-17T21:30:30.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043914478 2024-09-17T21:30:30.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539393+0000", 2024-09-17T21:30:30.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5226999999999998e-05 2024-09-17T21:30:30.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539818+0000", 2024-09-17T21:30:30.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042469300000000002 2024-09-17T21:30:30.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.107 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.107 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50858 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23330816~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.107 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:30.107 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536080774999999, 2024-09-17T21:30:30.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045010798999999997, 2024-09-17T21:30:30.107 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.107 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.107 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.107 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.107 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.108 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.108 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50858 2024-09-17T21:30:30.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.108 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:30.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:30.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:30.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:30:30.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495433+0000", 2024-09-17T21:30:30.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.787e-06 2024-09-17T21:30:30.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495438+0000", 2024-09-17T21:30:30.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:30:30.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495442+0000", 2024-09-17T21:30:30.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2640000000000002e-06 2024-09-17T21:30:30.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539852+0000", 2024-09-17T21:30:30.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044409592999999997 2024-09-17T21:30:30.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539940+0000", 2024-09-17T21:30:30.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7696999999999997e-05 2024-09-17T21:30:30.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540441+0000", 2024-09-17T21:30:30.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050164299999999997 2024-09-17T21:30:30.111 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.111 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.111 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.111 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50859 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24903680~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.111 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:30:30.111 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536063097, 2024-09-17T21:30:30.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045571569999999999, 2024-09-17T21:30:30.111 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.112 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.112 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.112 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.112 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.112 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.112 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50859 2024-09-17T21:30:30.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.112 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:30:30.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:30:30.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495449+0000", 2024-09-17T21:30:30.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:30:30.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495450+0000", 2024-09-17T21:30:30.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7889999999999999e-06 2024-09-17T21:30:30.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495456+0000", 2024-09-17T21:30:30.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5679999999999999e-06 2024-09-17T21:30:30.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495460+0000", 2024-09-17T21:30:30.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4989999999999996e-06 2024-09-17T21:30:30.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540472+0000", 2024-09-17T21:30:30.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045011585999999999 2024-09-17T21:30:30.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540554+0000", 2024-09-17T21:30:30.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2443999999999999e-05 2024-09-17T21:30:30.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541020+0000", 2024-09-17T21:30:30.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00046534300000000001 2024-09-17T21:30:30.115 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.115 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.115 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.116 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50860 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28573696~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.116 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:30.116 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536044744000002, 2024-09-17T21:30:30.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046167899999999998, 2024-09-17T21:30:30.116 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.116 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.116 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.116 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.116 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.116 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.116 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50860 2024-09-17T21:30:30.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.116 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:30.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:30.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:30.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:30:30.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495469+0000", 2024-09-17T21:30:30.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.99e-06 2024-09-17T21:30:30.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495475+0000", 2024-09-17T21:30:30.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:30:30.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495480+0000", 2024-09-17T21:30:30.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:30:30.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541050+0000", 2024-09-17T21:30:30.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045569853 2024-09-17T21:30:30.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541138+0000", 2024-09-17T21:30:30.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7997000000000005e-05 2024-09-17T21:30:30.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541634+0000", 2024-09-17T21:30:30.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00049685500000000004 2024-09-17T21:30:30.119 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.120 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.120 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.120 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50861 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 30146560~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:30.120 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:30.120 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.536025566999999, 2024-09-17T21:30:30.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046690553000000003, 2024-09-17T21:30:30.120 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.120 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.120 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:30.120 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.120 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.120 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.121 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50861 2024-09-17T21:30:30.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.121 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:30.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:30.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:30.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:30:30.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495488+0000", 2024-09-17T21:30:30.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:30:30.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495495+0000", 2024-09-17T21:30:30.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:30:30.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495500+0000", 2024-09-17T21:30:30.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8620000000000002e-06 2024-09-17T21:30:30.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541687+0000", 2024-09-17T21:30:30.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046186434999999998 2024-09-17T21:30:30.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541741+0000", 2024-09-17T21:30:30.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4228999999999999e-05 2024-09-17T21:30:30.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:30.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.542176+0000", 2024-09-17T21:30:30.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043531100000000002 2024-09-17T21:30:30.124 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.124 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.124 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.124 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.124 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.124 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:30.126 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:30:30.126 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:30.123+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:30.226 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:30.223+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:30.248 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:30.249 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:30.249 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:30.249 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:30.258 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:30:30.327 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:30.324+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:30.334 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-17T21:30:30.403 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:30.403 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:30.404 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:30.404 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:30.404 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:30.412 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:30:30.456 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:30.453+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:30.517 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:30.517 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:30.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:30.518 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:30.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.518 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21342 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2082816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:30.518 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:30.518 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.837700224999999, 2024-09-17T21:30:30.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022083523000000001, 2024-09-17T21:30:30.518 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.518 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.518 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.518 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.518 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.518 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.518 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21342 2024-09-17T21:30:30.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.519 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:30.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:30.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:30:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676659+0000", 2024-09-17T21:30:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0950000000000001e-06 2024-09-17T21:30:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676662+0000", 2024-09-17T21:30:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.971e-06 2024-09-17T21:30:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676665+0000", 2024-09-17T21:30:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6189999999999998e-06 2024-09-17T21:30:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698568+0000", 2024-09-17T21:30:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021903549000000001 2024-09-17T21:30:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698668+0000", 2024-09-17T21:30:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9778999999999995e-05 2024-09-17T21:30:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698741+0000", 2024-09-17T21:30:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000006e-05 2024-09-17T21:30:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21343 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2095104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.837690287000001, 2024-09-17T21:30:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022504024000000001, 2024-09-17T21:30:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21343 2024-09-17T21:30:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:30:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676669+0000", 2024-09-17T21:30:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.003e-06 2024-09-17T21:30:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676672+0000", 2024-09-17T21:30:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9280000000000002e-06 2024-09-17T21:30:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676674+0000", 2024-09-17T21:30:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5469999999999999e-06 2024-09-17T21:30:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699015+0000", 2024-09-17T21:30:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022340563000000001 2024-09-17T21:30:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699102+0000", 2024-09-17T21:30:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7425999999999999e-05 2024-09-17T21:30:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699172+0000", 2024-09-17T21:30:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9346999999999999e-05 2024-09-17T21:30:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45576 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.809456486999999, 2024-09-17T21:30:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022024660000000001, 2024-09-17T21:30:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45576 2024-09-17T21:30:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704902+0000", 2024-09-17T21:30:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9100000000000004e-07 2024-09-17T21:30:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704905+0000", 2024-09-17T21:30:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6139999999999999e-06 2024-09-17T21:30:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704911+0000", 2024-09-17T21:30:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8619999999999997e-06 2024-09-17T21:30:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704917+0000", 2024-09-17T21:30:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:30:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726681+0000", 2024-09-17T21:30:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021764569000000001 2024-09-17T21:30:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726726+0000", 2024-09-17T21:30:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4218999999999998e-05 2024-09-17T21:30:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726926+0000", 2024-09-17T21:30:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000200466 2024-09-17T21:30:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45577 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.809432830999999, 2024-09-17T21:30:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022269694999999999, 2024-09-17T21:30:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45577 2024-09-17T21:30:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704926+0000", 2024-09-17T21:30:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:30:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704928+0000", 2024-09-17T21:30:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9989999999999998e-06 2024-09-17T21:30:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704934+0000", 2024-09-17T21:30:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:30:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704939+0000", 2024-09-17T21:30:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5909999999999998e-06 2024-09-17T21:30:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726954+0000", 2024-09-17T21:30:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022014885000000001 2024-09-17T21:30:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726995+0000", 2024-09-17T21:30:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1474000000000003e-05 2024-09-17T21:30:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727195+0000", 2024-09-17T21:30:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019968300000000001 2024-09-17T21:30:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45578 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.809411587, 2024-09-17T21:30:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022484006000000001, 2024-09-17T21:30:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45578 2024-09-17T21:30:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704947+0000", 2024-09-17T21:30:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1399999999999997e-07 2024-09-17T21:30:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704949+0000", 2024-09-17T21:30:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0959999999999999e-06 2024-09-17T21:30:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704956+0000", 2024-09-17T21:30:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9330000000000004e-06 2024-09-17T21:30:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704961+0000", 2024-09-17T21:30:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6759999999999998e-06 2024-09-17T21:30:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727213+0000", 2024-09-17T21:30:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022252568 2024-09-17T21:30:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727259+0000", 2024-09-17T21:30:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5238000000000003e-05 2024-09-17T21:30:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727430+0000", 2024-09-17T21:30:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000171881 2024-09-17T21:30:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45579 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.809389659000001, 2024-09-17T21:30:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022749080000000001, 2024-09-17T21:30:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45579 2024-09-17T21:30:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704969+0000", 2024-09-17T21:30:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:30:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704971+0000", 2024-09-17T21:30:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.097e-06 2024-09-17T21:30:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704977+0000", 2024-09-17T21:30:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6329999999999999e-06 2024-09-17T21:30:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704983+0000", 2024-09-17T21:30:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:30:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727457+0000", 2024-09-17T21:30:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022474521000000001 2024-09-17T21:30:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727498+0000", 2024-09-17T21:30:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0558e-05 2024-09-17T21:30:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727717+0000", 2024-09-17T21:30:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219805 2024-09-17T21:30:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45650 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 20480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.808266352, 2024-09-17T21:30:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021996219000000001, 2024-09-17T21:30:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45650 2024-09-17T21:30:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5199999999999997e-07 2024-09-17T21:30:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706095+0000", 2024-09-17T21:30:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6079999999999998e-06 2024-09-17T21:30:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706101+0000", 2024-09-17T21:30:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:30:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706106+0000", 2024-09-17T21:30:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T21:30:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728000+0000", 2024-09-17T21:30:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021893378000000002 2024-09-17T21:30:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728039+0000", 2024-09-17T21:30:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9249000000000003e-05 2024-09-17T21:30:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728088+0000", 2024-09-17T21:30:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8785000000000001e-05 2024-09-17T21:30:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45652 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.808243521000001, 2024-09-17T21:30:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022100465999999999, 2024-09-17T21:30:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45652 2024-09-17T21:30:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706115+0000", 2024-09-17T21:30:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:30:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706117+0000", 2024-09-17T21:30:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-06 2024-09-17T21:30:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706122+0000", 2024-09-17T21:30:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.288e-06 2024-09-17T21:30:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706136+0000", 2024-09-17T21:30:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3409e-05 2024-09-17T21:30:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728112+0000", 2024-09-17T21:30:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021976724999999999 2024-09-17T21:30:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728151+0000", 2024-09-17T21:30:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8318999999999997e-05 2024-09-17T21:30:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728215+0000", 2024-09-17T21:30:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4172999999999997e-05 2024-09-17T21:30:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45653 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.808214311, 2024-09-17T21:30:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022249517999999999, 2024-09-17T21:30:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45653 2024-09-17T21:30:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T21:30:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706147+0000", 2024-09-17T21:30:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4430000000000002e-06 2024-09-17T21:30:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706152+0000", 2024-09-17T21:30:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5350000000000004e-06 2024-09-17T21:30:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706157+0000", 2024-09-17T21:30:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8420000000000004e-06 2024-09-17T21:30:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728246+0000", 2024-09-17T21:30:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022089237000000001 2024-09-17T21:30:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728309+0000", 2024-09-17T21:30:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2356999999999999e-05 2024-09-17T21:30:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:30:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4596000000000001e-05 2024-09-17T21:30:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45657 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 36864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.808183047, 2024-09-17T21:30:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022218252000000001, 2024-09-17T21:30:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45657 2024-09-17T21:30:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8299999999999997e-07 2024-09-17T21:30:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706178+0000", 2024-09-17T21:30:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3700000000000002e-06 2024-09-17T21:30:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706184+0000", 2024-09-17T21:30:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.144e-06 2024-09-17T21:30:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706190+0000", 2024-09-17T21:30:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8590000000000001e-06 2024-09-17T21:30:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728240+0000", 2024-09-17T21:30:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022049804999999999 2024-09-17T21:30:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728307+0000", 2024-09-17T21:30:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7477999999999993e-05 2024-09-17T21:30:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:30:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6112999999999994e-05 2024-09-17T21:30:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45658 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.808159926999998, 2024-09-17T21:30:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022376455, 2024-09-17T21:30:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45658 2024-09-17T21:30:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:30:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706200+0000", 2024-09-17T21:30:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:30:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706206+0000", 2024-09-17T21:30:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7039999999999999e-06 2024-09-17T21:30:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706211+0000", 2024-09-17T21:30:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9429999999999999e-06 2024-09-17T21:30:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728418+0000", 2024-09-17T21:30:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022206760999999998 2024-09-17T21:30:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728497+0000", 2024-09-17T21:30:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9493999999999995e-05 2024-09-17T21:30:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728574+0000", 2024-09-17T21:30:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7180000000000003e-05 2024-09-17T21:30:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45707 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 212992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.806455861, 2024-09-17T21:30:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022235550999999999, 2024-09-17T21:30:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45707 2024-09-17T21:30:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:30:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707905+0000", 2024-09-17T21:30:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2120000000000002e-06 2024-09-17T21:30:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707911+0000", 2024-09-17T21:30:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1929999999999998e-06 2024-09-17T21:30:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707915+0000", 2024-09-17T21:30:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:30:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729897+0000", 2024-09-17T21:30:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021981186 2024-09-17T21:30:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729944+0000", 2024-09-17T21:30:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571000000000002e-05 2024-09-17T21:30:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730138+0000", 2024-09-17T21:30:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000193451 2024-09-17T21:30:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45708 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.806435729, 2024-09-17T21:30:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022533742999999998, 2024-09-17T21:30:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45708 2024-09-17T21:30:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707923+0000", 2024-09-17T21:30:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:30:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707932+0000", 2024-09-17T21:30:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6430000000000007e-06 2024-09-17T21:30:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707939+0000", 2024-09-17T21:30:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3539999999999996e-06 2024-09-17T21:30:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707944+0000", 2024-09-17T21:30:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3249999999999998e-06 2024-09-17T21:30:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730176+0000", 2024-09-17T21:30:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022232393 2024-09-17T21:30:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730263+0000", 2024-09-17T21:30:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6638999999999995e-05 2024-09-17T21:30:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730456+0000", 2024-09-17T21:30:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019304899999999999 2024-09-17T21:30:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45752 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.806180509000001, 2024-09-17T21:30:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022426878000000001, 2024-09-17T21:30:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45752 2024-09-17T21:30:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708178+0000", 2024-09-17T21:30:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:30:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708181+0000", 2024-09-17T21:30:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.909e-06 2024-09-17T21:30:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708187+0000", 2024-09-17T21:30:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4999999999999996e-06 2024-09-17T21:30:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708193+0000", 2024-09-17T21:30:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:30:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730484+0000", 2024-09-17T21:30:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022291765000000002 2024-09-17T21:30:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730541+0000", 2024-09-17T21:30:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6483000000000001e-05 2024-09-17T21:30:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730604+0000", 2024-09-17T21:30:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3429000000000005e-05 2024-09-17T21:30:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45761 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.806152599000001, 2024-09-17T21:30:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022614408999999999, 2024-09-17T21:30:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45761 2024-09-17T21:30:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708206+0000", 2024-09-17T21:30:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4700000000000001e-07 2024-09-17T21:30:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708208+0000", 2024-09-17T21:30:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1509999999999998e-06 2024-09-17T21:30:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708214+0000", 2024-09-17T21:30:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.49e-06 2024-09-17T21:30:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708219+0000", 2024-09-17T21:30:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8119999999999998e-06 2024-09-17T21:30:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730641+0000", 2024-09-17T21:30:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022422765000000001 2024-09-17T21:30:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730731+0000", 2024-09-17T21:30:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9598999999999994e-05 2024-09-17T21:30:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730820+0000", 2024-09-17T21:30:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8844999999999994e-05 2024-09-17T21:30:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45778 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.805408995000001, 2024-09-17T21:30:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022018544000000001, 2024-09-17T21:30:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45778 2024-09-17T21:30:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708950+0000", 2024-09-17T21:30:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-17T21:30:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709307+0000", 2024-09-17T21:30:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000356848 2024-09-17T21:30:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709316+0000", 2024-09-17T21:30:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1430000000000004e-06 2024-09-17T21:30:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709328+0000", 2024-09-17T21:30:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2629999999999999e-05 2024-09-17T21:30:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730861+0000", 2024-09-17T21:30:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021532057 2024-09-17T21:30:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730905+0000", 2024-09-17T21:30:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4907000000000002e-05 2024-09-17T21:30:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730968+0000", 2024-09-17T21:30:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2108999999999998e-05 2024-09-17T21:30:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45785 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.804930811999998, 2024-09-17T21:30:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022044113000000001, 2024-09-17T21:30:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45785 2024-09-17T21:30:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709428+0000", 2024-09-17T21:30:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:30:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709430+0000", 2024-09-17T21:30:30.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:30:30.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709451+0000", 2024-09-17T21:30:30.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1372000000000001e-05 2024-09-17T21:30:30.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709472+0000", 2024-09-17T21:30:30.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1007999999999999e-05 2024-09-17T21:30:30.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731293+0000", 2024-09-17T21:30:30.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021821320000000002 2024-09-17T21:30:30.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731378+0000", 2024-09-17T21:30:30.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5059e-05 2024-09-17T21:30:30.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731471+0000", 2024-09-17T21:30:30.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2867000000000004e-05 2024-09-17T21:30:30.591 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.592 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.592 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.592 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45786 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:30.592 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:30.592 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.804872015000001, 2024-09-17T21:30:30.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022181821000000001, 2024-09-17T21:30:30.592 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.592 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.592 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.592 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.592 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.592 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.593 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45786 2024-09-17T21:30:30.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.593 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:30.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:30.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709487+0000", 2024-09-17T21:30:30.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1299999999999999e-07 2024-09-17T21:30:30.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709491+0000", 2024-09-17T21:30:30.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2089999999999999e-06 2024-09-17T21:30:30.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709498+0000", 2024-09-17T21:30:30.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3980000000000002e-06 2024-09-17T21:30:30.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709504+0000", 2024-09-17T21:30:30.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7219999999999996e-06 2024-09-17T21:30:30.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731499+0000", 2024-09-17T21:30:30.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021995297 2024-09-17T21:30:30.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731591+0000", 2024-09-17T21:30:30.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1373999999999996e-05 2024-09-17T21:30:30.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731668+0000", 2024-09-17T21:30:30.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7107999999999994e-05 2024-09-17T21:30:30.596 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.596 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.596 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.596 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50202 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'139, mlcod=45'137)", 2024-09-17T21:30:30.596 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:30:30.596 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.390136146, 2024-09-17T21:30:30.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059659441000000001, 2024-09-17T21:30:30.596 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.596 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.596 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:30.596 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:30:30.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:30:30.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124224+0000", 2024-09-17T21:30:30.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5000000000000002e-06 2024-09-17T21:30:30.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147385+0000", 2024-09-17T21:30:30.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023160544000000002 2024-09-17T21:30:30.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147411+0000", 2024-09-17T21:30:30.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6095999999999999e-05 2024-09-17T21:30:30.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147438+0000", 2024-09-17T21:30:30.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6982999999999999e-05 2024-09-17T21:30:30.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147504+0000", 2024-09-17T21:30:30.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6156999999999997e-05 2024-09-17T21:30:30.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147531+0000", 2024-09-17T21:30:30.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6741999999999999e-05 2024-09-17T21:30:30.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:30.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183820+0000", 2024-09-17T21:30:30.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036289090000000003 2024-09-17T21:30:30.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183881+0000", 2024-09-17T21:30:30.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1328999999999995e-05 2024-09-17T21:30:30.600 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.600 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.600 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.600 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50203 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'141, mlcod=45'137)", 2024-09-17T21:30:30.600 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:30:30.600 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.352873403, 2024-09-17T21:30:30.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023138823999999999, 2024-09-17T21:30:30.600 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.601 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.601 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:30.601 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:30:30.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:30:30.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161486+0000", 2024-09-17T21:30:30.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5290000000000001e-06 2024-09-17T21:30:30.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161496+0000", 2024-09-17T21:30:30.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0063e-05 2024-09-17T21:30:30.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161514+0000", 2024-09-17T21:30:30.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7728e-05 2024-09-17T21:30:30.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161524+0000", 2024-09-17T21:30:30.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0339e-05 2024-09-17T21:30:30.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170872+0000", 2024-09-17T21:30:30.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093473640000000004 2024-09-17T21:30:30.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170899+0000", 2024-09-17T21:30:30.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7321000000000002e-05 2024-09-17T21:30:30.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:30.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184580+0000", 2024-09-17T21:30:30.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013680988999999999 2024-09-17T21:30:30.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184623+0000", 2024-09-17T21:30:30.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3491000000000001e-05 2024-09-17T21:30:30.604 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.604 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.604 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.604 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.604 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.604 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:30.605 INFO:teuthology.orchestra.run.smithi129.stdout:3410 clone 27 order 22 su 32768 sc 7 2024-09-17T21:30:30.606 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:30:30.606 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:30.603+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:30.673 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:30:30.718 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:30.716+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:30.752 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:30.752 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:30.752 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:30.752 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:30.763 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:30:30.768 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:30:30.788 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:70181 3.0 3:05504739:::rbd_object_map.11201b996d31:head [call rbd.object_map_resize in=29b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-17T21:30:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:30.785937+0000", 2024-09-17T21:30:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.000620062, 2024-09-17T21:30:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00063850099999999998, 2024-09-17T21:30:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:30:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 70181 2024-09-17T21:30:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:30.785937+0000", 2024-09-17T21:30:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:30.785937+0000", 2024-09-17T21:30:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:30.785938+0000", 2024-09-17T21:30:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5850000000000001e-06 2024-09-17T21:30:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:30.785946+0000", 2024-09-17T21:30:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9719999999999992e-06 2024-09-17T21:30:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:30.785960+0000", 2024-09-17T21:30:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3844e-05 2024-09-17T21:30:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:30.785968+0000", 2024-09-17T21:30:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4320000000000002e-06 2024-09-17T21:30:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:30.786022+0000", 2024-09-17T21:30:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4129999999999998e-05 2024-09-17T21:30:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:30.786107+0000", 2024-09-17T21:30:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5274999999999999e-05 2024-09-17T21:30:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:30.786217+0000", 2024-09-17T21:30:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000110281 2024-09-17T21:30:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.793 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:30.793 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:30:30.793 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 1 2024-09-17T21:30:30.793 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:30.806 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone26 from 0xb7806d7 (overlap 0xb7806d7) to 0xa526c8e 2024-09-17T21:30:30.807 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:30.804+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:30.869 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:30.869 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:30.869 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:30.869 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:30.869 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:30.878 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:30:30.921 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:30.918+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:30.983 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10799 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:30:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.914883834999998, 2024-09-17T21:30:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039122745, 2024-09-17T21:30:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10799 2024-09-17T21:30:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:30:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:30:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065469+0000", 2024-09-17T21:30:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:30:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065477+0000", 2024-09-17T21:30:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1319999999999994e-06 2024-09-17T21:30:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065483+0000", 2024-09-17T21:30:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2040000000000002e-06 2024-09-17T21:30:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065487+0000", 2024-09-17T21:30:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.42e-06 2024-09-17T21:30:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099984+0000", 2024-09-17T21:30:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034496484000000001 2024-09-17T21:30:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100065+0000", 2024-09-17T21:30:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1200999999999997e-05 2024-09-17T21:30:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100422+0000", 2024-09-17T21:30:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035657700000000001 2024-09-17T21:30:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104214+0000", 2024-09-17T21:30:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003792045 2024-09-17T21:30:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104505+0000", 2024-09-17T21:30:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029141399999999999 2024-09-17T21:30:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104556+0000", 2024-09-17T21:30:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1388999999999999e-05 2024-09-17T21:30:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104591+0000", 2024-09-17T21:30:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4455999999999997e-05 2024-09-17T21:30:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12187 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.840791103999997, 2024-09-17T21:30:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038825750999999999, 2024-09-17T21:30:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12187 2024-09-17T21:30:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:30:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139563+0000", 2024-09-17T21:30:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.624e-06 2024-09-17T21:30:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139569+0000", 2024-09-17T21:30:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1090000000000003e-06 2024-09-17T21:30:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139573+0000", 2024-09-17T21:30:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4429999999999996e-06 2024-09-17T21:30:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178206+0000", 2024-09-17T21:30:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038632376000000003 2024-09-17T21:30:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178282+0000", 2024-09-17T21:30:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6076999999999997e-05 2024-09-17T21:30:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178387+0000", 2024-09-17T21:30:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104682 2024-09-17T21:30:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12188 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:30:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.840772923000003, 2024-09-17T21:30:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039053826999999999, 2024-09-17T21:30:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12188 2024-09-17T21:30:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:30:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:30:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139746+0000", 2024-09-17T21:30:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000167281 2024-09-17T21:30:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139749+0000", 2024-09-17T21:30:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9660000000000001e-06 2024-09-17T21:30:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139757+0000", 2024-09-17T21:30:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8029999999999997e-06 2024-09-17T21:30:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139771+0000", 2024-09-17T21:30:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3541e-05 2024-09-17T21:30:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178415+0000", 2024-09-17T21:30:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038644803999999998 2024-09-17T21:30:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178517+0000", 2024-09-17T21:30:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101298 2024-09-17T21:30:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178633+0000", 2024-09-17T21:30:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116134 2024-09-17T21:30:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12189 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:30:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.840572446000003, 2024-09-17T21:30:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039135991000000002, 2024-09-17T21:30:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12189 2024-09-17T21:30:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:30:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:30:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139780+0000", 2024-09-17T21:30:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:30:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139782+0000", 2024-09-17T21:30:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1950000000000002e-06 2024-09-17T21:30:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139788+0000", 2024-09-17T21:30:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0639999999999999e-06 2024-09-17T21:30:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139792+0000", 2024-09-17T21:30:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1960000000000001e-06 2024-09-17T21:30:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178671+0000", 2024-09-17T21:30:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038878176 2024-09-17T21:30:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178778+0000", 2024-09-17T21:30:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107278 2024-09-17T21:30:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178915+0000", 2024-09-17T21:30:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013757900000000001 2024-09-17T21:30:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12190 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.840553262999997, 2024-09-17T21:30:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039343781000000001, 2024-09-17T21:30:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12190 2024-09-17T21:30:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139801+0000", 2024-09-17T21:30:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.916e-06 2024-09-17T21:30:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139807+0000", 2024-09-17T21:30:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4540000000000004e-06 2024-09-17T21:30:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139812+0000", 2024-09-17T21:30:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9729999999999997e-06 2024-09-17T21:30:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178960+0000", 2024-09-17T21:30:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039147567000000001 2024-09-17T21:30:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179051+0000", 2024-09-17T21:30:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0754000000000006e-05 2024-09-17T21:30:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179142+0000", 2024-09-17T21:30:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1712999999999994e-05 2024-09-17T21:30:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12191 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.840533405000002, 2024-09-17T21:30:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039464336000000003, 2024-09-17T21:30:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12191 2024-09-17T21:30:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:30:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139821+0000", 2024-09-17T21:30:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1260000000000001e-06 2024-09-17T21:30:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139827+0000", 2024-09-17T21:30:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T21:30:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139840+0000", 2024-09-17T21:30:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2607e-05 2024-09-17T21:30:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179162+0000", 2024-09-17T21:30:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039322863999999999 2024-09-17T21:30:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179204+0000", 2024-09-17T21:30:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1938999999999999e-05 2024-09-17T21:30:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179283+0000", 2024-09-17T21:30:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8547000000000005e-05 2024-09-17T21:30:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12192 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:30:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.840505692999997, 2024-09-17T21:30:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039652318999999998, 2024-09-17T21:30:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12192 2024-09-17T21:30:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:30:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:30:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141458+0000", 2024-09-17T21:30:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001612098 2024-09-17T21:30:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141469+0000", 2024-09-17T21:30:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-17T21:30:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141478+0000", 2024-09-17T21:30:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T21:30:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141485+0000", 2024-09-17T21:30:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4760000000000003e-06 2024-09-17T21:30:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179317+0000", 2024-09-17T21:30:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037831832000000003 2024-09-17T21:30:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179395+0000", 2024-09-17T21:30:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8349000000000005e-05 2024-09-17T21:30:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179499+0000", 2024-09-17T21:30:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103628 2024-09-17T21:30:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12257 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.838316884999998, 2024-09-17T21:30:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038862751000000001, 2024-09-17T21:30:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12257 2024-09-17T21:30:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T21:30:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142037+0000", 2024-09-17T21:30:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.03e-06 2024-09-17T21:30:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142043+0000", 2024-09-17T21:30:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6319999999999996e-06 2024-09-17T21:30:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142052+0000", 2024-09-17T21:30:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2709999999999998e-06 2024-09-17T21:30:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180794+0000", 2024-09-17T21:30:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038741758000000001 2024-09-17T21:30:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180843+0000", 2024-09-17T21:30:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9021999999999999e-05 2024-09-17T21:30:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180898+0000", 2024-09-17T21:30:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673000000000003e-05 2024-09-17T21:30:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12258 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.838293368000002, 2024-09-17T21:30:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038950717000000003, 2024-09-17T21:30:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12258 2024-09-17T21:30:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199999999999999e-07 2024-09-17T21:30:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142061+0000", 2024-09-17T21:30:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8279999999999999e-06 2024-09-17T21:30:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142067+0000", 2024-09-17T21:30:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5600000000000001e-06 2024-09-17T21:30:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142071+0000", 2024-09-17T21:30:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6240000000000001e-06 2024-09-17T21:30:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180921+0000", 2024-09-17T21:30:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038849366000000003 2024-09-17T21:30:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180961+0000", 2024-09-17T21:30:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0219000000000002e-05 2024-09-17T21:30:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181009+0000", 2024-09-17T21:30:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8437999999999999e-05 2024-09-17T21:30:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12259 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.838266120999997, 2024-09-17T21:30:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039116832999999997, 2024-09-17T21:30:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12259 2024-09-17T21:30:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2200000000000004e-07 2024-09-17T21:30:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142088+0000", 2024-09-17T21:30:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0329999999999998e-06 2024-09-17T21:30:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142095+0000", 2024-09-17T21:30:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4320000000000004e-06 2024-09-17T21:30:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142100+0000", 2024-09-17T21:30:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:30:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181037+0000", 2024-09-17T21:30:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038937601000000002 2024-09-17T21:30:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181114+0000", 2024-09-17T21:30:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6857e-05 2024-09-17T21:30:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181203+0000", 2024-09-17T21:30:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8437999999999996e-05 2024-09-17T21:30:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12260 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:30:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.838245833999999, 2024-09-17T21:30:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039204184000000003, 2024-09-17T21:30:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12260 2024-09-17T21:30:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:30:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:30:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142107+0000", 2024-09-17T21:30:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:30:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142108+0000", 2024-09-17T21:30:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.753e-06 2024-09-17T21:30:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142114+0000", 2024-09-17T21:30:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7660000000000004e-06 2024-09-17T21:30:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142119+0000", 2024-09-17T21:30:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:30:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181216+0000", 2024-09-17T21:30:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039097191000000003 2024-09-17T21:30:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181260+0000", 2024-09-17T21:30:31.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4731000000000003e-05 2024-09-17T21:30:31.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181310+0000", 2024-09-17T21:30:31.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9818000000000003e-05 2024-09-17T21:30:31.036 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.036 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.036 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.037 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12261 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.037 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:30:31.037 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.838227510000003, 2024-09-17T21:30:31.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039292200999999999, 2024-09-17T21:30:31.037 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.037 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.037 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.037 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.037 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.037 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.037 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12261 2024-09-17T21:30:31.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.037 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:30:31.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:30:31.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142125+0000", 2024-09-17T21:30:31.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:30:31.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142127+0000", 2024-09-17T21:30:31.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.792e-06 2024-09-17T21:30:31.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142132+0000", 2024-09-17T21:30:31.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3979999999999998e-06 2024-09-17T21:30:31.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142136+0000", 2024-09-17T21:30:31.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.211e-06 2024-09-17T21:30:31.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181324+0000", 2024-09-17T21:30:31.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039187560000000003 2024-09-17T21:30:31.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181363+0000", 2024-09-17T21:30:31.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9375000000000002e-05 2024-09-17T21:30:31.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181417+0000", 2024-09-17T21:30:31.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3372000000000003e-05 2024-09-17T21:30:31.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.041 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.041 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12262 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.042 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:30:31.042 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.838209716999998, 2024-09-17T21:30:31.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039384875, 2024-09-17T21:30:31.042 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.042 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.042 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.042 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.042 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.042 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.042 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12262 2024-09-17T21:30:31.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.042 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:30:31.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:30:31.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142143+0000", 2024-09-17T21:30:31.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:30:31.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142148+0000", 2024-09-17T21:30:31.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4360000000000001e-06 2024-09-17T21:30:31.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142154+0000", 2024-09-17T21:30:31.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8719999999999999e-06 2024-09-17T21:30:31.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142158+0000", 2024-09-17T21:30:31.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.476e-06 2024-09-17T21:30:31.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181436+0000", 2024-09-17T21:30:31.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039277919000000001 2024-09-17T21:30:31.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181480+0000", 2024-09-17T21:30:31.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3649999999999997e-05 2024-09-17T21:30:31.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181527+0000", 2024-09-17T21:30:31.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7131e-05 2024-09-17T21:30:31.046 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.046 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.046 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.046 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12263 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1079296~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.046 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:31.046 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.838187282, 2024-09-17T21:30:31.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039473339000000003, 2024-09-17T21:30:31.047 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.047 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.047 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.047 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.047 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.047 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.047 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12263 2024-09-17T21:30:31.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.047 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:31.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:31.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:31.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:30:31.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142167+0000", 2024-09-17T21:30:31.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.81e-06 2024-09-17T21:30:31.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142172+0000", 2024-09-17T21:30:31.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:30:31.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142177+0000", 2024-09-17T21:30:31.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T21:30:31.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181553+0000", 2024-09-17T21:30:31.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039376130000000002 2024-09-17T21:30:31.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181592+0000", 2024-09-17T21:30:31.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9153999999999998e-05 2024-09-17T21:30:31.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181638+0000", 2024-09-17T21:30:31.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5864e-05 2024-09-17T21:30:31.051 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.051 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.051 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.051 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12278 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1095680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.051 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:31.051 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.838169307000001, 2024-09-17T21:30:31.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039454222999999997, 2024-09-17T21:30:31.051 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.051 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.052 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.052 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.052 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.052 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.052 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12278 2024-09-17T21:30:31.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.052 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:31.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:31.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:31.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:30:31.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142185+0000", 2024-09-17T21:30:31.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.697e-06 2024-09-17T21:30:31.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142191+0000", 2024-09-17T21:30:31.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:30:31.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142196+0000", 2024-09-17T21:30:31.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4829999999999999e-06 2024-09-17T21:30:31.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181548+0000", 2024-09-17T21:30:31.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039351200000000003 2024-09-17T21:30:31.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181591+0000", 2024-09-17T21:30:31.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3492000000000003e-05 2024-09-17T21:30:31.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181637+0000", 2024-09-17T21:30:31.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5847000000000003e-05 2024-09-17T21:30:31.056 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.056 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.056 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.056 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12279 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.056 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:31.056 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.838146018000003, 2024-09-17T21:30:31.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039539801999999999, 2024-09-17T21:30:31.056 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.056 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.056 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.056 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.057 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.057 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.057 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12279 2024-09-17T21:30:31.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.057 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:31.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:31.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:31.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.89e-07 2024-09-17T21:30:31.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142208+0000", 2024-09-17T21:30:31.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7579999999999999e-06 2024-09-17T21:30:31.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142214+0000", 2024-09-17T21:30:31.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:30:31.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142218+0000", 2024-09-17T21:30:31.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3429999999999998e-06 2024-09-17T21:30:31.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181653+0000", 2024-09-17T21:30:31.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039435322000000002 2024-09-17T21:30:31.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181698+0000", 2024-09-17T21:30:31.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4978999999999997e-05 2024-09-17T21:30:31.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181746+0000", 2024-09-17T21:30:31.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571999999999998e-05 2024-09-17T21:30:31.060 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.060 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.060 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.061 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12280 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1124352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.061 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:30:31.061 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.838128499, 2024-09-17T21:30:31.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039704358000000002, 2024-09-17T21:30:31.061 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.061 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.061 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.061 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.061 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.061 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.061 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12280 2024-09-17T21:30:31.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.062 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:30:31.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:30:31.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142224+0000", 2024-09-17T21:30:31.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:30:31.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142226+0000", 2024-09-17T21:30:31.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:30:31.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142231+0000", 2024-09-17T21:30:31.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:30:31.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142240+0000", 2024-09-17T21:30:31.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4590000000000004e-06 2024-09-17T21:30:31.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181774+0000", 2024-09-17T21:30:31.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039533833999999997 2024-09-17T21:30:31.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181859+0000", 2024-09-17T21:30:31.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4328999999999998e-05 2024-09-17T21:30:31.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181928+0000", 2024-09-17T21:30:31.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9139000000000003e-05 2024-09-17T21:30:31.065 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.065 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.065 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.065 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12281 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1136640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.066 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:30:31.066 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.838105593000002, 2024-09-17T21:30:31.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039787799999999998, 2024-09-17T21:30:31.066 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.066 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.066 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.066 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.066 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.066 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.066 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12281 2024-09-17T21:30:31.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.066 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:30:31.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:30:31.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142309+0000", 2024-09-17T21:30:31.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2323999999999995e-05 2024-09-17T21:30:31.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142317+0000", 2024-09-17T21:30:31.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1859999999999992e-06 2024-09-17T21:30:31.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142324+0000", 2024-09-17T21:30:31.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4699999999999996e-06 2024-09-17T21:30:31.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142330+0000", 2024-09-17T21:30:31.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4650000000000002e-06 2024-09-17T21:30:31.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181948+0000", 2024-09-17T21:30:31.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039618056999999998 2024-09-17T21:30:31.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181986+0000", 2024-09-17T21:30:31.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8442999999999998e-05 2024-09-17T21:30:31.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182034+0000", 2024-09-17T21:30:31.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7855000000000002e-05 2024-09-17T21:30:31.069 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.069 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.070 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.070 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12282 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1150976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.070 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:31.070 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.838013363999998, 2024-09-17T21:30:31.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039871919999999998, 2024-09-17T21:30:31.070 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.070 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.070 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.070 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.070 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.070 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.070 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12282 2024-09-17T21:30:31.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.071 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:31.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:31.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:31.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:30:31.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142341+0000", 2024-09-17T21:30:31.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.232e-06 2024-09-17T21:30:31.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142348+0000", 2024-09-17T21:30:31.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7320000000000001e-06 2024-09-17T21:30:31.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142353+0000", 2024-09-17T21:30:31.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:30:31.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182062+0000", 2024-09-17T21:30:31.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039709739000000001 2024-09-17T21:30:31.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182128+0000", 2024-09-17T21:30:31.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5739000000000002e-05 2024-09-17T21:30:31.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182210+0000", 2024-09-17T21:30:31.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:30:31.074 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.074 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12283 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:30:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.837992952999997, 2024-09-17T21:30:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039896434000000001, 2024-09-17T21:30:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:30:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12283 2024-09-17T21:30:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:30:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:30:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144852+0000", 2024-09-17T21:30:31.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024934079999999999 2024-09-17T21:30:31.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144856+0000", 2024-09-17T21:30:31.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:30:31.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144864+0000", 2024-09-17T21:30:31.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0160000000000008e-06 2024-09-17T21:30:31.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144869+0000", 2024-09-17T21:30:31.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:30:31.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182246+0000", 2024-09-17T21:30:31.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037377207000000003 2024-09-17T21:30:31.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182255+0000", 2024-09-17T21:30:31.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1840000000000005e-06 2024-09-17T21:30:31.079 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.079 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.079 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.079 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.079 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.079 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:31.081 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:30:31.081 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:31.078+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:31.181 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:31.179+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:31.225 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:31.226 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:31.226 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:31.226 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:31.234 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:30:31.282 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:31.279+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:31.338 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:31.338 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:31.338 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:31.338 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:31.338 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:31.346 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:30:31.390 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:31.387+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:31.450 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:31.451 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:31.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:31.451 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:31.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.451 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:36935 3.a 3:54c8759b:::rbd_data.1120e62eea89.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-17T21:30:31.451 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:31.451 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 34.488991360999997, 2024-09-17T21:30:31.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020540151999999999, 2024-09-17T21:30:31.451 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.451 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.451 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:31.451 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.451 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.452 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 36935 2024-09-17T21:30:31.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.452 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:31.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:31.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958910+0000", 2024-09-17T21:30:31.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:30:31.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958918+0000", 2024-09-17T21:30:31.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6469999999999998e-06 2024-09-17T21:30:31.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958933+0000", 2024-09-17T21:30:31.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4791e-05 2024-09-17T21:30:31.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958940+0000", 2024-09-17T21:30:31.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-17T21:30:31.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958990+0000", 2024-09-17T21:30:31.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9759000000000003e-05 2024-09-17T21:30:31.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.959039+0000", 2024-09-17T21:30:31.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9487000000000002e-05 2024-09-17T21:30:31.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:31.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.962098+0000", 2024-09-17T21:30:31.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003059258 2024-09-17T21:30:31.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:31.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979325+0000", 2024-09-17T21:30:31.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017226841 2024-09-17T21:30:31.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:31.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979343+0000", 2024-09-17T21:30:31.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7486000000000002e-05 2024-09-17T21:30:31.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:31.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979401+0000", 2024-09-17T21:30:31.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7980999999999999e-05 2024-09-17T21:30:31.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979449+0000", 2024-09-17T21:30:31.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8639e-05 2024-09-17T21:30:31.457 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.457 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.457 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.457 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39447 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 909312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.457 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:31.457 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.480053461000001, 2024-09-17T21:30:31.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017419153999999999, 2024-09-17T21:30:31.457 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.457 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.457 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.457 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.458 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.458 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.458 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39447 2024-09-17T21:30:31.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.458 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:31.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:31.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:31.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:30:31.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967849+0000", 2024-09-17T21:30:31.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4339999999999999e-06 2024-09-17T21:30:31.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967853+0000", 2024-09-17T21:30:31.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:30:31.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967857+0000", 2024-09-17T21:30:31.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:30:31.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985193+0000", 2024-09-17T21:30:31.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017335973000000001 2024-09-17T21:30:31.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985228+0000", 2024-09-17T21:30:31.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5154999999999998e-05 2024-09-17T21:30:31.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985266+0000", 2024-09-17T21:30:31.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8448000000000002e-05 2024-09-17T21:30:31.461 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.461 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.461 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.461 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39448 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 974848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.461 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:31.461 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.480040644999999, 2024-09-17T21:30:31.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017517066000000001, 2024-09-17T21:30:31.462 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.462 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.462 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.462 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.462 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.462 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.462 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39448 2024-09-17T21:30:31.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.462 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:31.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:31.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:31.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.72e-07 2024-09-17T21:30:31.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967861+0000", 2024-09-17T21:30:31.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.096e-06 2024-09-17T21:30:31.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967865+0000", 2024-09-17T21:30:31.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9339999999999999e-06 2024-09-17T21:30:31.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967868+0000", 2024-09-17T21:30:31.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.508e-06 2024-09-17T21:30:31.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985285+0000", 2024-09-17T21:30:31.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017417208 2024-09-17T21:30:31.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985339+0000", 2024-09-17T21:30:31.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3931000000000003e-05 2024-09-17T21:30:31.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985377+0000", 2024-09-17T21:30:31.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8016999999999999e-05 2024-09-17T21:30:31.465 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.466 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.466 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.466 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39449 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.466 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:31.466 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.480028534999999, 2024-09-17T21:30:31.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017595137, 2024-09-17T21:30:31.466 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.466 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.466 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.467 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39449 2024-09-17T21:30:31.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.467 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:31.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:31.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:31.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2700000000000001e-07 2024-09-17T21:30:31.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967873+0000", 2024-09-17T21:30:31.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.164e-06 2024-09-17T21:30:31.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967877+0000", 2024-09-17T21:30:31.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.23e-06 2024-09-17T21:30:31.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967880+0000", 2024-09-17T21:30:31.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.286e-06 2024-09-17T21:30:31.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985397+0000", 2024-09-17T21:30:31.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017516982 2024-09-17T21:30:31.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985431+0000", 2024-09-17T21:30:31.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3899999999999997e-05 2024-09-17T21:30:31.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985467+0000", 2024-09-17T21:30:31.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6347999999999998e-05 2024-09-17T21:30:31.470 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.470 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.470 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.470 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39450 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1105920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.470 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:31.470 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.480008278, 2024-09-17T21:30:31.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017677824000000002, 2024-09-17T21:30:31.471 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.471 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.471 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.471 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.471 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.471 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.471 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39450 2024-09-17T21:30:31.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.471 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:31.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:31.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967893+0000", 2024-09-17T21:30:31.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9100000000000004e-07 2024-09-17T21:30:31.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967895+0000", 2024-09-17T21:30:31.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:30:31.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967901+0000", 2024-09-17T21:30:31.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.083e-06 2024-09-17T21:30:31.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967906+0000", 2024-09-17T21:30:31.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:30:31.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985493+0000", 2024-09-17T21:30:31.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017587565999999999 2024-09-17T21:30:31.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985531+0000", 2024-09-17T21:30:31.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7490000000000002e-05 2024-09-17T21:30:31.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985570+0000", 2024-09-17T21:30:31.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9205000000000002e-05 2024-09-17T21:30:31.474 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.474 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.474 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.475 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39451 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1171456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.475 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:31.475 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.479987898000001, 2024-09-17T21:30:31.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017752337999999999, 2024-09-17T21:30:31.475 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.475 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.475 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.475 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.475 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.475 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.475 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39451 2024-09-17T21:30:31.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.476 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:31.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:31.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:31.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:30:31.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967915+0000", 2024-09-17T21:30:31.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780000000000001e-06 2024-09-17T21:30:31.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967922+0000", 2024-09-17T21:30:31.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7020000000000004e-06 2024-09-17T21:30:31.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967926+0000", 2024-09-17T21:30:31.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.809e-06 2024-09-17T21:30:31.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985590+0000", 2024-09-17T21:30:31.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017663913999999999 2024-09-17T21:30:31.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985627+0000", 2024-09-17T21:30:31.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6896e-05 2024-09-17T21:30:31.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985665+0000", 2024-09-17T21:30:31.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7793999999999998e-05 2024-09-17T21:30:31.479 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.479 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.479 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.479 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39452 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1236992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.479 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:31.479 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.479967955999999, 2024-09-17T21:30:31.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017829602999999999, 2024-09-17T21:30:31.480 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.480 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.480 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.480 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.480 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.480 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.480 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39452 2024-09-17T21:30:31.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.480 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:31.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:31.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:31.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:30:31.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967947+0000", 2024-09-17T21:30:31.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4280999999999999e-05 2024-09-17T21:30:31.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967954+0000", 2024-09-17T21:30:31.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3249999999999996e-06 2024-09-17T21:30:31.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967959+0000", 2024-09-17T21:30:31.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2299999999999999e-06 2024-09-17T21:30:31.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985687+0000", 2024-09-17T21:30:31.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017727844999999999 2024-09-17T21:30:31.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985725+0000", 2024-09-17T21:30:31.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8243e-05 2024-09-17T21:30:31.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985762+0000", 2024-09-17T21:30:31.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7023000000000001e-05 2024-09-17T21:30:31.484 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.484 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.484 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.484 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39453 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.484 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:31.484 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.479931468, 2024-09-17T21:30:31.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017893793000000002, 2024-09-17T21:30:31.484 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.484 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.485 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.485 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39453 2024-09-17T21:30:31.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.485 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:31.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:31.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967970+0000", 2024-09-17T21:30:31.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:30:31.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967972+0000", 2024-09-17T21:30:31.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0229999999999999e-06 2024-09-17T21:30:31.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967978+0000", 2024-09-17T21:30:31.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0430000000000004e-06 2024-09-17T21:30:31.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967982+0000", 2024-09-17T21:30:31.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2570000000000001e-06 2024-09-17T21:30:31.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985782+0000", 2024-09-17T21:30:31.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017799683 2024-09-17T21:30:31.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985818+0000", 2024-09-17T21:30:31.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6304999999999999e-05 2024-09-17T21:30:31.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985863+0000", 2024-09-17T21:30:31.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4953999999999997e-05 2024-09-17T21:30:31.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.488 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.489 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39454 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1368064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.489 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:31.489 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.47991214, 2024-09-17T21:30:31.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017968886, 2024-09-17T21:30:31.489 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.489 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.489 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.489 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.489 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.489 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.489 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39454 2024-09-17T21:30:31.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.489 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:31.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:31.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967989+0000", 2024-09-17T21:30:31.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.15e-07 2024-09-17T21:30:31.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967991+0000", 2024-09-17T21:30:31.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.945e-06 2024-09-17T21:30:31.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967997+0000", 2024-09-17T21:30:31.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0190000000000003e-06 2024-09-17T21:30:31.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968001+0000", 2024-09-17T21:30:31.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6650000000000002e-06 2024-09-17T21:30:31.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985883+0000", 2024-09-17T21:30:31.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017881971999999999 2024-09-17T21:30:31.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985920+0000", 2024-09-17T21:30:31.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-05 2024-09-17T21:30:31.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985957+0000", 2024-09-17T21:30:31.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7370000000000003e-05 2024-09-17T21:30:31.493 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.493 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.493 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.493 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39455 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.493 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:31.493 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.479892966000001, 2024-09-17T21:30:31.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018044437999999999, 2024-09-17T21:30:31.493 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.493 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.493 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.494 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.494 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.494 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39455 2024-09-17T21:30:31.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.494 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:31.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:31.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:31.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:30:31.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968010+0000", 2024-09-17T21:30:31.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.886e-06 2024-09-17T21:30:31.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968017+0000", 2024-09-17T21:30:31.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:30:31.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968021+0000", 2024-09-17T21:30:31.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850000000000004e-06 2024-09-17T21:30:31.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985977+0000", 2024-09-17T21:30:31.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017956368 2024-09-17T21:30:31.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986015+0000", 2024-09-17T21:30:31.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7610000000000001e-05 2024-09-17T21:30:31.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986052+0000", 2024-09-17T21:30:31.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7109e-05 2024-09-17T21:30:31.497 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.497 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.497 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.497 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39456 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1499136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.498 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:31.498 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.479872665999999, 2024-09-17T21:30:31.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018114514000000002, 2024-09-17T21:30:31.498 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.498 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.498 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.498 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39456 2024-09-17T21:30:31.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.498 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:31.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:31.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:31.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-17T21:30:31.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968030+0000", 2024-09-17T21:30:31.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0049999999999999e-06 2024-09-17T21:30:31.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968036+0000", 2024-09-17T21:30:31.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0129999999999999e-06 2024-09-17T21:30:31.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968041+0000", 2024-09-17T21:30:31.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8300000000000003e-06 2024-09-17T21:30:31.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986071+0000", 2024-09-17T21:30:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018029620999999999 2024-09-17T21:30:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986106+0000", 2024-09-17T21:30:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4867999999999999e-05 2024-09-17T21:30:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986142+0000", 2024-09-17T21:30:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6566000000000002e-05 2024-09-17T21:30:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39457 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1564672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.47985285, 2024-09-17T21:30:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018188644, 2024-09-17T21:30:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39457 2024-09-17T21:30:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7199999999999999e-07 2024-09-17T21:30:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968050+0000", 2024-09-17T21:30:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1110000000000002e-06 2024-09-17T21:30:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968057+0000", 2024-09-17T21:30:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6170000000000003e-06 2024-09-17T21:30:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968062+0000", 2024-09-17T21:30:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:30:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986163+0000", 2024-09-17T21:30:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018101435999999999 2024-09-17T21:30:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986199+0000", 2024-09-17T21:30:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.591e-05 2024-09-17T21:30:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986236+0000", 2024-09-17T21:30:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7416999999999998e-05 2024-09-17T21:30:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39458 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1630208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.479825909999999, 2024-09-17T21:30:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018255856000000001, 2024-09-17T21:30:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39458 2024-09-17T21:30:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:30:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968077+0000", 2024-09-17T21:30:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.779e-06 2024-09-17T21:30:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968081+0000", 2024-09-17T21:30:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.532e-06 2024-09-17T21:30:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968085+0000", 2024-09-17T21:30:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999999e-06 2024-09-17T21:30:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986256+0000", 2024-09-17T21:30:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018171273000000002 2024-09-17T21:30:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986293+0000", 2024-09-17T21:30:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6534e-05 2024-09-17T21:30:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986330+0000", 2024-09-17T21:30:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7536000000000001e-05 2024-09-17T21:30:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39459 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1695744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.479811332000001, 2024-09-17T21:30:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334493, 2024-09-17T21:30:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39459 2024-09-17T21:30:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968090+0000", 2024-09-17T21:30:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:30:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968091+0000", 2024-09-17T21:30:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3319999999999999e-06 2024-09-17T21:30:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968094+0000", 2024-09-17T21:30:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2590000000000001e-06 2024-09-17T21:30:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968097+0000", 2024-09-17T21:30:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:30:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986350+0000", 2024-09-17T21:30:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018253364000000001 2024-09-17T21:30:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986387+0000", 2024-09-17T21:30:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6350999999999998e-05 2024-09-17T21:30:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986424+0000", 2024-09-17T21:30:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6897999999999998e-05 2024-09-17T21:30:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39460 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1761280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.479800195999999, 2024-09-17T21:30:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018415918999999999, 2024-09-17T21:30:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39460 2024-09-17T21:30:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968101+0000", 2024-09-17T21:30:31.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T21:30:31.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968102+0000", 2024-09-17T21:30:31.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1689999999999999e-06 2024-09-17T21:30:31.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968105+0000", 2024-09-17T21:30:31.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:30:31.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968108+0000", 2024-09-17T21:30:31.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6630000000000002e-06 2024-09-17T21:30:31.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986442+0000", 2024-09-17T21:30:31.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334487 2024-09-17T21:30:31.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986479+0000", 2024-09-17T21:30:31.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6557000000000003e-05 2024-09-17T21:30:31.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986516+0000", 2024-09-17T21:30:31.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7648999999999998e-05 2024-09-17T21:30:31.519 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.519 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.519 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.519 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39461 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1826816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.519 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:31.519 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.479789436000001, 2024-09-17T21:30:31.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018499768, 2024-09-17T21:30:31.519 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.519 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.520 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.520 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.520 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.520 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.520 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39461 2024-09-17T21:30:31.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.520 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:31.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:31.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:31.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4699999999999998e-07 2024-09-17T21:30:31.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968112+0000", 2024-09-17T21:30:31.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-06 2024-09-17T21:30:31.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968116+0000", 2024-09-17T21:30:31.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1080000000000001e-06 2024-09-17T21:30:31.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968122+0000", 2024-09-17T21:30:31.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1070000000000001e-06 2024-09-17T21:30:31.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986537+0000", 2024-09-17T21:30:31.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018414834000000001 2024-09-17T21:30:31.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986572+0000", 2024-09-17T21:30:31.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5726000000000003e-05 2024-09-17T21:30:31.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986611+0000", 2024-09-17T21:30:31.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8584000000000002e-05 2024-09-17T21:30:31.523 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.523 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.523 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.524 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39462 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1892352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.524 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:31.524 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.479775071999999, 2024-09-17T21:30:31.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018583252000000001, 2024-09-17T21:30:31.524 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.524 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.524 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.524 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39462 2024-09-17T21:30:31.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.524 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:31.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:31.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968126+0000", 2024-09-17T21:30:31.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:31.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968127+0000", 2024-09-17T21:30:31.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.229e-06 2024-09-17T21:30:31.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968131+0000", 2024-09-17T21:30:31.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8010000000000001e-06 2024-09-17T21:30:31.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968134+0000", 2024-09-17T21:30:31.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8880000000000001e-06 2024-09-17T21:30:31.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986634+0000", 2024-09-17T21:30:31.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018500501999999999 2024-09-17T21:30:31.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986671+0000", 2024-09-17T21:30:31.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6881000000000001e-05 2024-09-17T21:30:31.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986709+0000", 2024-09-17T21:30:31.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7546999999999998e-05 2024-09-17T21:30:31.527 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.528 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.528 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.528 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39463 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1957888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.528 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:31.528 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.479763457000001, 2024-09-17T21:30:31.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018667186999999998, 2024-09-17T21:30:31.528 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.528 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.528 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.528 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.528 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.529 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.529 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39463 2024-09-17T21:30:31.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.529 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:31.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:31.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:31.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:30:31.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968138+0000", 2024-09-17T21:30:31.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:30:31.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968142+0000", 2024-09-17T21:30:31.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.145e-06 2024-09-17T21:30:31.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968144+0000", 2024-09-17T21:30:31.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6869999999999999e-06 2024-09-17T21:30:31.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986728+0000", 2024-09-17T21:30:31.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018584163000000001 2024-09-17T21:30:31.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986766+0000", 2024-09-17T21:30:31.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7987000000000001e-05 2024-09-17T21:30:31.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986804+0000", 2024-09-17T21:30:31.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7892000000000002e-05 2024-09-17T21:30:31.532 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.532 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.532 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.532 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39464 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.532 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:31.533 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.479753148, 2024-09-17T21:30:31.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018753723999999999, 2024-09-17T21:30:31.533 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.533 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.533 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.533 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.533 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.533 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.533 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39464 2024-09-17T21:30:31.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.533 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:31.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:31.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968148+0000", 2024-09-17T21:30:31.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:30:31.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968152+0000", 2024-09-17T21:30:31.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3749999999999996e-06 2024-09-17T21:30:31.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968155+0000", 2024-09-17T21:30:31.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.371e-06 2024-09-17T21:30:31.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968158+0000", 2024-09-17T21:30:31.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T21:30:31.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986824+0000", 2024-09-17T21:30:31.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018665779 2024-09-17T21:30:31.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986863+0000", 2024-09-17T21:30:31.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9273000000000002e-05 2024-09-17T21:30:31.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986901+0000", 2024-09-17T21:30:31.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7978000000000001e-05 2024-09-17T21:30:31.536 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.536 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.537 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.537 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39465 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:31.537 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:31.537 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.479737749000002, 2024-09-17T21:30:31.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018825341999999998, 2024-09-17T21:30:31.537 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.537 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.537 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.537 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.537 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.537 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.537 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39465 2024-09-17T21:30:31.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.538 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:31.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:31.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:31.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-17T21:30:31.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968164+0000", 2024-09-17T21:30:31.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1289999999999999e-06 2024-09-17T21:30:31.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968167+0000", 2024-09-17T21:30:31.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1379999999999999e-06 2024-09-17T21:30:31.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968170+0000", 2024-09-17T21:30:31.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:30:31.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986914+0000", 2024-09-17T21:30:31.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018743829 2024-09-17T21:30:31.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986950+0000", 2024-09-17T21:30:31.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5819999999999999e-05 2024-09-17T21:30:31.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986988+0000", 2024-09-17T21:30:31.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8362000000000003e-05 2024-09-17T21:30:31.541 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.541 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.541 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.541 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.541 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.541 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:31.542 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:30:31.543 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:30:31.544 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:31.540+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:31.643 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:31.641+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:31.664 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:31.665 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:31.681 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:30:31.682 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:30:31.686 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:31.686 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:31.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.686 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:70218 3.16 3.f1b49616 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-17T21:30:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.87908644300000005, 2024-09-17T21:30:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.87910604000000003, 2024-09-17T21:30:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for peered", 2024-09-17T21:30:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 70218 2024-09-17T21:30:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4799999999999998e-07 2024-09-17T21:30:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805642+0000", 2024-09-17T21:30:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2009999999999999e-06 2024-09-17T21:30:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805650+0000", 2024-09-17T21:30:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1000000000000004e-06 2024-09-17T21:30:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805655+0000", 2024-09-17T21:30:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:30:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805722+0000", 2024-09-17T21:30:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6878000000000006e-05 2024-09-17T21:30:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:30:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805733+0000", 2024-09-17T21:30:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0895e-05 2024-09-17T21:30:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-17T21:30:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 1 2024-09-17T21:30:31.691 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:31.694 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:30:31.697 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:31.697 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:31.712 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:31.712 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:31.729 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:31.730 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:31.745 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:31.742+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:31.747 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:31.748 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:31.754 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is set 2024-09-17T21:30:31.806 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:31.806 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:31.814 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:30:31.846 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:31.843+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:31.904 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-17T21:30:31.946 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:31.943+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:31.961 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:31.961 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:31.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:31.961 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:31.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.961 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:30:31.961 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:31.961 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.018480303000004, 2024-09-17T21:30:31.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61572000699999996, 2024-09-17T21:30:31.961 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.961 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.961 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:31.962 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.962 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:30:31.962 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:30:31.962 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T21:30:31.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.962 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:31.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:31.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939745+0000", 2024-09-17T21:30:31.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6899999999999999e-06 2024-09-17T21:30:31.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939753+0000", 2024-09-17T21:30:31.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1950000000000003e-06 2024-09-17T21:30:31.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939767+0000", 2024-09-17T21:30:31.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3429e-05 2024-09-17T21:30:31.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939776+0000", 2024-09-17T21:30:31.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:30:31.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939858+0000", 2024-09-17T21:30:31.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2496000000000004e-05 2024-09-17T21:30:31.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T21:30:31.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939868+0000", 2024-09-17T21:30:31.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1910000000000006e-06 2024-09-17T21:30:31.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940612+0000", 2024-09-17T21:30:31.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074429599999999998 2024-09-17T21:30:31.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:30:31.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940621+0000", 2024-09-17T21:30:31.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9139999999999997e-06 2024-09-17T21:30:31.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554006+0000", 2024-09-17T21:30:31.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61338505099999996 2024-09-17T21:30:31.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554166+0000", 2024-09-17T21:30:31.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015982600000000001 2024-09-17T21:30:31.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:30:31.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554296+0000", 2024-09-17T21:30:31.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013002399999999999 2024-09-17T21:30:31.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:31.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554936+0000", 2024-09-17T21:30:31.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064009400000000001 2024-09-17T21:30:31.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:31.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555389+0000", 2024-09-17T21:30:31.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045355200000000002 2024-09-17T21:30:31.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:31.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555432+0000", 2024-09-17T21:30:31.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2477e-05 2024-09-17T21:30:31.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555463+0000", 2024-09-17T21:30:31.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1612000000000002e-05 2024-09-17T21:30:31.968 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.968 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.968 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.968 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17527 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1005568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.968 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:31.968 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.553252471999997, 2024-09-17T21:30:31.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046452090000000001, 2024-09-17T21:30:31.969 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.969 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.969 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.969 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.969 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.969 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.969 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17527 2024-09-17T21:30:31.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.969 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:31.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:31.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405496+0000", 2024-09-17T21:30:31.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052498400000000004 2024-09-17T21:30:31.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405500+0000", 2024-09-17T21:30:31.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8410000000000002e-06 2024-09-17T21:30:31.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405511+0000", 2024-09-17T21:30:31.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0749e-05 2024-09-17T21:30:31.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405517+0000", 2024-09-17T21:30:31.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:30:31.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451275+0000", 2024-09-17T21:30:31.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045758134999999998 2024-09-17T21:30:31.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451355+0000", 2024-09-17T21:30:31.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9453e-05 2024-09-17T21:30:31.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451423+0000", 2024-09-17T21:30:31.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8782999999999995e-05 2024-09-17T21:30:31.972 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.973 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.973 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.973 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17615 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.973 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:31.973 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.548011983999999, 2024-09-17T21:30:31.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046741309000000002, 2024-09-17T21:30:31.973 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.973 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.973 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.973 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.973 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.973 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.973 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17615 2024-09-17T21:30:31.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.974 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:31.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:31.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:31.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:30:31.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410213+0000", 2024-09-17T21:30:31.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:30:31.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410217+0000", 2024-09-17T21:30:31.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8199999999999998e-06 2024-09-17T21:30:31.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410220+0000", 2024-09-17T21:30:31.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6900000000000001e-06 2024-09-17T21:30:31.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455264+0000", 2024-09-17T21:30:31.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045044002999999999 2024-09-17T21:30:31.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455338+0000", 2024-09-17T21:30:31.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4182000000000002e-05 2024-09-17T21:30:31.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.456953+0000", 2024-09-17T21:30:31.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016150089999999999 2024-09-17T21:30:31.977 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.977 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.977 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.977 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17616 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 872448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.977 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:31.977 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.547999932000003, 2024-09-17T21:30:31.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946039000000002, 2024-09-17T21:30:31.978 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.978 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.978 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.978 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.978 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.978 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.978 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17616 2024-09-17T21:30:31.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.978 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:31.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:31.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:31.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T21:30:31.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410225+0000", 2024-09-17T21:30:31.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3459999999999999e-06 2024-09-17T21:30:31.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410228+0000", 2024-09-17T21:30:31.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.083e-06 2024-09-17T21:30:31.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410232+0000", 2024-09-17T21:30:31.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4419999999999998e-06 2024-09-17T21:30:31.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457004+0000", 2024-09-17T21:30:31.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046771873999999998 2024-09-17T21:30:31.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457084+0000", 2024-09-17T21:30:31.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0506000000000005e-05 2024-09-17T21:30:31.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457170+0000", 2024-09-17T21:30:31.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5494000000000005e-05 2024-09-17T21:30:31.981 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.981 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.981 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.982 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17617 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.982 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:31.982 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.547988529999998, 2024-09-17T21:30:31.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047084168000000003, 2024-09-17T21:30:31.982 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.982 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.982 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.982 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.982 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.982 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.982 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17617 2024-09-17T21:30:31.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.982 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:31.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:31.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:31.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-17T21:30:31.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410237+0000", 2024-09-17T21:30:31.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1960000000000001e-06 2024-09-17T21:30:31.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410246+0000", 2024-09-17T21:30:31.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7880000000000001e-06 2024-09-17T21:30:31.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410250+0000", 2024-09-17T21:30:31.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.518e-06 2024-09-17T21:30:31.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457209+0000", 2024-09-17T21:30:31.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046959283999999997 2024-09-17T21:30:31.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457264+0000", 2024-09-17T21:30:31.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673999999999999e-05 2024-09-17T21:30:31.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457319+0000", 2024-09-17T21:30:31.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5497000000000001e-05 2024-09-17T21:30:31.986 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.986 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.986 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.986 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17618 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 899072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.986 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:31.986 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.547969086000002, 2024-09-17T21:30:31.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047204208999999997, 2024-09-17T21:30:31.986 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.986 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.986 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.987 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.987 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.987 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.987 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17618 2024-09-17T21:30:31.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.987 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:31.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:31.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:31.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8799999999999998e-07 2024-09-17T21:30:31.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410256+0000", 2024-09-17T21:30:31.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T21:30:31.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410260+0000", 2024-09-17T21:30:31.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6500000000000002e-06 2024-09-17T21:30:31.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410263+0000", 2024-09-17T21:30:31.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7089999999999998e-06 2024-09-17T21:30:31.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457355+0000", 2024-09-17T21:30:31.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047092756999999999 2024-09-17T21:30:31.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457407+0000", 2024-09-17T21:30:31.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1808999999999998e-05 2024-09-17T21:30:31.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457459+0000", 2024-09-17T21:30:31.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1684000000000002e-05 2024-09-17T21:30:31.990 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.990 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.990 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.991 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17619 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 913408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.991 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:31.991 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.547957848000003, 2024-09-17T21:30:31.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047293443999999997, 2024-09-17T21:30:31.991 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.991 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.991 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.991 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.991 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.991 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.991 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17619 2024-09-17T21:30:31.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.991 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:31.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:31.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:31.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:30:31.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410267+0000", 2024-09-17T21:30:31.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:30:31.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410270+0000", 2024-09-17T21:30:31.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.01e-06 2024-09-17T21:30:31.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410273+0000", 2024-09-17T21:30:31.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6410000000000002e-06 2024-09-17T21:30:31.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457473+0000", 2024-09-17T21:30:31.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047199829999999998 2024-09-17T21:30:31.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457517+0000", 2024-09-17T21:30:31.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4058999999999999e-05 2024-09-17T21:30:31.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457559+0000", 2024-09-17T21:30:31.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2404999999999998e-05 2024-09-17T21:30:31.995 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.995 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.995 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.995 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17620 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 927744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.995 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:31.995 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.547946676000002, 2024-09-17T21:30:31.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047402409, 2024-09-17T21:30:31.995 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:31.995 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:31.995 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:31.995 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:31.996 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:31.996 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:31.996 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17620 2024-09-17T21:30:31.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.996 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:31.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:31.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:31.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:31.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:31.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:31.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:31.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:31.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:30:31.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:31.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410278+0000", 2024-09-17T21:30:31.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1790000000000001e-06 2024-09-17T21:30:31.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:31.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410281+0000", 2024-09-17T21:30:31.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.061e-06 2024-09-17T21:30:31.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:31.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410284+0000", 2024-09-17T21:30:31.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.65e-06 2024-09-17T21:30:31.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:31.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457587+0000", 2024-09-17T21:30:31.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047303206 2024-09-17T21:30:31.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:31.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457633+0000", 2024-09-17T21:30:31.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6063999999999998e-05 2024-09-17T21:30:31.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:31.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457679+0000", 2024-09-17T21:30:31.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6007000000000002e-05 2024-09-17T21:30:31.999 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.999 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:31.999 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:31.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:31.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:31.999 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17621 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 940032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:31.999 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:31.999 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.54793626, 2024-09-17T21:30:32.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047486550000000002, 2024-09-17T21:30:32.000 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:32.000 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:32.000 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:32.000 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:32.000 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:32.000 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:32.000 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17621 2024-09-17T21:30:32.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.000 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:32.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:32.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:32.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:32.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:32.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:32.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410288+0000", 2024-09-17T21:30:32.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:30:32.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:32.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410289+0000", 2024-09-17T21:30:32.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1179999999999999e-06 2024-09-17T21:30:32.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:32.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410292+0000", 2024-09-17T21:30:32.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:30:32.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:32.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410294+0000", 2024-09-17T21:30:32.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T21:30:32.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:32.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457694+0000", 2024-09-17T21:30:32.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047399416999999999 2024-09-17T21:30:32.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:32.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457734+0000", 2024-09-17T21:30:32.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0528999999999998e-05 2024-09-17T21:30:32.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:32.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457774+0000", 2024-09-17T21:30:32.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9529e-05 2024-09-17T21:30:32.003 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.003 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:32.004 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.004 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17622 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:32.004 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:32.004 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.547926078000003, 2024-09-17T21:30:32.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047596619, 2024-09-17T21:30:32.004 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:32.004 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:32.004 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:32.004 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:32.004 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:32.004 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:32.004 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17622 2024-09-17T21:30:32.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.005 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:32.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:32.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:32.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:32.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:32.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:32.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:32.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:32.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:32.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410303+0000", 2024-09-17T21:30:32.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2979999999999999e-06 2024-09-17T21:30:32.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:32.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410307+0000", 2024-09-17T21:30:32.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:30:32.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:32.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410310+0000", 2024-09-17T21:30:32.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.146e-06 2024-09-17T21:30:32.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:32.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457801+0000", 2024-09-17T21:30:32.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047491322000000002 2024-09-17T21:30:32.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:32.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457850+0000", 2024-09-17T21:30:32.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8585000000000003e-05 2024-09-17T21:30:32.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:32.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457894+0000", 2024-09-17T21:30:32.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4455e-05 2024-09-17T21:30:32.008 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.008 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:32.008 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.009 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17623 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 968704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:32.009 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:32.009 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.547909693999998, 2024-09-17T21:30:32.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047689377999999998, 2024-09-17T21:30:32.009 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:32.009 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:32.009 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:32.009 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:32.009 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:32.009 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:32.009 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17623 2024-09-17T21:30:32.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.010 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:32.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:32.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:32.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:32.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:32.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:32.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:32.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:30:32.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:32.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410315+0000", 2024-09-17T21:30:32.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0499999999999999e-06 2024-09-17T21:30:32.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:32.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410319+0000", 2024-09-17T21:30:32.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.517e-06 2024-09-17T21:30:32.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:32.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410321+0000", 2024-09-17T21:30:32.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:30:32.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:32.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457922+0000", 2024-09-17T21:30:32.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047600638000000001 2024-09-17T21:30:32.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:32.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457964+0000", 2024-09-17T21:30:32.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2126000000000003e-05 2024-09-17T21:30:32.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:32.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458003+0000", 2024-09-17T21:30:32.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9138999999999999e-05 2024-09-17T21:30:32.014 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.014 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:32.014 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.014 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17624 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:32.014 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:32.014 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.547899110000003, 2024-09-17T21:30:32.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047798292999999999, 2024-09-17T21:30:32.014 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:32.014 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:32.014 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:32.014 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:32.015 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:32.015 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:32.015 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17624 2024-09-17T21:30:32.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.015 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:32.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:32.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:32.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:32.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:32.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:32.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:32.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:30:32.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:32.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410326+0000", 2024-09-17T21:30:32.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0610000000000001e-06 2024-09-17T21:30:32.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:32.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410329+0000", 2024-09-17T21:30:32.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0319999999999999e-06 2024-09-17T21:30:32.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:32.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410331+0000", 2024-09-17T21:30:32.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5749999999999999e-06 2024-09-17T21:30:32.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:32.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458033+0000", 2024-09-17T21:30:32.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047701371999999999 2024-09-17T21:30:32.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:32.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458079+0000", 2024-09-17T21:30:32.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:30:32.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:32.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458123+0000", 2024-09-17T21:30:32.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850999999999997e-05 2024-09-17T21:30:32.019 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.019 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:32.019 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.019 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17625 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:32.019 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:32.019 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.547888950000001, 2024-09-17T21:30:32.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047883051000000003, 2024-09-17T21:30:32.019 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:32.019 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:32.019 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:32.019 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:32.019 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:32.019 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:32.020 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17625 2024-09-17T21:30:32.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.020 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:32.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:32.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:32.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:32.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:32.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:32.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:32.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:32.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:32.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410336+0000", 2024-09-17T21:30:32.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1310000000000001e-06 2024-09-17T21:30:32.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:32.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410339+0000", 2024-09-17T21:30:32.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000001e-06 2024-09-17T21:30:32.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:32.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410342+0000", 2024-09-17T21:30:32.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7460000000000001e-06 2024-09-17T21:30:32.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:32.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458137+0000", 2024-09-17T21:30:32.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047794931999999998 2024-09-17T21:30:32.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:32.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458178+0000", 2024-09-17T21:30:32.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1013999999999997e-05 2024-09-17T21:30:32.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:32.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458218+0000", 2024-09-17T21:30:32.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9675999999999998e-05 2024-09-17T21:30:32.023 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.023 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:32.023 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.023 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17626 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1009664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:32.023 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:32.023 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.547878329, 2024-09-17T21:30:32.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047991601000000002, 2024-09-17T21:30:32.023 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:32.023 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:32.024 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:32.024 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:32.024 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:32.024 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:32.024 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17626 2024-09-17T21:30:32.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.024 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:32.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:32.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:32.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:32.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:32.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:32.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410346+0000", 2024-09-17T21:30:32.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:30:32.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:32.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410347+0000", 2024-09-17T21:30:32.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.114e-06 2024-09-17T21:30:32.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:32.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410350+0000", 2024-09-17T21:30:32.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.101e-06 2024-09-17T21:30:32.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:32.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410353+0000", 2024-09-17T21:30:32.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T21:30:32.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:32.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458245+0000", 2024-09-17T21:30:32.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0478921 2024-09-17T21:30:32.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:32.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458292+0000", 2024-09-17T21:30:32.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397999999999999e-05 2024-09-17T21:30:32.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:32.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458337+0000", 2024-09-17T21:30:32.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5221e-05 2024-09-17T21:30:32.030 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.030 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:32.030 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.031 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17627 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:32.031 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:32.031 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.547841228000003, 2024-09-17T21:30:32.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048047383999999999, 2024-09-17T21:30:32.031 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:32.031 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:32.032 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:32.032 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:32.032 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:32.032 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:32.032 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17627 2024-09-17T21:30:32.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.032 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:32.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:32.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:32.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:32.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:32.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:32.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410383+0000", 2024-09-17T21:30:32.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:30:32.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:32.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410385+0000", 2024-09-17T21:30:32.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:30:32.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:32.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410391+0000", 2024-09-17T21:30:32.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:30:32.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:32.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410395+0000", 2024-09-17T21:30:32.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6539999999999998e-06 2024-09-17T21:30:32.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:32.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458350+0000", 2024-09-17T21:30:32.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047954855999999997 2024-09-17T21:30:32.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:32.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458392+0000", 2024-09-17T21:30:32.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2094000000000001e-05 2024-09-17T21:30:32.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:32.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458430+0000", 2024-09-17T21:30:32.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7474000000000001e-05 2024-09-17T21:30:32.036 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.036 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:32.036 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.037 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17628 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:32.037 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:32.037 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.547821667000001, 2024-09-17T21:30:32.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048142676000000002, 2024-09-17T21:30:32.037 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:32.037 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:32.037 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:32.037 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:32.037 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:32.037 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:32.038 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17628 2024-09-17T21:30:32.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.038 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:32.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:32.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:32.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:32.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:32.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:32.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:32.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:30:32.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:32.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410404+0000", 2024-09-17T21:30:32.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-06 2024-09-17T21:30:32.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:32.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410411+0000", 2024-09-17T21:30:32.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6850000000000004e-06 2024-09-17T21:30:32.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:32.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410415+0000", 2024-09-17T21:30:32.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6750000000000001e-06 2024-09-17T21:30:32.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:32.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458457+0000", 2024-09-17T21:30:32.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048042045999999998 2024-09-17T21:30:32.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:32.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458501+0000", 2024-09-17T21:30:32.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3661000000000001e-05 2024-09-17T21:30:32.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:32.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458545+0000", 2024-09-17T21:30:32.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3529000000000003e-05 2024-09-17T21:30:32.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.041 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:32.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.041 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17687 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:32.042 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:32.042 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.545376714, 2024-09-17T21:30:32.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050670448, 2024-09-17T21:30:32.042 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:32.042 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:32.042 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:32.042 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:32.042 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:32.042 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:32.042 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17687 2024-09-17T21:30:32.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.042 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:32.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:32.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:32.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:32.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:32.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:32.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412848+0000", 2024-09-17T21:30:32.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9100000000000003e-07 2024-09-17T21:30:32.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:32.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413102+0000", 2024-09-17T21:30:32.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025415600000000001 2024-09-17T21:30:32.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:32.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413111+0000", 2024-09-17T21:30:32.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2010000000000005e-06 2024-09-17T21:30:32.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:32.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413116+0000", 2024-09-17T21:30:32.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:30:32.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:32.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458559+0000", 2024-09-17T21:30:32.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045442914000000001 2024-09-17T21:30:32.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:32.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458632+0000", 2024-09-17T21:30:32.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3207000000000006e-05 2024-09-17T21:30:32.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:32.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458785+0000", 2024-09-17T21:30:32.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015226700000000001 2024-09-17T21:30:32.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:32.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.461648+0000", 2024-09-17T21:30:32.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028633069999999998 2024-09-17T21:30:32.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:32.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463416+0000", 2024-09-17T21:30:32.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017680230000000001 2024-09-17T21:30:32.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:32.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463475+0000", 2024-09-17T21:30:32.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9333000000000003e-05 2024-09-17T21:30:32.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:32.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463517+0000", 2024-09-17T21:30:32.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2162000000000001e-05 2024-09-17T21:30:32.047 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.047 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:32.047 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.048 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17688 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [write 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:32.048 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:32.048 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.545097845999997, 2024-09-17T21:30:32.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050625098, 2024-09-17T21:30:32.048 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:32.048 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:32.048 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:32.048 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:32.048 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:32.048 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:32.048 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17688 2024-09-17T21:30:32.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.049 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:32.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:32.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:32.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:32.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:32.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:32.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:32.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:30:32.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:32.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413128+0000", 2024-09-17T21:30:32.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.813e-06 2024-09-17T21:30:32.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:32.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413132+0000", 2024-09-17T21:30:32.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:30:32.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:32.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413135+0000", 2024-09-17T21:30:32.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:30:32.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:32.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459756+0000", 2024-09-17T21:30:32.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046621293000000001 2024-09-17T21:30:32.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:32.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459816+0000", 2024-09-17T21:30:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9156000000000001e-05 2024-09-17T21:30:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459912+0000", 2024-09-17T21:30:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6302000000000001e-05 2024-09-17T21:30:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463231+0000", 2024-09-17T21:30:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033188290000000001 2024-09-17T21:30:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:32.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463701+0000", 2024-09-17T21:30:32.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047068199999999999 2024-09-17T21:30:32.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:32.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463728+0000", 2024-09-17T21:30:32.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6726000000000001e-05 2024-09-17T21:30:32.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:32.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463751+0000", 2024-09-17T21:30:32.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2824e-05 2024-09-17T21:30:32.054 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.054 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:32.054 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.054 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17691 3.d 3:b3a055f3:::rbd_data.11207744cc2c.0000000000000083:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:32.054 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:32.054 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.541552942999999, 2024-09-17T21:30:32.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049309370999999998, 2024-09-17T21:30:32.054 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:32.054 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:32.054 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:32.054 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:32.054 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:32.055 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:32.055 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17691 2024-09-17T21:30:32.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.055 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:32.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:32.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:32.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:32.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:32.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:32.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416672+0000", 2024-09-17T21:30:32.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.47e-07 2024-09-17T21:30:32.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:32.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416915+0000", 2024-09-17T21:30:32.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024344500000000001 2024-09-17T21:30:32.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:32.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416924+0000", 2024-09-17T21:30:32.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7739999999999993e-06 2024-09-17T21:30:32.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:32.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416929+0000", 2024-09-17T21:30:32.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6990000000000002e-06 2024-09-17T21:30:32.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:32.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460414+0000", 2024-09-17T21:30:32.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043485466 2024-09-17T21:30:32.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:32.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460540+0000", 2024-09-17T21:30:32.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012608899999999999 2024-09-17T21:30:32.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:32.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460746+0000", 2024-09-17T21:30:32.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000205577 2024-09-17T21:30:32.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:32.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463800+0000", 2024-09-17T21:30:32.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030542030000000001 2024-09-17T21:30:32.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:32.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465867+0000", 2024-09-17T21:30:32.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020665929999999998 2024-09-17T21:30:32.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:32.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465914+0000", 2024-09-17T21:30:32.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497000000000003e-05 2024-09-17T21:30:32.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:32.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465980+0000", 2024-09-17T21:30:32.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6081000000000006e-05 2024-09-17T21:30:32.060 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.060 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:32.060 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.060 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:70218 3.16 3:68692d8f:::rbd_data.1120b9193b9f.000000000000001b:head [stat,delete] snapc 1d=[1d] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-17T21:30:32.060 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:32.060 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.152585854, 2024-09-17T21:30:32.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.951400786, 2024-09-17T21:30:32.060 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:32.061 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:32.061 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:32.061 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:32.061 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:32.061 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:32.061 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 70218 2024-09-17T21:30:32.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.061 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:32.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:32.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:32.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:32.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:32.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:32.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:32.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:32.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4799999999999998e-07 2024-09-17T21:30:32.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:32.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805642+0000", 2024-09-17T21:30:32.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2009999999999999e-06 2024-09-17T21:30:32.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:32.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805650+0000", 2024-09-17T21:30:32.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1000000000000004e-06 2024-09-17T21:30:32.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:32.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805655+0000", 2024-09-17T21:30:32.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:30:32.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:32.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805722+0000", 2024-09-17T21:30:32.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6878000000000006e-05 2024-09-17T21:30:32.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:30:32.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805733+0000", 2024-09-17T21:30:32.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0895e-05 2024-09-17T21:30:32.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:32.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755252+0000", 2024-09-17T21:30:32.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.94951843800000002 2024-09-17T21:30:32.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:32.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755398+0000", 2024-09-17T21:30:32.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014622600000000001 2024-09-17T21:30:32.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:32.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:32.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:32.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.757039+0000", 2024-09-17T21:30:32.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001640819 2024-09-17T21:30:32.065 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.065 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:32.065 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.065 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:32.065 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:32.065 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:32.066 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:32.067 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:32.067 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:32.068 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:30:32.069 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:32.069 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:32.064+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:32.111 INFO:teuthology.orchestra.run.smithi129.stdout:3411 write 0x6b04b3e thru 0x6b0c3a1 (0x7864 bytes) 2024-09-17T21:30:32.141 INFO:teuthology.orchestra.run.smithi129.stdout:3412 punch from 0x5e0e4a0 to 0x5e1cb36, (0xe696 bytes) 2024-09-17T21:30:32.141 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:32.141 INFO:teuthology.orchestra.run.smithi129.stdout:3414 write 0x2ca0b2 thru 0x2d1de2 (0x7d31 bytes) 2024-09-17T21:30:32.167 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:32.164+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:32.180 INFO:teuthology.orchestra.run.smithi129.stdout:3415 punch from 0x1d8b6e2 to 0x1d94478, (0x8d96 bytes) 2024-09-17T21:30:32.180 INFO:teuthology.orchestra.run.smithi129.stdout:3416 write 0x198ba28 thru 0x198f925 (0x3efe bytes) 2024-09-17T21:30:32.181 INFO:teuthology.orchestra.run.smithi129.stdout:3417 write 0x3b750dc thru 0x3b7bff6 (0x6f1b bytes) 2024-09-17T21:30:32.205 INFO:teuthology.orchestra.run.smithi129.stdout:3418 write 0x797131a thru 0x797d3d2 (0xc0b9 bytes) 2024-09-17T21:30:32.220 INFO:teuthology.orchestra.run.smithi129.stdout:3419 write 0xba4f479 thru 0xba4fba3 (0x72b bytes) 2024-09-17T21:30:32.228 INFO:teuthology.orchestra.run.smithi129.stdout:3420 trunc from 0xba4fba4 to 0xe767612 2024-09-17T21:30:32.241 INFO:teuthology.orchestra.run.smithi129.stdout:3421 trunc from 0xe767612 to 0x8ecd441 2024-09-17T21:30:32.245 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:32.245 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:32.259 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:30:32.265 INFO:teuthology.orchestra.run.smithi129.stdout:3422 punch from 0x315779 to 0x322276, (0xcafd bytes) 2024-09-17T21:30:32.265 INFO:teuthology.orchestra.run.smithi129.stdout:3424 writesame 0xbfe43f thru 0xc02b60 data_size 0xf(0x4722 bytes) 2024-09-17T21:30:32.267 INFO:teuthology.orchestra.run.smithi129.stdout:3425 read 0x5495948 thru 0x549ef4d (0x9606 bytes) 2024-09-17T21:30:32.267 INFO:teuthology.orchestra.run.smithi129.stdout:3426 punch from 0x31019b5 to 0x310f0ae, (0xd6f9 bytes) 2024-09-17T21:30:32.267 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:32.267 INFO:teuthology.orchestra.run.smithi129.stdout:3430 clone 28 order 18 su 16384 sc 2 2024-09-17T21:30:32.268 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:32.265+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:32.366 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:32.366 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:32.368 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:32.366+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:32.375 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:30:32.469 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:32.467+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:32.517 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:32.517 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:32.526 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:30:32.570 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:32.568+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:32.629 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:32.629 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:32.637 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:30:32.671 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:32.669+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:32.741 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:32.741 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:32.750 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:30:32.794 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:32.792+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:32.835 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone27 from 0x8ecd441 (overlap 0x8ecd441) to 0x63f6160 2024-09-17T21:30:32.872 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:32.869+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:32.873 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:32.873 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:32.885 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:30:32.973 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:32.970+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:32.999 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-17T21:30:33.048 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:33.048 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:33.063 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:30:33.110 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:33.107+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:33.174 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:33.171+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:33.250 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:33.250 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:33.261 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:30:33.302 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:33.300+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:33.375 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:33.373+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:33.414 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:33.414 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:33.423 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:30:33.476 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:33.473+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:33.489 INFO:teuthology.orchestra.run.smithi129.stdout:3431 trunc from 0x8ecd441 to 0x8831ff0 2024-09-17T21:30:33.512 INFO:teuthology.orchestra.run.smithi129.stdout:3433 writesame 0xc85772f thru 0xc865732 data_size 0x76(0xe004 bytes) 2024-09-17T21:30:33.513 INFO:teuthology.orchestra.run.smithi129.stdout:3434 write 0x7694a95 thru 0x76999ac (0x4f18 bytes) 2024-09-17T21:30:33.517 INFO:teuthology.orchestra.run.smithi129.stdout:3436 write 0xbbea303 thru 0xbbef431 (0x512f bytes) 2024-09-17T21:30:33.524 INFO:teuthology.orchestra.run.smithi129.stdout:3438 writesame 0x5524723 thru 0x552b837 data_size 0x1(0x7115 bytes) 2024-09-17T21:30:33.534 INFO:teuthology.orchestra.run.smithi129.stdout:3439 writesame 0x86735b8 thru 0x867481e data_size 0x1(0x1267 bytes) 2024-09-17T21:30:33.539 INFO:teuthology.orchestra.run.smithi129.stdout:3440 write 0x65bde6c thru 0x65c1414 (0x35a9 bytes) 2024-09-17T21:30:33.550 INFO:teuthology.orchestra.run.smithi129.stdout:3441 write 0xce4c4eb thru 0xce54b71 (0x8687 bytes) 2024-09-17T21:30:33.554 INFO:teuthology.orchestra.run.smithi129.stdout:3442 read 0x305e2c5 thru 0x305f7b6 (0x14f2 bytes) 2024-09-17T21:30:33.554 INFO:teuthology.orchestra.run.smithi129.stdout:3443 write 0x554606b thru 0x55506b1 (0xa647 bytes) 2024-09-17T21:30:33.560 INFO:teuthology.orchestra.run.smithi129.stdout:3444 write 0xd066670 thru 0xd06e4ef (0x7e80 bytes) 2024-09-17T21:30:33.564 INFO:teuthology.orchestra.run.smithi129.stdout:3445 read 0xaf7a7a2 thru 0xaf7c469 (0x1cc8 bytes) 2024-09-17T21:30:33.565 INFO:teuthology.orchestra.run.smithi129.stdout:3449 punch from 0xa90be1c to 0xa917b46, (0xbd2a bytes) 2024-09-17T21:30:33.565 INFO:teuthology.orchestra.run.smithi129.stdout:3450 write 0x5094958 thru 0x509a087 (0x5730 bytes) 2024-09-17T21:30:33.569 INFO:teuthology.orchestra.run.smithi129.stdout:3451 read 0x365dbd0 thru 0x365e53d (0x96e bytes) 2024-09-17T21:30:33.569 INFO:teuthology.orchestra.run.smithi129.stdout:3453 writesame 0x9e64afe thru 0x9e6ac7e data_size 0x1(0x6181 bytes) 2024-09-17T21:30:33.577 INFO:teuthology.orchestra.run.smithi129.stdout:3454 write 0xcf0ee34 thru 0xcf1d679 (0xe846 bytes) 2024-09-17T21:30:33.577 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:33.574+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:33.580 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:33.580 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:33.581 INFO:teuthology.orchestra.run.smithi129.stdout:3455 write 0xcccd34c thru 0xcccd8a9 (0x55e bytes) 2024-09-17T21:30:33.583 INFO:teuthology.orchestra.run.smithi129.stdout:3456 read 0x2878662 thru 0x28786e4 (0x83 bytes) 2024-09-17T21:30:33.583 INFO:teuthology.orchestra.run.smithi129.stdout:3457 read 0xae8b068 thru 0xae9a3b3 (0xf34c bytes) 2024-09-17T21:30:33.584 INFO:teuthology.orchestra.run.smithi129.stdout:3458 write 0x16af065 thru 0x16bd49d (0xe439 bytes) 2024-09-17T21:30:33.589 INFO:teuthology.orchestra.run.smithi129.stdout:3461 punch from 0xc2989a6 to 0xc2a173f, (0x8d99 bytes) 2024-09-17T21:30:33.589 INFO:teuthology.orchestra.run.smithi129.stdout:3462 writesame 0x7c3286 thru 0x7ceaf3 data_size 0x81(0xb86e bytes) 2024-09-17T21:30:33.592 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:30:33.597 INFO:teuthology.orchestra.run.smithi129.stdout:3464 trunc from 0xd06e4f0 to 0xaad34df 2024-09-17T21:30:33.608 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:33.608 INFO:teuthology.orchestra.run.smithi129.stdout:3467 punch from 0x10c880a to 0x10ceb23, (0x6319 bytes) 2024-09-17T21:30:33.608 INFO:teuthology.orchestra.run.smithi129.stdout:3468 read 0x2d79989 thru 0x2d7dd0e (0x4386 bytes) 2024-09-17T21:30:33.614 INFO:teuthology.orchestra.run.smithi129.stdout:3469 write 0xb41874c thru 0xb422770 (0xa025 bytes) 2024-09-17T21:30:33.618 INFO:teuthology.orchestra.run.smithi129.stdout:3470 read 0x6897016 thru 0x68a3db7 (0xcda2 bytes) 2024-09-17T21:30:33.626 INFO:teuthology.orchestra.run.smithi129.stdout:3473 write 0xcbfc8e9 thru 0xcc0a7ab (0xdec3 bytes) 2024-09-17T21:30:33.630 INFO:teuthology.orchestra.run.smithi129.stdout:3474 read 0x38c703 thru 0x38d1cb (0xac9 bytes) 2024-09-17T21:30:33.630 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:33.630 INFO:teuthology.orchestra.run.smithi129.stdout:3477 write 0xc92b281 thru 0xc930a55 (0x57d5 bytes) 2024-09-17T21:30:33.636 INFO:teuthology.orchestra.run.smithi129.stdout:3478 trunc from 0xcc0a7ac to 0xa7ff86a 2024-09-17T21:30:33.654 INFO:teuthology.orchestra.run.smithi129.stdout:3479 write 0xc6e9a69 thru 0xc6ec57f (0x2b17 bytes) 2024-09-17T21:30:33.657 INFO:teuthology.orchestra.run.smithi129.stdout:3480 writesame 0x52f59 thru 0x53da6 data_size 0x1(0xe4e bytes) 2024-09-17T21:30:33.670 INFO:teuthology.orchestra.run.smithi129.stdout:3484 trunc from 0xc6ec580 to 0x6f40f85 2024-09-17T21:30:33.678 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:33.675+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:33.699 INFO:teuthology.orchestra.run.smithi129.stdout:3485 writesame 0xa4777c2 thru 0xa47bdaf data_size 0x1(0x45ee bytes) 2024-09-17T21:30:33.699 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:33.708 INFO:teuthology.orchestra.run.smithi129.stdout:3487 write 0xc7dc8d5 thru 0xc7ea77c (0xdea8 bytes) 2024-09-17T21:30:33.712 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:33.712 INFO:teuthology.orchestra.run.smithi129.stdout:3491 write 0xa5b6e16 thru 0xa5c13eb (0xa5d6 bytes) 2024-09-17T21:30:33.716 INFO:teuthology.orchestra.run.smithi129.stdout:3492 write 0x1c5f660 thru 0x1c6bed2 (0xc873 bytes) 2024-09-17T21:30:33.716 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:33.716 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:33.722 INFO:teuthology.orchestra.run.smithi129.stdout:3493 read 0x5dbfbb7 thru 0x5dc18b8 (0x1d02 bytes) 2024-09-17T21:30:33.722 INFO:teuthology.orchestra.run.smithi129.stdout:3494 read 0x9c7710f thru 0x9c85c62 (0xeb54 bytes) 2024-09-17T21:30:33.727 INFO:teuthology.orchestra.run.smithi129.stdout:3495 trunc from 0xc7ea77d to 0x84880e5 2024-09-17T21:30:33.728 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:30:33.736 INFO:teuthology.orchestra.run.smithi129.stdout:3496 write 0x78bf5da thru 0x78c32b8 (0x3cdf bytes) 2024-09-17T21:30:33.742 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:33.743 INFO:teuthology.orchestra.run.smithi129.stdout:3500 read 0x25b58ed thru 0x25bec92 (0x93a6 bytes) 2024-09-17T21:30:33.743 INFO:teuthology.orchestra.run.smithi129.stdout:3501 read 0x2641476 thru 0x2641be7 (0x772 bytes) 2024-09-17T21:30:33.743 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:33.757 INFO:teuthology.orchestra.run.smithi129.stdout:3504 write 0xe0aee47 thru 0xe0baf9e (0xc158 bytes) 2024-09-17T21:30:33.760 INFO:teuthology.orchestra.run.smithi129.stdout:3505 read 0xd61de30 thru 0xd61e79a (0x96b bytes) 2024-09-17T21:30:33.767 INFO:teuthology.orchestra.run.smithi129.stdout:3506 trunc from 0xe0baf9f to 0x7debb84 2024-09-17T21:30:33.774 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:30:33.779 INFO:teuthology.orchestra.run.smithi129.stdout:3508 write 0x4bf4527 thru 0x4bfa4b0 (0x5f8a bytes) 2024-09-17T21:30:33.779 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:33.776+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:33.795 INFO:teuthology.orchestra.run.smithi129.stdout:3509 write 0xce9e548 thru 0xcea3417 (0x4ed0 bytes) 2024-09-17T21:30:33.799 INFO:teuthology.orchestra.run.smithi129.stdout:3510 read 0x7c2b4be thru 0x7c35db9 (0xa8fc bytes) 2024-09-17T21:30:33.799 INFO:teuthology.orchestra.run.smithi129.stdout:3512 punch from 0x7567f7f to 0x7569f9e, (0x201f bytes) 2024-09-17T21:30:33.799 INFO:teuthology.orchestra.run.smithi129.stdout:3513 writesame 0x8861d51 thru 0x887127b data_size 0x1(0xf52b bytes) 2024-09-17T21:30:33.822 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:33.822 INFO:teuthology.orchestra.run.smithi129.stdout:3515 punch from 0xcde8863 to 0xcdefe7d, (0x761a bytes) 2024-09-17T21:30:33.822 INFO:teuthology.orchestra.run.smithi129.stdout:3516 write 0xb8e582 thru 0xb98412 (0x9e91 bytes) 2024-09-17T21:30:33.829 INFO:teuthology.orchestra.run.smithi129.stdout:3517 write 0xa4912af thru 0xa49f322 (0xe074 bytes) 2024-09-17T21:30:33.833 INFO:teuthology.orchestra.run.smithi129.stdout:3518 read 0x263484c thru 0x2634c61 (0x416 bytes) 2024-09-17T21:30:33.834 INFO:teuthology.orchestra.run.smithi129.stdout:3519 punch from 0x610e90b to 0x61165ca, (0x7cbf bytes) 2024-09-17T21:30:33.834 INFO:teuthology.orchestra.run.smithi129.stdout:3520 punch from 0x8ad1c65 to 0x8ad3474, (0x180f bytes) 2024-09-17T21:30:33.848 INFO:teuthology.orchestra.run.smithi129.stdout:3521 trunc from 0xcea3418 to 0x2d4a585 2024-09-17T21:30:33.866 INFO:teuthology.orchestra.run.smithi129.stdout:3522 read 0x2ca1084 thru 0x2ca2ce7 (0x1c64 bytes) 2024-09-17T21:30:33.866 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:33.866 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:33.879 INFO:teuthology.orchestra.run.smithi129.stdout:3523 write 0x7be7aa0 thru 0x7bf0e1b (0x937c bytes) 2024-09-17T21:30:33.879 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:33.877+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:33.883 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:30:33.892 INFO:teuthology.orchestra.run.smithi129.stdout:3525 write 0xa9b5e55 thru 0xa9b91ef (0x339b bytes) 2024-09-17T21:30:33.900 INFO:teuthology.orchestra.run.smithi129.stdout:3526 trunc from 0xa9b91f0 to 0x750c541 2024-09-17T21:30:33.921 INFO:teuthology.orchestra.run.smithi129.stdout:3527 write 0xaee66e8 thru 0xaee6c24 (0x53d bytes) 2024-09-17T21:30:33.936 INFO:teuthology.orchestra.run.smithi129.stdout:3528 trunc from 0xaee6c25 to 0x122e2c3 2024-09-17T21:30:33.960 INFO:teuthology.orchestra.run.smithi129.stdout:3530 writesame 0x3a19c48 thru 0x3a25e2c data_size 0x7(0xc1e5 bytes) 2024-09-17T21:30:33.960 INFO:teuthology.orchestra.run.smithi129.stdout:3531 read 0xffbe22 thru 0xffdd86 (0x1f65 bytes) 2024-09-17T21:30:33.960 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:33.976 INFO:teuthology.orchestra.run.smithi129.stdout:3534 write 0xcb915b4 thru 0xcb9aee8 (0x9935 bytes) 2024-09-17T21:30:33.980 INFO:teuthology.orchestra.run.smithi129.stdout:3537 read 0x34ee525 thru 0x34f1363 (0x2e3f bytes) 2024-09-17T21:30:33.980 INFO:teuthology.orchestra.run.smithi129.stdout:3539 write 0x7bcdbf0 thru 0x7bdb003 (0xd414 bytes) 2024-09-17T21:30:33.981 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:33.977+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:33.984 INFO:teuthology.orchestra.run.smithi129.stdout:3540 write 0xc1120b0 thru 0xc11ab10 (0x8a61 bytes) 2024-09-17T21:30:33.988 INFO:teuthology.orchestra.run.smithi129.stdout:3541 read 0x86a4c98 thru 0x86b3072 (0xe3db bytes) 2024-09-17T21:30:33.988 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:33.988 INFO:teuthology.orchestra.run.smithi129.stdout:3543 write 0x744132d thru 0x744a43e (0x9112 bytes) 2024-09-17T21:30:33.991 INFO:teuthology.orchestra.run.smithi129.stdout:3544 write 0x53885a6 thru 0x538df1b (0x5976 bytes) 2024-09-17T21:30:33.996 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:33.996 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:34.005 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:30:34.007 INFO:teuthology.orchestra.run.smithi129.stdout:3545 trunc from 0xcb9aee9 to 0x3c41c5a 2024-09-17T21:30:34.017 INFO:teuthology.orchestra.run.smithi129.stdout:3547 write 0x1176c26 thru 0x117eb9d (0x7f78 bytes) 2024-09-17T21:30:34.025 INFO:teuthology.orchestra.run.smithi129.stdout:3548 read 0x12e2174 thru 0x12e4edb (0x2d68 bytes) 2024-09-17T21:30:34.031 INFO:teuthology.orchestra.run.smithi129.stdout:3549 write 0x472dd14 thru 0x473bbbc (0xdea9 bytes) 2024-09-17T21:30:34.046 INFO:teuthology.orchestra.run.smithi129.stdout:3550 trunc from 0x473bbbd to 0xe149f4d 2024-09-17T21:30:34.056 INFO:teuthology.orchestra.run.smithi129.stdout:3551 trunc from 0xe149f4d to 0xa05030d 2024-09-17T21:30:34.061 INFO:teuthology.orchestra.run.smithi129.stdout:3552 read 0x449d90b thru 0x44a6f44 (0x963a bytes) 2024-09-17T21:30:34.062 INFO:teuthology.orchestra.run.smithi129.stdout:3553 read 0x2448883 thru 0x244e3a9 (0x5b27 bytes) 2024-09-17T21:30:34.062 INFO:teuthology.orchestra.run.smithi129.stdout:3554 writesame 0x4b7e6a2 thru 0x4b85914 data_size 0x1(0x7273 bytes) 2024-09-17T21:30:34.070 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:30:34.074 INFO:teuthology.orchestra.run.smithi129.stdout:3556 write 0x26e181a thru 0x26e647c (0x4c63 bytes) 2024-09-17T21:30:34.079 INFO:teuthology.orchestra.run.smithi129.stdout:3557 write 0x8b436b2 thru 0x8b4fd6d (0xc6bc bytes) 2024-09-17T21:30:34.081 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:34.078+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:34.084 INFO:teuthology.orchestra.run.smithi129.stdout:3558 read 0x425d33f thru 0x425fb1b (0x27dd bytes) 2024-09-17T21:30:34.090 INFO:teuthology.orchestra.run.smithi129.stdout:3560 write 0xa752a38 thru 0xa753115 (0x6de bytes) 2024-09-17T21:30:34.093 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:34.094 INFO:teuthology.orchestra.run.smithi129.stdout:3562 read 0x2dddb3b thru 0x2ded65a (0xfb20 bytes) 2024-09-17T21:30:34.094 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:34.095 INFO:teuthology.orchestra.run.smithi129.stdout:3564 trunc from 0xa753116 to 0x97f0088 2024-09-17T21:30:34.105 INFO:teuthology.orchestra.run.smithi129.stdout:3566 write 0x9450531 thru 0x9450867 (0x337 bytes) 2024-09-17T21:30:34.108 INFO:teuthology.orchestra.run.smithi129.stdout:3567 writesame 0x77a23aa thru 0x77aef4c data_size 0x1(0xcba3 bytes) 2024-09-17T21:30:34.125 INFO:teuthology.orchestra.run.smithi129.stdout:3569 read 0x7aec60e thru 0x7af2407 (0x5dfa bytes) 2024-09-17T21:30:34.125 INFO:teuthology.orchestra.run.smithi129.stdout:3570 punch from 0x37652e to 0x382088, (0xbb5a bytes) 2024-09-17T21:30:34.126 INFO:teuthology.orchestra.run.smithi129.stdout:3571 read 0x48ee557 thru 0x48f6d8c (0x8836 bytes) 2024-09-17T21:30:34.126 INFO:teuthology.orchestra.run.smithi129.stdout:3572 write 0xd8744 thru 0xe16c5 (0x8f82 bytes) 2024-09-17T21:30:34.133 INFO:teuthology.orchestra.run.smithi129.stdout:3573 read 0x313442a thru 0x313447f (0x56 bytes) 2024-09-17T21:30:34.133 INFO:teuthology.orchestra.run.smithi129.stdout:3574 read 0x3f7dd9f thru 0x3f80a8e (0x2cf0 bytes) 2024-09-17T21:30:34.142 INFO:teuthology.orchestra.run.smithi129.stdout:3575 writesame 0xc191f3f thru 0xc1a0297 data_size 0x1(0xe359 bytes) 2024-09-17T21:30:34.153 INFO:teuthology.orchestra.run.smithi129.stdout:3576 trunc from 0xc1a0298 to 0x25ad11a 2024-09-17T21:30:34.160 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:34.160 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:34.170 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:30:34.182 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:34.179+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:34.182 INFO:teuthology.orchestra.run.smithi129.stdout:3577 write 0xc283422 thru 0xc28c36b (0x8f4a bytes) 2024-09-17T21:30:34.189 INFO:teuthology.orchestra.run.smithi129.stdout:3578 punch from 0x1dae2ba to 0x1db733b, (0x9081 bytes) 2024-09-17T21:30:34.189 INFO:teuthology.orchestra.run.smithi129.stdout:3579 read 0x49d50ab thru 0x49d9011 (0x3f67 bytes) 2024-09-17T21:30:34.189 INFO:teuthology.orchestra.run.smithi129.stdout:3580 read 0x5a63633 thru 0x5a6f96c (0xc33a bytes) 2024-09-17T21:30:34.189 INFO:teuthology.orchestra.run.smithi129.stdout:3581 punch from 0x9385691 to 0x9389aea, (0x4459 bytes) 2024-09-17T21:30:34.189 INFO:teuthology.orchestra.run.smithi129.stdout:3582 read 0x560000b thru 0x5605599 (0x558f bytes) 2024-09-17T21:30:34.189 INFO:teuthology.orchestra.run.smithi129.stdout:3583 write 0x7205549 thru 0x7213931 (0xe3e9 bytes) 2024-09-17T21:30:34.190 INFO:teuthology.orchestra.run.smithi129.stdout:3586 writesame 0x1d7e3ac thru 0x1d81bea data_size 0x11(0x383f bytes) 2024-09-17T21:30:34.194 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:34.194 INFO:teuthology.orchestra.run.smithi129.stdout:3589 write 0x3616223 thru 0x3625d40 (0xfb1e bytes) 2024-09-17T21:30:34.198 INFO:teuthology.orchestra.run.smithi129.stdout:3590 trunc from 0xc28c36c to 0xc16f87d 2024-09-17T21:30:34.207 INFO:teuthology.orchestra.run.smithi129.stdout:3592 write 0x9092949 thru 0x9092a23 (0xdb bytes) 2024-09-17T21:30:34.211 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:34.211 INFO:teuthology.orchestra.run.smithi129.stdout:3594 read 0x533f633 thru 0x5341118 (0x1ae6 bytes) 2024-09-17T21:30:34.212 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:34.212 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:34.213 INFO:teuthology.orchestra.run.smithi129.stdout:3596 trunc from 0xc16f87d to 0xd27fcb1 2024-09-17T21:30:34.218 INFO:teuthology.orchestra.run.smithi129.stdout:3598 write 0xc27ddf0 thru 0xc2829e7 (0x4bf8 bytes) 2024-09-17T21:30:34.225 INFO:teuthology.orchestra.run.smithi129.stdout:3599 read 0xad0b98 thru 0xad6409 (0x5872 bytes) 2024-09-17T21:30:34.226 INFO:teuthology.orchestra.run.smithi129.stdout:3600 trunc from 0xd27fcb1 to 0xbf4c5db 2024-09-17T21:30:34.230 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:30:34.230 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:30:34.237 INFO:teuthology.orchestra.run.smithi129.stdout:3601 writesame 0x4acae0b thru 0x4ad1723 data_size 0x1(0x6919 bytes) 2024-09-17T21:30:34.247 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:34.247 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:34.258 INFO:teuthology.orchestra.run.smithi129.stdout:3602 writesame 0xde96ff5 thru 0xde9edb1 data_size 0x1(0x7dbd bytes) 2024-09-17T21:30:34.258 INFO:teuthology.orchestra.run.smithi129.stdout:3603 write 0xcbe2716 thru 0xcbeef6a (0xc855 bytes) 2024-09-17T21:30:34.262 INFO:teuthology.orchestra.run.smithi129.stdout:3604 read 0x78d23ac thru 0x78ddff3 (0xbc48 bytes) 2024-09-17T21:30:34.262 INFO:teuthology.orchestra.run.smithi129.stdout:3605 read 0x612427a thru 0x6131858 (0xd5df bytes) 2024-09-17T21:30:34.262 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:34.262 INFO:teuthology.orchestra.run.smithi129.stdout:3607 write 0x408e463 thru 0x4092dfb (0x4999 bytes) 2024-09-17T21:30:34.264 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:34.264 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:34.266 INFO:teuthology.orchestra.run.smithi129.stdout:3608 trunc from 0xde9edb2 to 0xcf4f8f8 2024-09-17T21:30:34.271 INFO:teuthology.orchestra.run.smithi129.stdout:3610 writesame 0x74e0ccf thru 0x74e1753 data_size 0x1(0xa85 bytes) 2024-09-17T21:30:34.274 INFO:teuthology.orchestra.run.smithi129.stdout:3611 write 0x96fe5d6 thru 0x9700061 (0x1a8c bytes) 2024-09-17T21:30:34.282 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:34.283 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:34.283 INFO:teuthology.orchestra.run.smithi129.stdout:3612 punch from 0x234ca88 to 0x235999b, (0xcf13 bytes) 2024-09-17T21:30:34.283 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:34.279+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:34.287 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:34.287 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21342 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2082816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 44.608314542000002, 2024-09-17T21:30:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022083523000000001, 2024-09-17T21:30:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21342 2024-09-17T21:30:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:30:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676659+0000", 2024-09-17T21:30:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0950000000000001e-06 2024-09-17T21:30:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676662+0000", 2024-09-17T21:30:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.971e-06 2024-09-17T21:30:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676665+0000", 2024-09-17T21:30:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6189999999999998e-06 2024-09-17T21:30:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698568+0000", 2024-09-17T21:30:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021903549000000001 2024-09-17T21:30:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698668+0000", 2024-09-17T21:30:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9778999999999995e-05 2024-09-17T21:30:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698741+0000", 2024-09-17T21:30:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000006e-05 2024-09-17T21:30:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21343 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2095104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 44.608304603999997, 2024-09-17T21:30:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022504024000000001, 2024-09-17T21:30:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21343 2024-09-17T21:30:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:30:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676669+0000", 2024-09-17T21:30:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.003e-06 2024-09-17T21:30:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676672+0000", 2024-09-17T21:30:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9280000000000002e-06 2024-09-17T21:30:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676674+0000", 2024-09-17T21:30:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5469999999999999e-06 2024-09-17T21:30:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699015+0000", 2024-09-17T21:30:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022340563000000001 2024-09-17T21:30:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699102+0000", 2024-09-17T21:30:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7425999999999999e-05 2024-09-17T21:30:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699172+0000", 2024-09-17T21:30:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9346999999999999e-05 2024-09-17T21:30:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45576 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.580070804000002, 2024-09-17T21:30:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022024660000000001, 2024-09-17T21:30:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45576 2024-09-17T21:30:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704902+0000", 2024-09-17T21:30:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9100000000000004e-07 2024-09-17T21:30:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704905+0000", 2024-09-17T21:30:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6139999999999999e-06 2024-09-17T21:30:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704911+0000", 2024-09-17T21:30:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8619999999999997e-06 2024-09-17T21:30:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704917+0000", 2024-09-17T21:30:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:30:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726681+0000", 2024-09-17T21:30:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021764569000000001 2024-09-17T21:30:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726726+0000", 2024-09-17T21:30:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4218999999999998e-05 2024-09-17T21:30:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726926+0000", 2024-09-17T21:30:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000200466 2024-09-17T21:30:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45577 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.580047147999998, 2024-09-17T21:30:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022269694999999999, 2024-09-17T21:30:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45577 2024-09-17T21:30:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704926+0000", 2024-09-17T21:30:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:30:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704928+0000", 2024-09-17T21:30:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9989999999999998e-06 2024-09-17T21:30:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704934+0000", 2024-09-17T21:30:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:30:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704939+0000", 2024-09-17T21:30:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5909999999999998e-06 2024-09-17T21:30:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726954+0000", 2024-09-17T21:30:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022014885000000001 2024-09-17T21:30:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726995+0000", 2024-09-17T21:30:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1474000000000003e-05 2024-09-17T21:30:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727195+0000", 2024-09-17T21:30:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019968300000000001 2024-09-17T21:30:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45578 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.580025903999999, 2024-09-17T21:30:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022484006000000001, 2024-09-17T21:30:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45578 2024-09-17T21:30:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704947+0000", 2024-09-17T21:30:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1399999999999997e-07 2024-09-17T21:30:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704949+0000", 2024-09-17T21:30:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0959999999999999e-06 2024-09-17T21:30:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704956+0000", 2024-09-17T21:30:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9330000000000004e-06 2024-09-17T21:30:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704961+0000", 2024-09-17T21:30:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6759999999999998e-06 2024-09-17T21:30:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727213+0000", 2024-09-17T21:30:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022252568 2024-09-17T21:30:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727259+0000", 2024-09-17T21:30:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5238000000000003e-05 2024-09-17T21:30:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727430+0000", 2024-09-17T21:30:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000171881 2024-09-17T21:30:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45579 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.580003976, 2024-09-17T21:30:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022749080000000001, 2024-09-17T21:30:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45579 2024-09-17T21:30:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704969+0000", 2024-09-17T21:30:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:30:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704971+0000", 2024-09-17T21:30:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.097e-06 2024-09-17T21:30:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704977+0000", 2024-09-17T21:30:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6329999999999999e-06 2024-09-17T21:30:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704983+0000", 2024-09-17T21:30:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:30:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727457+0000", 2024-09-17T21:30:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022474521000000001 2024-09-17T21:30:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727498+0000", 2024-09-17T21:30:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0558e-05 2024-09-17T21:30:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727717+0000", 2024-09-17T21:30:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219805 2024-09-17T21:30:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45650 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 20480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.578880669, 2024-09-17T21:30:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021996219000000001, 2024-09-17T21:30:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45650 2024-09-17T21:30:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706092+0000", 2024-09-17T21:30:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5199999999999997e-07 2024-09-17T21:30:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706095+0000", 2024-09-17T21:30:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6079999999999998e-06 2024-09-17T21:30:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706101+0000", 2024-09-17T21:30:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:30:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706106+0000", 2024-09-17T21:30:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T21:30:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728000+0000", 2024-09-17T21:30:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021893378000000002 2024-09-17T21:30:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728039+0000", 2024-09-17T21:30:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9249000000000003e-05 2024-09-17T21:30:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728088+0000", 2024-09-17T21:30:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8785000000000001e-05 2024-09-17T21:30:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45652 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.578857838000001, 2024-09-17T21:30:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022100465999999999, 2024-09-17T21:30:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45652 2024-09-17T21:30:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706115+0000", 2024-09-17T21:30:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:30:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706117+0000", 2024-09-17T21:30:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-06 2024-09-17T21:30:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706122+0000", 2024-09-17T21:30:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.288e-06 2024-09-17T21:30:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706136+0000", 2024-09-17T21:30:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3409e-05 2024-09-17T21:30:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728112+0000", 2024-09-17T21:30:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021976724999999999 2024-09-17T21:30:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728151+0000", 2024-09-17T21:30:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8318999999999997e-05 2024-09-17T21:30:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728215+0000", 2024-09-17T21:30:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4172999999999997e-05 2024-09-17T21:30:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45653 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.578828628, 2024-09-17T21:30:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022249517999999999, 2024-09-17T21:30:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45653 2024-09-17T21:30:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T21:30:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706147+0000", 2024-09-17T21:30:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4430000000000002e-06 2024-09-17T21:30:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706152+0000", 2024-09-17T21:30:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5350000000000004e-06 2024-09-17T21:30:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706157+0000", 2024-09-17T21:30:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8420000000000004e-06 2024-09-17T21:30:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728246+0000", 2024-09-17T21:30:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022089237000000001 2024-09-17T21:30:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728309+0000", 2024-09-17T21:30:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2356999999999999e-05 2024-09-17T21:30:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:30:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4596000000000001e-05 2024-09-17T21:30:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45657 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 36864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.578797364, 2024-09-17T21:30:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022218252000000001, 2024-09-17T21:30:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45657 2024-09-17T21:30:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8299999999999997e-07 2024-09-17T21:30:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706178+0000", 2024-09-17T21:30:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3700000000000002e-06 2024-09-17T21:30:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706184+0000", 2024-09-17T21:30:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.144e-06 2024-09-17T21:30:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706190+0000", 2024-09-17T21:30:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8590000000000001e-06 2024-09-17T21:30:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728240+0000", 2024-09-17T21:30:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022049804999999999 2024-09-17T21:30:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728307+0000", 2024-09-17T21:30:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7477999999999993e-05 2024-09-17T21:30:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:30:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6112999999999994e-05 2024-09-17T21:30:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45658 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.578774244000002, 2024-09-17T21:30:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022376455, 2024-09-17T21:30:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45658 2024-09-17T21:30:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:30:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706200+0000", 2024-09-17T21:30:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:30:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706206+0000", 2024-09-17T21:30:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7039999999999999e-06 2024-09-17T21:30:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706211+0000", 2024-09-17T21:30:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9429999999999999e-06 2024-09-17T21:30:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728418+0000", 2024-09-17T21:30:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022206760999999998 2024-09-17T21:30:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728497+0000", 2024-09-17T21:30:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9493999999999995e-05 2024-09-17T21:30:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728574+0000", 2024-09-17T21:30:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7180000000000003e-05 2024-09-17T21:30:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45707 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 212992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.577070178, 2024-09-17T21:30:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022235550999999999, 2024-09-17T21:30:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45707 2024-09-17T21:30:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:30:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707905+0000", 2024-09-17T21:30:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2120000000000002e-06 2024-09-17T21:30:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707911+0000", 2024-09-17T21:30:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1929999999999998e-06 2024-09-17T21:30:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707915+0000", 2024-09-17T21:30:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:30:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729897+0000", 2024-09-17T21:30:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021981186 2024-09-17T21:30:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729944+0000", 2024-09-17T21:30:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571000000000002e-05 2024-09-17T21:30:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730138+0000", 2024-09-17T21:30:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000193451 2024-09-17T21:30:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45708 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.577050046, 2024-09-17T21:30:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022533742999999998, 2024-09-17T21:30:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45708 2024-09-17T21:30:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707923+0000", 2024-09-17T21:30:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:30:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707932+0000", 2024-09-17T21:30:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6430000000000007e-06 2024-09-17T21:30:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707939+0000", 2024-09-17T21:30:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3539999999999996e-06 2024-09-17T21:30:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707944+0000", 2024-09-17T21:30:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3249999999999998e-06 2024-09-17T21:30:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730176+0000", 2024-09-17T21:30:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022232393 2024-09-17T21:30:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730263+0000", 2024-09-17T21:30:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6638999999999995e-05 2024-09-17T21:30:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730456+0000", 2024-09-17T21:30:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019304899999999999 2024-09-17T21:30:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45752 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.576794826, 2024-09-17T21:30:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022426878000000001, 2024-09-17T21:30:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45752 2024-09-17T21:30:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708178+0000", 2024-09-17T21:30:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:30:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708181+0000", 2024-09-17T21:30:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.909e-06 2024-09-17T21:30:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708187+0000", 2024-09-17T21:30:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4999999999999996e-06 2024-09-17T21:30:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708193+0000", 2024-09-17T21:30:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:30:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730484+0000", 2024-09-17T21:30:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022291765000000002 2024-09-17T21:30:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730541+0000", 2024-09-17T21:30:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6483000000000001e-05 2024-09-17T21:30:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730604+0000", 2024-09-17T21:30:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3429000000000005e-05 2024-09-17T21:30:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45761 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.576766916, 2024-09-17T21:30:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022614408999999999, 2024-09-17T21:30:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45761 2024-09-17T21:30:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708206+0000", 2024-09-17T21:30:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4700000000000001e-07 2024-09-17T21:30:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708208+0000", 2024-09-17T21:30:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1509999999999998e-06 2024-09-17T21:30:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708214+0000", 2024-09-17T21:30:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.49e-06 2024-09-17T21:30:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708219+0000", 2024-09-17T21:30:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8119999999999998e-06 2024-09-17T21:30:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730641+0000", 2024-09-17T21:30:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022422765000000001 2024-09-17T21:30:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730731+0000", 2024-09-17T21:30:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9598999999999994e-05 2024-09-17T21:30:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730820+0000", 2024-09-17T21:30:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8844999999999994e-05 2024-09-17T21:30:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45778 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.576023312, 2024-09-17T21:30:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022018544000000001, 2024-09-17T21:30:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45778 2024-09-17T21:30:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708950+0000", 2024-09-17T21:30:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-17T21:30:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709307+0000", 2024-09-17T21:30:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000356848 2024-09-17T21:30:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709316+0000", 2024-09-17T21:30:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1430000000000004e-06 2024-09-17T21:30:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709328+0000", 2024-09-17T21:30:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2629999999999999e-05 2024-09-17T21:30:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730861+0000", 2024-09-17T21:30:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021532057 2024-09-17T21:30:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730905+0000", 2024-09-17T21:30:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4907000000000002e-05 2024-09-17T21:30:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730968+0000", 2024-09-17T21:30:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2108999999999998e-05 2024-09-17T21:30:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45785 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.575545128999998, 2024-09-17T21:30:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022044113000000001, 2024-09-17T21:30:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45785 2024-09-17T21:30:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709428+0000", 2024-09-17T21:30:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:30:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709430+0000", 2024-09-17T21:30:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:30:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709451+0000", 2024-09-17T21:30:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1372000000000001e-05 2024-09-17T21:30:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709472+0000", 2024-09-17T21:30:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1007999999999999e-05 2024-09-17T21:30:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731293+0000", 2024-09-17T21:30:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021821320000000002 2024-09-17T21:30:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731378+0000", 2024-09-17T21:30:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5059e-05 2024-09-17T21:30:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731471+0000", 2024-09-17T21:30:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2867000000000004e-05 2024-09-17T21:30:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45786 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.575486332000001, 2024-09-17T21:30:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022181821000000001, 2024-09-17T21:30:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45786 2024-09-17T21:30:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709487+0000", 2024-09-17T21:30:34.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1299999999999999e-07 2024-09-17T21:30:34.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709491+0000", 2024-09-17T21:30:34.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2089999999999999e-06 2024-09-17T21:30:34.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709498+0000", 2024-09-17T21:30:34.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3980000000000002e-06 2024-09-17T21:30:34.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709504+0000", 2024-09-17T21:30:34.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7219999999999996e-06 2024-09-17T21:30:34.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731499+0000", 2024-09-17T21:30:34.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021995297 2024-09-17T21:30:34.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731591+0000", 2024-09-17T21:30:34.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1373999999999996e-05 2024-09-17T21:30:34.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731668+0000", 2024-09-17T21:30:34.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7107999999999994e-05 2024-09-17T21:30:34.372 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.372 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.372 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.372 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50202 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'139, mlcod=45'137)", 2024-09-17T21:30:34.372 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:30:34.373 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.160750462999999, 2024-09-17T21:30:34.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059659441000000001, 2024-09-17T21:30:34.373 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.373 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.373 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:34.373 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:30:34.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:30:34.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124224+0000", 2024-09-17T21:30:34.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5000000000000002e-06 2024-09-17T21:30:34.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147385+0000", 2024-09-17T21:30:34.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023160544000000002 2024-09-17T21:30:34.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147411+0000", 2024-09-17T21:30:34.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6095999999999999e-05 2024-09-17T21:30:34.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147438+0000", 2024-09-17T21:30:34.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6982999999999999e-05 2024-09-17T21:30:34.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147504+0000", 2024-09-17T21:30:34.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6156999999999997e-05 2024-09-17T21:30:34.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147531+0000", 2024-09-17T21:30:34.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6741999999999999e-05 2024-09-17T21:30:34.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:34.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183820+0000", 2024-09-17T21:30:34.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036289090000000003 2024-09-17T21:30:34.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183881+0000", 2024-09-17T21:30:34.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1328999999999995e-05 2024-09-17T21:30:34.377 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.377 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.377 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.377 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50203 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'141, mlcod=45'137)", 2024-09-17T21:30:34.377 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:30:34.377 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.12348772, 2024-09-17T21:30:34.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023138823999999999, 2024-09-17T21:30:34.377 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.377 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.378 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:34.378 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:30:34.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:30:34.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161486+0000", 2024-09-17T21:30:34.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5290000000000001e-06 2024-09-17T21:30:34.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161496+0000", 2024-09-17T21:30:34.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0063e-05 2024-09-17T21:30:34.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161514+0000", 2024-09-17T21:30:34.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7728e-05 2024-09-17T21:30:34.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161524+0000", 2024-09-17T21:30:34.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0339e-05 2024-09-17T21:30:34.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170872+0000", 2024-09-17T21:30:34.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093473640000000004 2024-09-17T21:30:34.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170899+0000", 2024-09-17T21:30:34.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7321000000000002e-05 2024-09-17T21:30:34.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:34.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184580+0000", 2024-09-17T21:30:34.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013680988999999999 2024-09-17T21:30:34.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184623+0000", 2024-09-17T21:30:34.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3491000000000001e-05 2024-09-17T21:30:34.382 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.382 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.382 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.382 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.382 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.382 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:34.383 INFO:teuthology.orchestra.run.smithi129.stdout:3613 trunc from 0xcf4f8f8 to 0x39d85e8 2024-09-17T21:30:34.383 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:34.384 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:34.384 INFO:teuthology.orchestra.run.smithi129.stdout:3614 read 0x26a343d thru 0x26aba48 (0x860c bytes) 2024-09-17T21:30:34.384 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:34.385 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:34.385 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:30:34.386 INFO:teuthology.orchestra.run.smithi129.stdout:3615 writesame 0xd08bbee thru 0xd091971 data_size 0x85(0x5d84 bytes) 2024-09-17T21:30:34.386 INFO:teuthology.orchestra.run.smithi129.stdout:3616 punch from 0x5cf7d6b to 0x5d00bdd, (0x8e72 bytes) 2024-09-17T21:30:34.386 INFO:teuthology.orchestra.run.smithi129.stdout:3617 read 0x4627864 thru 0x4635b33 (0xe2d0 bytes) 2024-09-17T21:30:34.386 INFO:teuthology.orchestra.run.smithi129.stdout:3618 writesame 0xc78003e thru 0xc78f58d data_size 0x9d(0xf550 bytes) 2024-09-17T21:30:34.386 INFO:teuthology.orchestra.run.smithi129.stdout:3620 read 0x3a5c22d thru 0x3a694cd (0xd2a1 bytes) 2024-09-17T21:30:34.387 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:34.387 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:34.387 INFO:teuthology.orchestra.run.smithi129.stdout:3622 trunc from 0xd091972 to 0x5574efb 2024-09-17T21:30:34.389 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:34.389 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:34.389 INFO:teuthology.orchestra.run.smithi129.stdout:3626 writesame 0x9df414c thru 0x9dff06d data_size 0x1d(0xaf22 bytes) 2024-09-17T21:30:34.389 INFO:teuthology.orchestra.run.smithi129.stdout:3628 writesame 0xc139c2d thru 0xc1422c2 data_size 0x6b(0x8696 bytes) 2024-09-17T21:30:34.389 INFO:teuthology.orchestra.run.smithi129.stdout:3629 writesame 0x4a4d8ae thru 0x4a50116 data_size 0x1(0x2869 bytes) 2024-09-17T21:30:34.389 INFO:teuthology.orchestra.run.smithi129.stdout:3631 write 0x7fa9b7e thru 0x7fb9aac (0xff2f bytes) 2024-09-17T21:30:34.389 INFO:teuthology.orchestra.run.smithi129.stdout:3632 write 0xa490efb thru 0xa49bfb9 (0xb0bf bytes) 2024-09-17T21:30:34.389 INFO:teuthology.orchestra.run.smithi129.stdout:3633 read 0x53b6627 thru 0x53b730a (0xce4 bytes) 2024-09-17T21:30:34.389 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:34.390 INFO:teuthology.orchestra.run.smithi129.stdout:3636 punch from 0xc42201 to 0xc4d528, (0xb327 bytes) 2024-09-17T21:30:34.390 INFO:teuthology.orchestra.run.smithi129.stdout:3637 read 0x66359c4 thru 0x6644a37 (0xf074 bytes) 2024-09-17T21:30:34.390 INFO:teuthology.orchestra.run.smithi129.stdout:3639 read 0x88fb709 thru 0x8908d2b (0xd623 bytes) 2024-09-17T21:30:34.390 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:34.381+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:34.390 INFO:teuthology.orchestra.run.smithi129.stdout:3640 trunc from 0xc1422c3 to 0x9994452 2024-09-17T21:30:34.402 INFO:teuthology.orchestra.run.smithi129.stdout:3641 trunc from 0x9994452 to 0x33e1c28 2024-09-17T21:30:34.411 INFO:teuthology.orchestra.run.smithi129.stdout:3643 read 0x2c53680 thru 0x2c56570 (0x2ef1 bytes) 2024-09-17T21:30:34.411 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:34.411 INFO:teuthology.orchestra.run.smithi129.stdout:3645 read 0x2fb654 thru 0x3068c7 (0xb274 bytes) 2024-09-17T21:30:34.412 INFO:teuthology.orchestra.run.smithi129.stdout:3646 read 0x1d74f41 thru 0x1d7795f (0x2a1f bytes) 2024-09-17T21:30:34.427 INFO:teuthology.orchestra.run.smithi129.stdout:3647 write 0xa1acd30 thru 0xa1b54af (0x8780 bytes) 2024-09-17T21:30:34.443 INFO:teuthology.orchestra.run.smithi129.stdout:3648 trunc from 0xa1b54b0 to 0xbbf79c 2024-09-17T21:30:34.461 INFO:teuthology.orchestra.run.smithi129.stdout:3649 write 0x1b95637 thru 0x1ba4847 (0xf211 bytes) 2024-09-17T21:30:34.465 INFO:teuthology.orchestra.run.smithi129.stdout:3650 punch from 0x1320929 to 0x132a04f, (0x9726 bytes) 2024-09-17T21:30:34.479 INFO:teuthology.orchestra.run.smithi129.stdout:3651 trunc from 0x1ba4848 to 0xd4230ff 2024-09-17T21:30:34.484 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:34.481+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:34.484 INFO:teuthology.orchestra.run.smithi129.stdout:3652 write 0xc617754 thru 0xc61af36 (0x37e3 bytes) 2024-09-17T21:30:34.488 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:34.488 INFO:teuthology.orchestra.run.smithi129.stdout:3654 write 0x49240c3 thru 0x4930874 (0xc7b2 bytes) 2024-09-17T21:30:34.492 INFO:teuthology.orchestra.run.smithi129.stdout:3655 read 0x562091b thru 0x5621dbf (0x14a5 bytes) 2024-09-17T21:30:34.492 INFO:teuthology.orchestra.run.smithi129.stdout:3657 read 0xd4183a2 thru 0xd420600 (0x825f bytes) 2024-09-17T21:30:34.492 INFO:teuthology.orchestra.run.smithi129.stdout:3658 read 0x2b4b3eb thru 0x2b51bdd (0x67f3 bytes) 2024-09-17T21:30:34.494 INFO:teuthology.orchestra.run.smithi129.stdout:3659 trunc from 0xd4230ff to 0xb774e5b 2024-09-17T21:30:34.503 INFO:teuthology.orchestra.run.smithi129.stdout:3662 clone 29 order 21 su 16384 sc 13 2024-09-17T21:30:34.536 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:34.536 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:34.536 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:34.536 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:34.544 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:30:34.588 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:34.585+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:34.649 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:34.649 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:34.649 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:34.649 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:34.649 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:34.657 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:30:34.698 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:34.695+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:34.776 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is unset 2024-09-17T21:30:34.787 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:34.784+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:34.805 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10799 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:30:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.737452167999997, 2024-09-17T21:30:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039122745, 2024-09-17T21:30:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10799 2024-09-17T21:30:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:30:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:30:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065469+0000", 2024-09-17T21:30:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:30:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065477+0000", 2024-09-17T21:30:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1319999999999994e-06 2024-09-17T21:30:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065483+0000", 2024-09-17T21:30:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2040000000000002e-06 2024-09-17T21:30:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065487+0000", 2024-09-17T21:30:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.42e-06 2024-09-17T21:30:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099984+0000", 2024-09-17T21:30:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034496484000000001 2024-09-17T21:30:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100065+0000", 2024-09-17T21:30:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1200999999999997e-05 2024-09-17T21:30:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100422+0000", 2024-09-17T21:30:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035657700000000001 2024-09-17T21:30:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104214+0000", 2024-09-17T21:30:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003792045 2024-09-17T21:30:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104505+0000", 2024-09-17T21:30:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029141399999999999 2024-09-17T21:30:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104556+0000", 2024-09-17T21:30:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1388999999999999e-05 2024-09-17T21:30:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104591+0000", 2024-09-17T21:30:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4455999999999997e-05 2024-09-17T21:30:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12187 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.663359436999997, 2024-09-17T21:30:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038825750999999999, 2024-09-17T21:30:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12187 2024-09-17T21:30:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:30:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139563+0000", 2024-09-17T21:30:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.624e-06 2024-09-17T21:30:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139569+0000", 2024-09-17T21:30:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1090000000000003e-06 2024-09-17T21:30:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139573+0000", 2024-09-17T21:30:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4429999999999996e-06 2024-09-17T21:30:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178206+0000", 2024-09-17T21:30:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038632376000000003 2024-09-17T21:30:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178282+0000", 2024-09-17T21:30:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6076999999999997e-05 2024-09-17T21:30:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178387+0000", 2024-09-17T21:30:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104682 2024-09-17T21:30:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12188 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:30:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.663341256000002, 2024-09-17T21:30:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039053826999999999, 2024-09-17T21:30:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12188 2024-09-17T21:30:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:30:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:30:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139746+0000", 2024-09-17T21:30:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000167281 2024-09-17T21:30:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139749+0000", 2024-09-17T21:30:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9660000000000001e-06 2024-09-17T21:30:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139757+0000", 2024-09-17T21:30:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8029999999999997e-06 2024-09-17T21:30:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139771+0000", 2024-09-17T21:30:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3541e-05 2024-09-17T21:30:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178415+0000", 2024-09-17T21:30:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038644803999999998 2024-09-17T21:30:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178517+0000", 2024-09-17T21:30:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101298 2024-09-17T21:30:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178633+0000", 2024-09-17T21:30:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116134 2024-09-17T21:30:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12189 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:30:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.663140779000003, 2024-09-17T21:30:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039135991000000002, 2024-09-17T21:30:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12189 2024-09-17T21:30:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:30:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:30:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139780+0000", 2024-09-17T21:30:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:30:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139782+0000", 2024-09-17T21:30:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1950000000000002e-06 2024-09-17T21:30:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139788+0000", 2024-09-17T21:30:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0639999999999999e-06 2024-09-17T21:30:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139792+0000", 2024-09-17T21:30:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1960000000000001e-06 2024-09-17T21:30:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178671+0000", 2024-09-17T21:30:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038878176 2024-09-17T21:30:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178778+0000", 2024-09-17T21:30:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107278 2024-09-17T21:30:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178915+0000", 2024-09-17T21:30:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013757900000000001 2024-09-17T21:30:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12190 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.663121596000003, 2024-09-17T21:30:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039343781000000001, 2024-09-17T21:30:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12190 2024-09-17T21:30:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139801+0000", 2024-09-17T21:30:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.916e-06 2024-09-17T21:30:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139807+0000", 2024-09-17T21:30:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4540000000000004e-06 2024-09-17T21:30:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139812+0000", 2024-09-17T21:30:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9729999999999997e-06 2024-09-17T21:30:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178960+0000", 2024-09-17T21:30:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039147567000000001 2024-09-17T21:30:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179051+0000", 2024-09-17T21:30:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0754000000000006e-05 2024-09-17T21:30:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179142+0000", 2024-09-17T21:30:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1712999999999994e-05 2024-09-17T21:30:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12191 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.663101738000002, 2024-09-17T21:30:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039464336000000003, 2024-09-17T21:30:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12191 2024-09-17T21:30:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:30:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139821+0000", 2024-09-17T21:30:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1260000000000001e-06 2024-09-17T21:30:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139827+0000", 2024-09-17T21:30:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T21:30:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139840+0000", 2024-09-17T21:30:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2607e-05 2024-09-17T21:30:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179162+0000", 2024-09-17T21:30:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039322863999999999 2024-09-17T21:30:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179204+0000", 2024-09-17T21:30:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1938999999999999e-05 2024-09-17T21:30:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179283+0000", 2024-09-17T21:30:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8547000000000005e-05 2024-09-17T21:30:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12192 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:30:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.663074025999997, 2024-09-17T21:30:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039652318999999998, 2024-09-17T21:30:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12192 2024-09-17T21:30:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:30:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:30:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141458+0000", 2024-09-17T21:30:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001612098 2024-09-17T21:30:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141469+0000", 2024-09-17T21:30:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-17T21:30:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141478+0000", 2024-09-17T21:30:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T21:30:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141485+0000", 2024-09-17T21:30:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4760000000000003e-06 2024-09-17T21:30:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179317+0000", 2024-09-17T21:30:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037831832000000003 2024-09-17T21:30:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179395+0000", 2024-09-17T21:30:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8349000000000005e-05 2024-09-17T21:30:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179499+0000", 2024-09-17T21:30:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103628 2024-09-17T21:30:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12257 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.660885217999997, 2024-09-17T21:30:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038862751000000001, 2024-09-17T21:30:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12257 2024-09-17T21:30:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T21:30:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142037+0000", 2024-09-17T21:30:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.03e-06 2024-09-17T21:30:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142043+0000", 2024-09-17T21:30:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6319999999999996e-06 2024-09-17T21:30:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142052+0000", 2024-09-17T21:30:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2709999999999998e-06 2024-09-17T21:30:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180794+0000", 2024-09-17T21:30:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038741758000000001 2024-09-17T21:30:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180843+0000", 2024-09-17T21:30:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9021999999999999e-05 2024-09-17T21:30:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180898+0000", 2024-09-17T21:30:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673000000000003e-05 2024-09-17T21:30:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12258 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.660861701000002, 2024-09-17T21:30:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038950717000000003, 2024-09-17T21:30:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12258 2024-09-17T21:30:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199999999999999e-07 2024-09-17T21:30:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142061+0000", 2024-09-17T21:30:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8279999999999999e-06 2024-09-17T21:30:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142067+0000", 2024-09-17T21:30:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5600000000000001e-06 2024-09-17T21:30:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142071+0000", 2024-09-17T21:30:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6240000000000001e-06 2024-09-17T21:30:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180921+0000", 2024-09-17T21:30:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038849366000000003 2024-09-17T21:30:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180961+0000", 2024-09-17T21:30:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0219000000000002e-05 2024-09-17T21:30:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181009+0000", 2024-09-17T21:30:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8437999999999999e-05 2024-09-17T21:30:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12259 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.660834454000003, 2024-09-17T21:30:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039116832999999997, 2024-09-17T21:30:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.850 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.850 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.850 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12259 2024-09-17T21:30:34.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.850 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:34.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:34.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:34.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2200000000000004e-07 2024-09-17T21:30:34.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142088+0000", 2024-09-17T21:30:34.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0329999999999998e-06 2024-09-17T21:30:34.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142095+0000", 2024-09-17T21:30:34.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4320000000000004e-06 2024-09-17T21:30:34.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142100+0000", 2024-09-17T21:30:34.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:30:34.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181037+0000", 2024-09-17T21:30:34.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038937601000000002 2024-09-17T21:30:34.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181114+0000", 2024-09-17T21:30:34.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6857e-05 2024-09-17T21:30:34.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181203+0000", 2024-09-17T21:30:34.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8437999999999996e-05 2024-09-17T21:30:34.853 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.853 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.853 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.853 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12260 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.853 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:30:34.854 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.660814166999998, 2024-09-17T21:30:34.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039204184000000003, 2024-09-17T21:30:34.854 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.854 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.854 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.854 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12260 2024-09-17T21:30:34.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.854 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:30:34.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:30:34.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142107+0000", 2024-09-17T21:30:34.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:30:34.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142108+0000", 2024-09-17T21:30:34.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.753e-06 2024-09-17T21:30:34.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142114+0000", 2024-09-17T21:30:34.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7660000000000004e-06 2024-09-17T21:30:34.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142119+0000", 2024-09-17T21:30:34.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:30:34.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181216+0000", 2024-09-17T21:30:34.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039097191000000003 2024-09-17T21:30:34.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181260+0000", 2024-09-17T21:30:34.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4731000000000003e-05 2024-09-17T21:30:34.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181310+0000", 2024-09-17T21:30:34.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9818000000000003e-05 2024-09-17T21:30:34.857 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.857 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.857 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.858 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12261 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.858 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:30:34.858 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.660795843000002, 2024-09-17T21:30:34.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039292200999999999, 2024-09-17T21:30:34.858 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.858 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.858 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.858 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.858 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.858 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.858 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12261 2024-09-17T21:30:34.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.859 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:30:34.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:30:34.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142125+0000", 2024-09-17T21:30:34.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:30:34.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142127+0000", 2024-09-17T21:30:34.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.792e-06 2024-09-17T21:30:34.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142132+0000", 2024-09-17T21:30:34.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3979999999999998e-06 2024-09-17T21:30:34.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142136+0000", 2024-09-17T21:30:34.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.211e-06 2024-09-17T21:30:34.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181324+0000", 2024-09-17T21:30:34.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039187560000000003 2024-09-17T21:30:34.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181363+0000", 2024-09-17T21:30:34.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9375000000000002e-05 2024-09-17T21:30:34.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181417+0000", 2024-09-17T21:30:34.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3372000000000003e-05 2024-09-17T21:30:34.862 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.862 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.862 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.862 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12262 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.862 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:30:34.862 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.660778049999998, 2024-09-17T21:30:34.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039384875, 2024-09-17T21:30:34.862 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.862 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.862 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.863 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.863 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.863 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.863 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12262 2024-09-17T21:30:34.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.863 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:30:34.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:30:34.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142143+0000", 2024-09-17T21:30:34.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:30:34.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142148+0000", 2024-09-17T21:30:34.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4360000000000001e-06 2024-09-17T21:30:34.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142154+0000", 2024-09-17T21:30:34.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8719999999999999e-06 2024-09-17T21:30:34.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142158+0000", 2024-09-17T21:30:34.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.476e-06 2024-09-17T21:30:34.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181436+0000", 2024-09-17T21:30:34.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039277919000000001 2024-09-17T21:30:34.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181480+0000", 2024-09-17T21:30:34.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3649999999999997e-05 2024-09-17T21:30:34.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181527+0000", 2024-09-17T21:30:34.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7131e-05 2024-09-17T21:30:34.866 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.866 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.866 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.867 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12263 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1079296~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.867 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:34.867 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.660755614999999, 2024-09-17T21:30:34.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039473339000000003, 2024-09-17T21:30:34.867 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.867 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.867 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.867 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.867 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.867 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.867 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12263 2024-09-17T21:30:34.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.868 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:34.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:34.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:34.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:30:34.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142167+0000", 2024-09-17T21:30:34.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.81e-06 2024-09-17T21:30:34.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142172+0000", 2024-09-17T21:30:34.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:30:34.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142177+0000", 2024-09-17T21:30:34.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T21:30:34.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181553+0000", 2024-09-17T21:30:34.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039376130000000002 2024-09-17T21:30:34.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181592+0000", 2024-09-17T21:30:34.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9153999999999998e-05 2024-09-17T21:30:34.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181638+0000", 2024-09-17T21:30:34.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5864e-05 2024-09-17T21:30:34.871 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.871 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.871 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.871 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12278 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1095680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.872 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:34.872 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.660737640000001, 2024-09-17T21:30:34.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039454222999999997, 2024-09-17T21:30:34.872 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.872 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.872 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.872 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.872 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.872 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.872 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12278 2024-09-17T21:30:34.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.872 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:34.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:34.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:34.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:30:34.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142185+0000", 2024-09-17T21:30:34.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.697e-06 2024-09-17T21:30:34.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142191+0000", 2024-09-17T21:30:34.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:30:34.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142196+0000", 2024-09-17T21:30:34.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4829999999999999e-06 2024-09-17T21:30:34.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181548+0000", 2024-09-17T21:30:34.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039351200000000003 2024-09-17T21:30:34.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181591+0000", 2024-09-17T21:30:34.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3492000000000003e-05 2024-09-17T21:30:34.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181637+0000", 2024-09-17T21:30:34.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5847000000000003e-05 2024-09-17T21:30:34.876 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.876 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.876 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.876 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12279 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.876 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:34.876 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.660714351000003, 2024-09-17T21:30:34.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039539801999999999, 2024-09-17T21:30:34.876 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.877 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.877 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.877 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.877 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.877 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.877 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12279 2024-09-17T21:30:34.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.877 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:34.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:34.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:34.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.89e-07 2024-09-17T21:30:34.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142208+0000", 2024-09-17T21:30:34.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7579999999999999e-06 2024-09-17T21:30:34.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142214+0000", 2024-09-17T21:30:34.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:30:34.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142218+0000", 2024-09-17T21:30:34.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3429999999999998e-06 2024-09-17T21:30:34.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181653+0000", 2024-09-17T21:30:34.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039435322000000002 2024-09-17T21:30:34.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181698+0000", 2024-09-17T21:30:34.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4978999999999997e-05 2024-09-17T21:30:34.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181746+0000", 2024-09-17T21:30:34.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571999999999998e-05 2024-09-17T21:30:34.880 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.880 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.880 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.881 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12280 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1124352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.881 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:30:34.881 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.660696831999999, 2024-09-17T21:30:34.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039704358000000002, 2024-09-17T21:30:34.881 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.881 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.881 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.881 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.881 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.881 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.881 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12280 2024-09-17T21:30:34.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.881 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:30:34.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:30:34.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142224+0000", 2024-09-17T21:30:34.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:30:34.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142226+0000", 2024-09-17T21:30:34.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:30:34.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142231+0000", 2024-09-17T21:30:34.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:30:34.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142240+0000", 2024-09-17T21:30:34.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4590000000000004e-06 2024-09-17T21:30:34.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181774+0000", 2024-09-17T21:30:34.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039533833999999997 2024-09-17T21:30:34.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181859+0000", 2024-09-17T21:30:34.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4328999999999998e-05 2024-09-17T21:30:34.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181928+0000", 2024-09-17T21:30:34.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9139000000000003e-05 2024-09-17T21:30:34.885 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.885 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.885 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.885 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12281 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1136640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.885 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:30:34.885 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.660673926000001, 2024-09-17T21:30:34.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039787799999999998, 2024-09-17T21:30:34.885 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.885 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.885 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.885 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.886 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.886 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.886 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12281 2024-09-17T21:30:34.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.886 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:30:34.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:30:34.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142309+0000", 2024-09-17T21:30:34.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2323999999999995e-05 2024-09-17T21:30:34.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142317+0000", 2024-09-17T21:30:34.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1859999999999992e-06 2024-09-17T21:30:34.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142324+0000", 2024-09-17T21:30:34.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4699999999999996e-06 2024-09-17T21:30:34.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142330+0000", 2024-09-17T21:30:34.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4650000000000002e-06 2024-09-17T21:30:34.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181948+0000", 2024-09-17T21:30:34.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039618056999999998 2024-09-17T21:30:34.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181986+0000", 2024-09-17T21:30:34.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8442999999999998e-05 2024-09-17T21:30:34.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182034+0000", 2024-09-17T21:30:34.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7855000000000002e-05 2024-09-17T21:30:34.889 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.889 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.889 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.889 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12282 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1150976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.889 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:34.890 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.660581696999998, 2024-09-17T21:30:34.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039871919999999998, 2024-09-17T21:30:34.890 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.890 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.890 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:34.890 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.890 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.890 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.890 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12282 2024-09-17T21:30:34.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.890 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:34.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:34.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:34.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:30:34.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142341+0000", 2024-09-17T21:30:34.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.232e-06 2024-09-17T21:30:34.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142348+0000", 2024-09-17T21:30:34.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7320000000000001e-06 2024-09-17T21:30:34.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142353+0000", 2024-09-17T21:30:34.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:30:34.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182062+0000", 2024-09-17T21:30:34.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039709739000000001 2024-09-17T21:30:34.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:34.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182128+0000", 2024-09-17T21:30:34.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5739000000000002e-05 2024-09-17T21:30:34.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182210+0000", 2024-09-17T21:30:34.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:30:34.894 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.894 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.894 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.894 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12283 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:34.894 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:30:34.894 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.660561285999997, 2024-09-17T21:30:34.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039896434000000001, 2024-09-17T21:30:34.894 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:34.894 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:34.894 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:30:34.894 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:34.894 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:34.895 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:34.895 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12283 2024-09-17T21:30:34.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.895 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:34.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:34.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:30:34.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:34.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:30:34.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:34.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:34.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144852+0000", 2024-09-17T21:30:34.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024934079999999999 2024-09-17T21:30:34.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:34.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144856+0000", 2024-09-17T21:30:34.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:30:34.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:34.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144864+0000", 2024-09-17T21:30:34.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0160000000000008e-06 2024-09-17T21:30:34.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:34.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144869+0000", 2024-09-17T21:30:34.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:30:34.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:34.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182246+0000", 2024-09-17T21:30:34.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037377207000000003 2024-09-17T21:30:34.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:34.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:34.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:34.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182255+0000", 2024-09-17T21:30:34.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1840000000000005e-06 2024-09-17T21:30:34.897 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.898 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.898 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.898 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:34.898 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:34.898 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:34.899 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone28 from 0xb774e5b (overlap 0xbbf79c) to 0x965fb0 2024-09-17T21:30:34.899 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:34.896+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:34.900 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:30:34.940 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-17T21:30:35.001 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:34.997+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:35.071 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:35.071 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:35.071 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:35.071 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:35.081 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:30:35.102 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:35.099+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:35.186 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:35.187 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:35.187 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:35.187 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:35.187 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:35.195 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:30:35.202 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:35.199+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:35.284 INFO:teuthology.orchestra.run.smithi129.stdout:3663 write 0x8bb02b6 thru 0x8bb4684 (0x43cf bytes) 2024-09-17T21:30:35.293 INFO:teuthology.orchestra.run.smithi129.stdout:3664 read 0x1021e7c thru 0x1022897 (0xa1c bytes) 2024-09-17T21:30:35.293 INFO:teuthology.orchestra.run.smithi129.stdout:3665 punch from 0xada79bc to 0xadabbc0, (0x4204 bytes) 2024-09-17T21:30:35.294 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.294 INFO:teuthology.orchestra.run.smithi129.stdout:3668 read 0x4200573 thru 0x4206b14 (0x65a2 bytes) 2024-09-17T21:30:35.294 INFO:teuthology.orchestra.run.smithi129.stdout:3669 read 0x1eec99e thru 0x1eed063 (0x6c6 bytes) 2024-09-17T21:30:35.294 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.300 INFO:teuthology.orchestra.run.smithi129.stdout:3672 write 0xc0831dd thru 0xc092466 (0xf28a bytes) 2024-09-17T21:30:35.303 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:36935 3.a 3:54c8759b:::rbd_data.1120e62eea89.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-17T21:30:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.340703091999998, 2024-09-17T21:30:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020540151999999999, 2024-09-17T21:30:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 36935 2024-09-17T21:30:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958910+0000", 2024-09-17T21:30:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:30:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958918+0000", 2024-09-17T21:30:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6469999999999998e-06 2024-09-17T21:30:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958933+0000", 2024-09-17T21:30:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4791e-05 2024-09-17T21:30:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958940+0000", 2024-09-17T21:30:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-17T21:30:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958990+0000", 2024-09-17T21:30:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9759000000000003e-05 2024-09-17T21:30:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.959039+0000", 2024-09-17T21:30:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9487000000000002e-05 2024-09-17T21:30:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.962098+0000", 2024-09-17T21:30:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003059258 2024-09-17T21:30:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979325+0000", 2024-09-17T21:30:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017226841 2024-09-17T21:30:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979343+0000", 2024-09-17T21:30:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7486000000000002e-05 2024-09-17T21:30:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979401+0000", 2024-09-17T21:30:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7980999999999999e-05 2024-09-17T21:30:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979449+0000", 2024-09-17T21:30:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8639e-05 2024-09-17T21:30:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39447 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 909312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331765191999999, 2024-09-17T21:30:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017419153999999999, 2024-09-17T21:30:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39447 2024-09-17T21:30:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:30:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967849+0000", 2024-09-17T21:30:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4339999999999999e-06 2024-09-17T21:30:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967853+0000", 2024-09-17T21:30:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:30:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967857+0000", 2024-09-17T21:30:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:30:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985193+0000", 2024-09-17T21:30:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017335973000000001 2024-09-17T21:30:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985228+0000", 2024-09-17T21:30:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5154999999999998e-05 2024-09-17T21:30:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985266+0000", 2024-09-17T21:30:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8448000000000002e-05 2024-09-17T21:30:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39448 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 974848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331752375999997, 2024-09-17T21:30:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017517066000000001, 2024-09-17T21:30:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39448 2024-09-17T21:30:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.72e-07 2024-09-17T21:30:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967861+0000", 2024-09-17T21:30:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.096e-06 2024-09-17T21:30:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967865+0000", 2024-09-17T21:30:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9339999999999999e-06 2024-09-17T21:30:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967868+0000", 2024-09-17T21:30:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.508e-06 2024-09-17T21:30:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985285+0000", 2024-09-17T21:30:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017417208 2024-09-17T21:30:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985339+0000", 2024-09-17T21:30:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3931000000000003e-05 2024-09-17T21:30:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985377+0000", 2024-09-17T21:30:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8016999999999999e-05 2024-09-17T21:30:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39449 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331740265999997, 2024-09-17T21:30:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017595137, 2024-09-17T21:30:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39449 2024-09-17T21:30:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2700000000000001e-07 2024-09-17T21:30:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967873+0000", 2024-09-17T21:30:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.164e-06 2024-09-17T21:30:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967877+0000", 2024-09-17T21:30:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.23e-06 2024-09-17T21:30:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967880+0000", 2024-09-17T21:30:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.286e-06 2024-09-17T21:30:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985397+0000", 2024-09-17T21:30:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017516982 2024-09-17T21:30:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985431+0000", 2024-09-17T21:30:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3899999999999997e-05 2024-09-17T21:30:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985467+0000", 2024-09-17T21:30:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6347999999999998e-05 2024-09-17T21:30:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39450 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1105920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331720009000001, 2024-09-17T21:30:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017677824000000002, 2024-09-17T21:30:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39450 2024-09-17T21:30:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967893+0000", 2024-09-17T21:30:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9100000000000004e-07 2024-09-17T21:30:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967895+0000", 2024-09-17T21:30:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:30:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967901+0000", 2024-09-17T21:30:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.083e-06 2024-09-17T21:30:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967906+0000", 2024-09-17T21:30:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:30:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985493+0000", 2024-09-17T21:30:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017587565999999999 2024-09-17T21:30:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985531+0000", 2024-09-17T21:30:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7490000000000002e-05 2024-09-17T21:30:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985570+0000", 2024-09-17T21:30:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9205000000000002e-05 2024-09-17T21:30:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39451 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1171456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331699628999999, 2024-09-17T21:30:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017752337999999999, 2024-09-17T21:30:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39451 2024-09-17T21:30:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:30:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967915+0000", 2024-09-17T21:30:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780000000000001e-06 2024-09-17T21:30:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967922+0000", 2024-09-17T21:30:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7020000000000004e-06 2024-09-17T21:30:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967926+0000", 2024-09-17T21:30:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.809e-06 2024-09-17T21:30:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985590+0000", 2024-09-17T21:30:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017663913999999999 2024-09-17T21:30:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985627+0000", 2024-09-17T21:30:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6896e-05 2024-09-17T21:30:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985665+0000", 2024-09-17T21:30:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7793999999999998e-05 2024-09-17T21:30:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39452 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1236992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331679686999998, 2024-09-17T21:30:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017829602999999999, 2024-09-17T21:30:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39452 2024-09-17T21:30:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:30:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967947+0000", 2024-09-17T21:30:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4280999999999999e-05 2024-09-17T21:30:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967954+0000", 2024-09-17T21:30:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3249999999999996e-06 2024-09-17T21:30:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967959+0000", 2024-09-17T21:30:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2299999999999999e-06 2024-09-17T21:30:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985687+0000", 2024-09-17T21:30:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017727844999999999 2024-09-17T21:30:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985725+0000", 2024-09-17T21:30:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8243e-05 2024-09-17T21:30:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985762+0000", 2024-09-17T21:30:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7023000000000001e-05 2024-09-17T21:30:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39453 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331643198999998, 2024-09-17T21:30:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017893793000000002, 2024-09-17T21:30:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39453 2024-09-17T21:30:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967970+0000", 2024-09-17T21:30:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:30:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967972+0000", 2024-09-17T21:30:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0229999999999999e-06 2024-09-17T21:30:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967978+0000", 2024-09-17T21:30:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0430000000000004e-06 2024-09-17T21:30:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967982+0000", 2024-09-17T21:30:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2570000000000001e-06 2024-09-17T21:30:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985782+0000", 2024-09-17T21:30:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017799683 2024-09-17T21:30:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985818+0000", 2024-09-17T21:30:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6304999999999999e-05 2024-09-17T21:30:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985863+0000", 2024-09-17T21:30:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4953999999999997e-05 2024-09-17T21:30:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39454 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1368064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331623870999998, 2024-09-17T21:30:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017968886, 2024-09-17T21:30:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39454 2024-09-17T21:30:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967989+0000", 2024-09-17T21:30:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.15e-07 2024-09-17T21:30:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967991+0000", 2024-09-17T21:30:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.945e-06 2024-09-17T21:30:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967997+0000", 2024-09-17T21:30:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0190000000000003e-06 2024-09-17T21:30:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968001+0000", 2024-09-17T21:30:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6650000000000002e-06 2024-09-17T21:30:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985883+0000", 2024-09-17T21:30:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017881971999999999 2024-09-17T21:30:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985920+0000", 2024-09-17T21:30:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-05 2024-09-17T21:30:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985957+0000", 2024-09-17T21:30:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7370000000000003e-05 2024-09-17T21:30:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39455 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331604697000003, 2024-09-17T21:30:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018044437999999999, 2024-09-17T21:30:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39455 2024-09-17T21:30:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:30:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968010+0000", 2024-09-17T21:30:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.886e-06 2024-09-17T21:30:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968017+0000", 2024-09-17T21:30:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:30:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968021+0000", 2024-09-17T21:30:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850000000000004e-06 2024-09-17T21:30:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985977+0000", 2024-09-17T21:30:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017956368 2024-09-17T21:30:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986015+0000", 2024-09-17T21:30:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7610000000000001e-05 2024-09-17T21:30:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986052+0000", 2024-09-17T21:30:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7109e-05 2024-09-17T21:30:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39456 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1499136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331584397, 2024-09-17T21:30:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018114514000000002, 2024-09-17T21:30:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39456 2024-09-17T21:30:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-17T21:30:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968030+0000", 2024-09-17T21:30:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0049999999999999e-06 2024-09-17T21:30:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968036+0000", 2024-09-17T21:30:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0129999999999999e-06 2024-09-17T21:30:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968041+0000", 2024-09-17T21:30:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8300000000000003e-06 2024-09-17T21:30:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986071+0000", 2024-09-17T21:30:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018029620999999999 2024-09-17T21:30:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986106+0000", 2024-09-17T21:30:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4867999999999999e-05 2024-09-17T21:30:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986142+0000", 2024-09-17T21:30:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6566000000000002e-05 2024-09-17T21:30:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39457 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1564672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331564581000002, 2024-09-17T21:30:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018188644, 2024-09-17T21:30:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39457 2024-09-17T21:30:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7199999999999999e-07 2024-09-17T21:30:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968050+0000", 2024-09-17T21:30:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1110000000000002e-06 2024-09-17T21:30:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968057+0000", 2024-09-17T21:30:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6170000000000003e-06 2024-09-17T21:30:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968062+0000", 2024-09-17T21:30:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:30:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986163+0000", 2024-09-17T21:30:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018101435999999999 2024-09-17T21:30:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986199+0000", 2024-09-17T21:30:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.591e-05 2024-09-17T21:30:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986236+0000", 2024-09-17T21:30:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7416999999999998e-05 2024-09-17T21:30:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39458 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1630208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331537640999997, 2024-09-17T21:30:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018255856000000001, 2024-09-17T21:30:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39458 2024-09-17T21:30:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:30:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968077+0000", 2024-09-17T21:30:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.779e-06 2024-09-17T21:30:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968081+0000", 2024-09-17T21:30:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.532e-06 2024-09-17T21:30:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968085+0000", 2024-09-17T21:30:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999999e-06 2024-09-17T21:30:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986256+0000", 2024-09-17T21:30:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018171273000000002 2024-09-17T21:30:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986293+0000", 2024-09-17T21:30:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6534e-05 2024-09-17T21:30:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986330+0000", 2024-09-17T21:30:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7536000000000001e-05 2024-09-17T21:30:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39459 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1695744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331523062999999, 2024-09-17T21:30:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334493, 2024-09-17T21:30:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39459 2024-09-17T21:30:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968090+0000", 2024-09-17T21:30:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:30:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968091+0000", 2024-09-17T21:30:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3319999999999999e-06 2024-09-17T21:30:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968094+0000", 2024-09-17T21:30:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2590000000000001e-06 2024-09-17T21:30:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968097+0000", 2024-09-17T21:30:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:30:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986350+0000", 2024-09-17T21:30:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018253364000000001 2024-09-17T21:30:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986387+0000", 2024-09-17T21:30:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6350999999999998e-05 2024-09-17T21:30:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986424+0000", 2024-09-17T21:30:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6897999999999998e-05 2024-09-17T21:30:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39460 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1761280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331511927000001, 2024-09-17T21:30:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018415918999999999, 2024-09-17T21:30:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39460 2024-09-17T21:30:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968101+0000", 2024-09-17T21:30:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T21:30:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968102+0000", 2024-09-17T21:30:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1689999999999999e-06 2024-09-17T21:30:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968105+0000", 2024-09-17T21:30:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:30:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968108+0000", 2024-09-17T21:30:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6630000000000002e-06 2024-09-17T21:30:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986442+0000", 2024-09-17T21:30:35.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334487 2024-09-17T21:30:35.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986479+0000", 2024-09-17T21:30:35.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6557000000000003e-05 2024-09-17T21:30:35.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986516+0000", 2024-09-17T21:30:35.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7648999999999998e-05 2024-09-17T21:30:35.370 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.370 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.370 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.371 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39461 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1826816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.371 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:35.371 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331501166999999, 2024-09-17T21:30:35.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018499768, 2024-09-17T21:30:35.371 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.371 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.371 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.371 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39461 2024-09-17T21:30:35.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.372 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:35.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:35.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:35.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4699999999999998e-07 2024-09-17T21:30:35.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968112+0000", 2024-09-17T21:30:35.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-06 2024-09-17T21:30:35.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968116+0000", 2024-09-17T21:30:35.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1080000000000001e-06 2024-09-17T21:30:35.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968122+0000", 2024-09-17T21:30:35.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1070000000000001e-06 2024-09-17T21:30:35.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986537+0000", 2024-09-17T21:30:35.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018414834000000001 2024-09-17T21:30:35.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986572+0000", 2024-09-17T21:30:35.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5726000000000003e-05 2024-09-17T21:30:35.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986611+0000", 2024-09-17T21:30:35.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8584000000000002e-05 2024-09-17T21:30:35.375 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.375 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.375 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.375 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39462 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1892352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.375 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:35.375 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331486802999997, 2024-09-17T21:30:35.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018583252000000001, 2024-09-17T21:30:35.375 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.375 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.376 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.376 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.376 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.376 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.376 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39462 2024-09-17T21:30:35.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.376 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:35.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:35.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968126+0000", 2024-09-17T21:30:35.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:35.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968127+0000", 2024-09-17T21:30:35.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.229e-06 2024-09-17T21:30:35.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968131+0000", 2024-09-17T21:30:35.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8010000000000001e-06 2024-09-17T21:30:35.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968134+0000", 2024-09-17T21:30:35.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8880000000000001e-06 2024-09-17T21:30:35.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986634+0000", 2024-09-17T21:30:35.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018500501999999999 2024-09-17T21:30:35.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986671+0000", 2024-09-17T21:30:35.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6881000000000001e-05 2024-09-17T21:30:35.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986709+0000", 2024-09-17T21:30:35.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7546999999999998e-05 2024-09-17T21:30:35.379 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.379 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.379 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.379 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39463 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1957888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.380 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:35.380 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331475187999999, 2024-09-17T21:30:35.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018667186999999998, 2024-09-17T21:30:35.380 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.380 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.380 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.380 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39463 2024-09-17T21:30:35.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.380 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:35.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:35.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:35.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:30:35.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968138+0000", 2024-09-17T21:30:35.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:30:35.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968142+0000", 2024-09-17T21:30:35.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.145e-06 2024-09-17T21:30:35.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968144+0000", 2024-09-17T21:30:35.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6869999999999999e-06 2024-09-17T21:30:35.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986728+0000", 2024-09-17T21:30:35.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018584163000000001 2024-09-17T21:30:35.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986766+0000", 2024-09-17T21:30:35.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7987000000000001e-05 2024-09-17T21:30:35.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986804+0000", 2024-09-17T21:30:35.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7892000000000002e-05 2024-09-17T21:30:35.384 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.384 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.384 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.384 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39464 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.384 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:35.384 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331464879000002, 2024-09-17T21:30:35.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018753723999999999, 2024-09-17T21:30:35.384 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.384 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.384 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.384 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.384 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.385 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.385 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39464 2024-09-17T21:30:35.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.385 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:35.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:35.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968148+0000", 2024-09-17T21:30:35.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:30:35.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968152+0000", 2024-09-17T21:30:35.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3749999999999996e-06 2024-09-17T21:30:35.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968155+0000", 2024-09-17T21:30:35.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.371e-06 2024-09-17T21:30:35.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968158+0000", 2024-09-17T21:30:35.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T21:30:35.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986824+0000", 2024-09-17T21:30:35.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018665779 2024-09-17T21:30:35.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986863+0000", 2024-09-17T21:30:35.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9273000000000002e-05 2024-09-17T21:30:35.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986901+0000", 2024-09-17T21:30:35.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7978000000000001e-05 2024-09-17T21:30:35.388 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.388 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.388 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.388 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39465 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:35.388 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:35.388 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.331449480000003, 2024-09-17T21:30:35.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018825341999999998, 2024-09-17T21:30:35.388 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.388 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.389 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.389 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39465 2024-09-17T21:30:35.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.389 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:35.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:35.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:35.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-17T21:30:35.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968164+0000", 2024-09-17T21:30:35.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1289999999999999e-06 2024-09-17T21:30:35.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968167+0000", 2024-09-17T21:30:35.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1379999999999999e-06 2024-09-17T21:30:35.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968170+0000", 2024-09-17T21:30:35.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:30:35.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986914+0000", 2024-09-17T21:30:35.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018743829 2024-09-17T21:30:35.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986950+0000", 2024-09-17T21:30:35.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5819999999999999e-05 2024-09-17T21:30:35.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986988+0000", 2024-09-17T21:30:35.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8362000000000003e-05 2024-09-17T21:30:35.392 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.392 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.392 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.392 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.392 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.392 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:35.393 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:35.299+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:35.393 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:30:35.395 INFO:teuthology.orchestra.run.smithi129.stdout:3673 punch from 0x723e422 to 0x724c884, (0xe462 bytes) 2024-09-17T21:30:35.395 INFO:teuthology.orchestra.run.smithi129.stdout:3674 read 0xa9e1fd2 thru 0xa9e7fbb (0x5fea bytes) 2024-09-17T21:30:35.395 INFO:teuthology.orchestra.run.smithi129.stdout:3675 punch from 0xba394a2 to 0xba45ca2, (0xc800 bytes) 2024-09-17T21:30:35.395 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.396 INFO:teuthology.orchestra.run.smithi129.stdout:3678 punch from 0x1a9b761 to 0x1aa75dc, (0xbe7b bytes) 2024-09-17T21:30:35.396 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.396 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.396 INFO:teuthology.orchestra.run.smithi129.stdout:3682 writesame 0x34e685d thru 0x34ea567 data_size 0x1(0x3d0b bytes) 2024-09-17T21:30:35.396 INFO:teuthology.orchestra.run.smithi129.stdout:3683 writesame 0x6fb0687 thru 0x6fb4ba5 data_size 0x1(0x451f bytes) 2024-09-17T21:30:35.396 INFO:teuthology.orchestra.run.smithi129.stdout:3684 trunc from 0xc092467 to 0xb039ec9 2024-09-17T21:30:35.396 INFO:teuthology.orchestra.run.smithi129.stdout:3685 read 0x53a0ec4 thru 0x53ab7ab (0xa8e8 bytes) 2024-09-17T21:30:35.396 INFO:teuthology.orchestra.run.smithi129.stdout:3686 read 0x20304d3 thru 0x20330f1 (0x2c1f bytes) 2024-09-17T21:30:35.396 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.396 INFO:teuthology.orchestra.run.smithi129.stdout:3689 write 0xeaa091f thru 0xeaa6752 (0x5e34 bytes) 2024-09-17T21:30:35.396 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.396 INFO:teuthology.orchestra.run.smithi129.stdout:3691 trunc from 0xeaa6753 to 0xa26da79 2024-09-17T21:30:35.396 INFO:teuthology.orchestra.run.smithi129.stdout:3692 read 0x4073d9b thru 0x407b165 (0x73cb bytes) 2024-09-17T21:30:35.396 INFO:teuthology.orchestra.run.smithi129.stdout:3693 read 0x84ec81c thru 0x84f2e70 (0x6655 bytes) 2024-09-17T21:30:35.396 INFO:teuthology.orchestra.run.smithi129.stdout:3694 write 0x2d9853 thru 0x2e2d59 (0x9507 bytes) 2024-09-17T21:30:35.403 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:35.401+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:35.409 INFO:teuthology.orchestra.run.smithi129.stdout:3695 writesame 0x90fc69 thru 0x91a93f data_size 0x81(0xacd7 bytes) 2024-09-17T21:30:35.426 INFO:teuthology.orchestra.run.smithi129.stdout:3699 read 0xa033556 thru 0xa03f1c5 (0xbc70 bytes) 2024-09-17T21:30:35.426 INFO:teuthology.orchestra.run.smithi129.stdout:3701 punch from 0x9886513 to 0x988cd48, (0x6835 bytes) 2024-09-17T21:30:35.432 INFO:teuthology.orchestra.run.smithi129.stdout:3703 write 0xaed74b7 thru 0xaee63b9 (0xef03 bytes) 2024-09-17T21:30:35.436 INFO:teuthology.orchestra.run.smithi129.stdout:3705 trunc from 0xaee63ba to 0xc08bd66 2024-09-17T21:30:35.448 INFO:teuthology.orchestra.run.smithi129.stdout:3708 trunc from 0xc08bd66 to 0x652a663 2024-09-17T21:30:35.478 INFO:teuthology.orchestra.run.smithi129.stdout:3710 trunc from 0x652a663 to 0x5b08cd9 2024-09-17T21:30:35.486 INFO:teuthology.orchestra.run.smithi129.stdout:3711 punch from 0xbbe5b7 to 0xbc46c6, (0x610f bytes) 2024-09-17T21:30:35.486 INFO:teuthology.orchestra.run.smithi129.stdout:3712 read 0xe36a71 thru 0xe3c7d9 (0x5d69 bytes) 2024-09-17T21:30:35.502 INFO:teuthology.orchestra.run.smithi129.stdout:3713 write 0xe276581 thru 0xe284995 (0xe415 bytes) 2024-09-17T21:30:35.504 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:35.501+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:35.519 INFO:teuthology.orchestra.run.smithi129.stdout:3714 trunc from 0xe284996 to 0x8a8a2ec 2024-09-17T21:30:35.532 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.532 INFO:teuthology.orchestra.run.smithi129.stdout:3716 punch from 0x7c777b9 to 0x7c85f6b, (0xe7b2 bytes) 2024-09-17T21:30:35.532 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.532 INFO:teuthology.orchestra.run.smithi129.stdout:3718 punch from 0x19aa86e to 0x19aa8e9, (0x7b bytes) 2024-09-17T21:30:35.540 INFO:teuthology.orchestra.run.smithi129.stdout:3719 trunc from 0x8a8a2ec to 0x1a9c335 2024-09-17T21:30:35.558 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:35.558 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:35.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.558 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:83423 3.f 3:f4447bff:::rbd_object_map.1120deeff1ce:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:35.558 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:35.550485+0000", 2024-09-17T21:30:35.558 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0051198160000000001, 2024-09-17T21:30:35.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005137282, 2024-09-17T21:30:35.558 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.558 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.558 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:30:35.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.559 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.559 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.559 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 83423 2024-09-17T21:30:35.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.559 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550485+0000", 2024-09-17T21:30:35.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550485+0000", 2024-09-17T21:30:35.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550486+0000", 2024-09-17T21:30:35.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:30:35.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550490+0000", 2024-09-17T21:30:35.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999999e-06 2024-09-17T21:30:35.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550498+0000", 2024-09-17T21:30:35.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4440000000000008e-06 2024-09-17T21:30:35.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550506+0000", 2024-09-17T21:30:35.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6410000000000002e-06 2024-09-17T21:30:35.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.554216+0000", 2024-09-17T21:30:35.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003709983 2024-09-17T21:30:35.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.554260+0000", 2024-09-17T21:30:35.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4438999999999999e-05 2024-09-17T21:30:35.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:35.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.554446+0000", 2024-09-17T21:30:35.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018540700000000001 2024-09-17T21:30:35.562 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.562 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.562 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.563 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:83426 3.f 3:f4447bff:::rbd_object_map.1120deeff1ce:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:35.563 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:35.550711+0000", 2024-09-17T21:30:35.563 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0048940779999999996, 2024-09-17T21:30:35.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004954267, 2024-09-17T21:30:35.563 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.563 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.563 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:30:35.563 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.563 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.563 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.563 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 83426 2024-09-17T21:30:35.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.564 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550711+0000", 2024-09-17T21:30:35.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550711+0000", 2024-09-17T21:30:35.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550712+0000", 2024-09-17T21:30:35.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0999999999999997e-07 2024-09-17T21:30:35.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550716+0000", 2024-09-17T21:30:35.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4070000000000003e-06 2024-09-17T21:30:35.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550729+0000", 2024-09-17T21:30:35.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2821e-05 2024-09-17T21:30:35.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550735+0000", 2024-09-17T21:30:35.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.891e-06 2024-09-17T21:30:35.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.554850+0000", 2024-09-17T21:30:35.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0041154039999999996 2024-09-17T21:30:35.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.554951+0000", 2024-09-17T21:30:35.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010045800000000001 2024-09-17T21:30:35.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:35.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.555280+0000", 2024-09-17T21:30:35.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000329721 2024-09-17T21:30:35.567 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.567 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.567 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.567 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:83427 3.f 3.ffde222f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:35.567 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:35.552522+0000", 2024-09-17T21:30:35.567 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0030828349999999999, 2024-09-17T21:30:35.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031762040000000002, 2024-09-17T21:30:35.567 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.567 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.567 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:30:35.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.568 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.568 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.568 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 83427 2024-09-17T21:30:35.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.568 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.552522+0000", 2024-09-17T21:30:35.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.552522+0000", 2024-09-17T21:30:35.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.552523+0000", 2024-09-17T21:30:35.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5700000000000001e-07 2024-09-17T21:30:35.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.552527+0000", 2024-09-17T21:30:35.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8e-06 2024-09-17T21:30:35.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.552538+0000", 2024-09-17T21:30:35.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1245e-05 2024-09-17T21:30:35.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.552544+0000", 2024-09-17T21:30:35.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1079999999999998e-06 2024-09-17T21:30:35.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.555701+0000", 2024-09-17T21:30:35.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031568529999999998 2024-09-17T21:30:35.570 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.570 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.571 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.571 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:83429 3.f 3.ffde222f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:35.571 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:35.552660+0000", 2024-09-17T21:30:35.571 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0029448539999999998, 2024-09-17T21:30:35.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030644119999999999, 2024-09-17T21:30:35.571 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.571 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.571 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:30:35.571 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.571 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.571 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.572 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 83429 2024-09-17T21:30:35.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.572 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.552660+0000", 2024-09-17T21:30:35.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.552660+0000", 2024-09-17T21:30:35.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.552661+0000", 2024-09-17T21:30:35.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6499999999999999e-07 2024-09-17T21:30:35.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.552664+0000", 2024-09-17T21:30:35.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.444e-06 2024-09-17T21:30:35.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.552669+0000", 2024-09-17T21:30:35.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2769999999999996e-06 2024-09-17T21:30:35.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.552673+0000", 2024-09-17T21:30:35.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3919999999999999e-06 2024-09-17T21:30:35.574 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.574 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.574 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.574 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:83428 3.12 e69/25 3:4a4031ea:::rbd_data.1120deeff1ce.000000000000000d:head v 69'143, mlcod=68'141)", 2024-09-17T21:30:35.574 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:35.553008+0000", 2024-09-17T21:30:35.574 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0025966520000000001, 2024-09-17T21:30:35.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027425570000000001, 2024-09-17T21:30:35.575 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.575 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.575 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.575 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553008+0000", 2024-09-17T21:30:35.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553008+0000", 2024-09-17T21:30:35.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553009+0000", 2024-09-17T21:30:35.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1900000000000002e-07 2024-09-17T21:30:35.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553016+0000", 2024-09-17T21:30:35.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5899999999999996e-06 2024-09-17T21:30:35.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553025+0000", 2024-09-17T21:30:35.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2690000000000005e-06 2024-09-17T21:30:35.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553033+0000", 2024-09-17T21:30:35.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7470000000000005e-06 2024-09-17T21:30:35.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553083+0000", 2024-09-17T21:30:35.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0093000000000003e-05 2024-09-17T21:30:35.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553112+0000", 2024-09-17T21:30:35.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8719999999999999e-05 2024-09-17T21:30:35.578 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.578 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.578 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.578 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:83430 3.f 3.cc3a5b8f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:35.578 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:35.553199+0000", 2024-09-17T21:30:35.578 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.002405707, 2024-09-17T21:30:35.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002576666, 2024-09-17T21:30:35.579 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.579 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.579 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:30:35.579 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.579 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.579 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.579 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 83430 2024-09-17T21:30:35.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.579 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553199+0000", 2024-09-17T21:30:35.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553199+0000", 2024-09-17T21:30:35.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553200+0000", 2024-09-17T21:30:35.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3699999999999999e-07 2024-09-17T21:30:35.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553205+0000", 2024-09-17T21:30:35.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6410000000000001e-06 2024-09-17T21:30:35.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553216+0000", 2024-09-17T21:30:35.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0795000000000001e-05 2024-09-17T21:30:35.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553223+0000", 2024-09-17T21:30:35.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5399999999999998e-06 2024-09-17T21:30:35.581 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.581 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.582 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.582 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:83431 3.f 3.ffde222f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:35.582 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:35.554809+0000", 2024-09-17T21:30:35.582 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.000796534, 2024-09-17T21:30:35.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00099100899999999994, 2024-09-17T21:30:35.582 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.582 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.582 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:30:35.582 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.582 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.582 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.583 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 83431 2024-09-17T21:30:35.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.583 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.554809+0000", 2024-09-17T21:30:35.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.554809+0000", 2024-09-17T21:30:35.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.554809+0000", 2024-09-17T21:30:35.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0999999999999997e-07 2024-09-17T21:30:35.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.554814+0000", 2024-09-17T21:30:35.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T21:30:35.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.554824+0000", 2024-09-17T21:30:35.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5710000000000004e-06 2024-09-17T21:30:35.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.554838+0000", 2024-09-17T21:30:35.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3866000000000001e-05 2024-09-17T21:30:35.585 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.585 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.585 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.585 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4384.0:83395 3.f e69/25)", 2024-09-17T21:30:35.585 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:35.555093+0000", 2024-09-17T21:30:35.585 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00051197199999999997, 2024-09-17T21:30:35.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00072868599999999998, 2024-09-17T21:30:35.585 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.585 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.586 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:30:35.586 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.555093+0000", 2024-09-17T21:30:35.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.555093+0000", 2024-09-17T21:30:35.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.555094+0000", 2024-09-17T21:30:35.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.99e-07 2024-09-17T21:30:35.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.555098+0000", 2024-09-17T21:30:35.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0110000000000002e-06 2024-09-17T21:30:35.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.555108+0000", 2024-09-17T21:30:35.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0268999999999999e-05 2024-09-17T21:30:35.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.555115+0000", 2024-09-17T21:30:35.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1130000000000003e-06 2024-09-17T21:30:35.588 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.588 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.588 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.588 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4384.0:83398 3.f e69/25)", 2024-09-17T21:30:35.588 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:35.555127+0000", 2024-09-17T21:30:35.588 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00047778499999999999, 2024-09-17T21:30:35.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00072755200000000002, 2024-09-17T21:30:35.589 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.589 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.589 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:30:35.589 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.555127+0000", 2024-09-17T21:30:35.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.555127+0000", 2024-09-17T21:30:35.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.555128+0000", 2024-09-17T21:30:35.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.84e-07 2024-09-17T21:30:35.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.555129+0000", 2024-09-17T21:30:35.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:30:35.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.555133+0000", 2024-09-17T21:30:35.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0489999999999997e-06 2024-09-17T21:30:35.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.555136+0000", 2024-09-17T21:30:35.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.756e-06 2024-09-17T21:30:35.591 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.591 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.591 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.591 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:83395 3.f 3:f4447bff:::rbd_object_map.1120deeff1ce:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:35.592 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:35.548379+0000", 2024-09-17T21:30:35.592 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0072262009999999998, 2024-09-17T21:30:35.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075011879999999998, 2024-09-17T21:30:35.592 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.592 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.592 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:30:35.592 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.592 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.592 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.592 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 83395 2024-09-17T21:30:35.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.592 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.548379+0000", 2024-09-17T21:30:35.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.548379+0000", 2024-09-17T21:30:35.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.548379+0000", 2024-09-17T21:30:35.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5299999999999999e-07 2024-09-17T21:30:35.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.548382+0000", 2024-09-17T21:30:35.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5129999999999999e-06 2024-09-17T21:30:35.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.548387+0000", 2024-09-17T21:30:35.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8459999999999999e-06 2024-09-17T21:30:35.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.548390+0000", 2024-09-17T21:30:35.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4189999999999998e-06 2024-09-17T21:30:35.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.549962+0000", 2024-09-17T21:30:35.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001571703 2024-09-17T21:30:35.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550037+0000", 2024-09-17T21:30:35.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5197999999999994e-05 2024-09-17T21:30:35.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:35.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550508+0000", 2024-09-17T21:30:35.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047105900000000002 2024-09-17T21:30:35.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:35.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553096+0000", 2024-09-17T21:30:35.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025877140000000001 2024-09-17T21:30:35.596 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.596 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.596 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.596 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:83398 3.f 3:f4447bff:::rbd_object_map.1120deeff1ce:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:35.596 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:35.548672+0000", 2024-09-17T21:30:35.596 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0069334100000000001, 2024-09-17T21:30:35.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0072447350000000004, 2024-09-17T21:30:35.597 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.597 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.597 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:30:35.597 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.597 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.597 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.597 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 83398 2024-09-17T21:30:35.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.597 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.548672+0000", 2024-09-17T21:30:35.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.548672+0000", 2024-09-17T21:30:35.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.548673+0000", 2024-09-17T21:30:35.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:30:35.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.548677+0000", 2024-09-17T21:30:35.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8409999999999999e-06 2024-09-17T21:30:35.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.548686+0000", 2024-09-17T21:30:35.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:30:35.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.548693+0000", 2024-09-17T21:30:35.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4150000000000002e-06 2024-09-17T21:30:35.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.551013+0000", 2024-09-17T21:30:35.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023197500000000002 2024-09-17T21:30:35.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.551065+0000", 2024-09-17T21:30:35.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:30:35.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:35.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.551260+0000", 2024-09-17T21:30:35.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019515799999999999 2024-09-17T21:30:35.601 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.601 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.601 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.601 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:83402 3.f 3:f4447bff:::rbd_object_map.1120deeff1ce:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:35.601 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:35.548906+0000", 2024-09-17T21:30:35.601 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0066987799999999997, 2024-09-17T21:30:35.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0070427550000000004, 2024-09-17T21:30:35.601 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.601 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.601 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:30:35.601 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.602 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 83402 2024-09-17T21:30:35.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.602 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.548906+0000", 2024-09-17T21:30:35.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.548906+0000", 2024-09-17T21:30:35.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.548907+0000", 2024-09-17T21:30:35.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:30:35.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.548913+0000", 2024-09-17T21:30:35.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3720000000000004e-06 2024-09-17T21:30:35.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.548923+0000", 2024-09-17T21:30:35.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0368e-05 2024-09-17T21:30:35.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.548930+0000", 2024-09-17T21:30:35.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5880000000000003e-06 2024-09-17T21:30:35.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.551676+0000", 2024-09-17T21:30:35.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027460269999999998 2024-09-17T21:30:35.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.551732+0000", 2024-09-17T21:30:35.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6684999999999997e-05 2024-09-17T21:30:35.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:35.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.552014+0000", 2024-09-17T21:30:35.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00028133599999999998 2024-09-17T21:30:35.605 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.605 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.605 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.605 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:83409 3.f 3:f4447bff:::rbd_object_map.1120deeff1ce:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:35.606 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:35.549390+0000", 2024-09-17T21:30:35.606 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0062148569999999998, 2024-09-17T21:30:35.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0065964880000000002, 2024-09-17T21:30:35.606 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.606 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.606 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:30:35.606 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.606 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.606 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.606 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 83409 2024-09-17T21:30:35.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.606 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.549390+0000", 2024-09-17T21:30:35.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.549390+0000", 2024-09-17T21:30:35.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.549391+0000", 2024-09-17T21:30:35.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3099999999999997e-07 2024-09-17T21:30:35.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.549395+0000", 2024-09-17T21:30:35.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0180000000000003e-06 2024-09-17T21:30:35.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.549403+0000", 2024-09-17T21:30:35.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4290000000000001e-06 2024-09-17T21:30:35.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.549409+0000", 2024-09-17T21:30:35.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9950000000000002e-06 2024-09-17T21:30:35.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.552417+0000", 2024-09-17T21:30:35.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030075290000000001 2024-09-17T21:30:35.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.552463+0000", 2024-09-17T21:30:35.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6242000000000002e-05 2024-09-17T21:30:35.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:35.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.552685+0000", 2024-09-17T21:30:35.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00022225200000000001 2024-09-17T21:30:35.610 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.610 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.610 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.610 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:83413 3.f 3:f4447bff:::rbd_object_map.1120deeff1ce:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:35.610 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:35.549771+0000", 2024-09-17T21:30:35.610 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0058335440000000004, 2024-09-17T21:30:35.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062639469999999997, 2024-09-17T21:30:35.610 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.610 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.611 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:30:35.611 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.611 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.611 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.611 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 83413 2024-09-17T21:30:35.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.611 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.549771+0000", 2024-09-17T21:30:35.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.549771+0000", 2024-09-17T21:30:35.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.549772+0000", 2024-09-17T21:30:35.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2099999999999997e-07 2024-09-17T21:30:35.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.549775+0000", 2024-09-17T21:30:35.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2890000000000002e-06 2024-09-17T21:30:35.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.549782+0000", 2024-09-17T21:30:35.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4389999999999997e-06 2024-09-17T21:30:35.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.549794+0000", 2024-09-17T21:30:35.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:30:35.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553120+0000", 2024-09-17T21:30:35.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033261300000000001 2024-09-17T21:30:35.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553165+0000", 2024-09-17T21:30:35.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5668999999999999e-05 2024-09-17T21:30:35.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:35.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553370+0000", 2024-09-17T21:30:35.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020424500000000001 2024-09-17T21:30:35.614 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.614 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.614 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.615 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:83422 3.f 3:f4447bff:::rbd_object_map.1120deeff1ce:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:35.615 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:35.550430+0000", 2024-09-17T21:30:35.615 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0051753779999999996, 2024-09-17T21:30:35.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056557179999999997, 2024-09-17T21:30:35.615 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.615 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.615 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:30:35.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.615 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 83422 2024-09-17T21:30:35.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.616 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550430+0000", 2024-09-17T21:30:35.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550430+0000", 2024-09-17T21:30:35.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550430+0000", 2024-09-17T21:30:35.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.75e-07 2024-09-17T21:30:35.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550434+0000", 2024-09-17T21:30:35.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:30:35.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550443+0000", 2024-09-17T21:30:35.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6770000000000001e-06 2024-09-17T21:30:35.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.550449+0000", 2024-09-17T21:30:35.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1770000000000003e-06 2024-09-17T21:30:35.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553658+0000", 2024-09-17T21:30:35.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0032084589999999999 2024-09-17T21:30:35.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553702+0000", 2024-09-17T21:30:35.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4793999999999998e-05 2024-09-17T21:30:35.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:35.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:35.553890+0000", 2024-09-17T21:30:35.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018730600000000001 2024-09-17T21:30:35.619 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.619 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.619 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.619 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.619 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-17T21:30:35.619 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 15 2024-09-17T21:30:35.619 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:35.625 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:30:35.625 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:35.617+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:35.626 INFO:teuthology.orchestra.run.smithi129.stdout:3720 read 0x815ae1 thru 0x81d7cd (0x7ced bytes) 2024-09-17T21:30:35.626 INFO:teuthology.orchestra.run.smithi129.stdout:3722 read 0x3cc295 thru 0x3da37d (0xe0e9 bytes) 2024-09-17T21:30:35.626 INFO:teuthology.orchestra.run.smithi129.stdout:3723 write 0xfc8352 thru 0xfd109a (0x8d49 bytes) 2024-09-17T21:30:35.626 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.626 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.627 INFO:teuthology.orchestra.run.smithi129.stdout:3727 trunc from 0x1a9c335 to 0x640bbf9 2024-09-17T21:30:35.627 INFO:teuthology.orchestra.run.smithi129.stdout:3728 read 0xd1929d thru 0xd218d1 (0x8635 bytes) 2024-09-17T21:30:35.627 INFO:teuthology.orchestra.run.smithi129.stdout:3729 read 0x53d9d8b thru 0x53e8db2 (0xf028 bytes) 2024-09-17T21:30:35.627 INFO:teuthology.orchestra.run.smithi129.stdout:3730 trunc from 0x640bbf9 to 0x97626df 2024-09-17T21:30:35.627 INFO:teuthology.orchestra.run.smithi129.stdout:3731 punch from 0x1ea2656 to 0x1ea3091, (0xa3b bytes) 2024-09-17T21:30:35.627 INFO:teuthology.orchestra.run.smithi129.stdout:3732 read 0x189dbb1 thru 0x18a64f2 (0x8942 bytes) 2024-09-17T21:30:35.627 INFO:teuthology.orchestra.run.smithi129.stdout:3733 trunc from 0x97626df to 0xb210ec7 2024-09-17T21:30:35.627 INFO:teuthology.orchestra.run.smithi129.stdout:3734 punch from 0x21d7e2a to 0x21e02df, (0x84b5 bytes) 2024-09-17T21:30:35.627 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.627 INFO:teuthology.orchestra.run.smithi129.stdout:3738 punch from 0x34c9023 to 0x34cf16b, (0x6148 bytes) 2024-09-17T21:30:35.627 INFO:teuthology.orchestra.run.smithi129.stdout:3739 write 0xae0298 thru 0xae0a82 (0x7eb bytes) 2024-09-17T21:30:35.627 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.627 INFO:teuthology.orchestra.run.smithi129.stdout:3741 trunc from 0xb210ec7 to 0x3f388ef 2024-09-17T21:30:35.627 INFO:teuthology.orchestra.run.smithi129.stdout:3742 read 0xa1d33f thru 0xa28262 (0xaf24 bytes) 2024-09-17T21:30:35.628 INFO:teuthology.orchestra.run.smithi129.stdout:3743 write 0xc8cbda thru 0xc9a83c (0xdc63 bytes) 2024-09-17T21:30:35.630 INFO:teuthology.orchestra.run.smithi129.stdout:3745 read 0x30423c5 thru 0x30437c1 (0x13fd bytes) 2024-09-17T21:30:35.641 INFO:teuthology.orchestra.run.smithi129.stdout:3746 trunc from 0x3f388ef to 0xcabf366 2024-09-17T21:30:35.647 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.647 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.647 INFO:teuthology.orchestra.run.smithi129.stdout:3751 read 0xb8391c2 thru 0xb8396aa (0x4e9 bytes) 2024-09-17T21:30:35.649 INFO:teuthology.orchestra.run.smithi129.stdout:3752 trunc from 0xcabf366 to 0xb2d3706 2024-09-17T21:30:35.655 INFO:teuthology.orchestra.run.smithi129.stdout:3753 writesame 0x17ea5a2 thru 0x17f2dd1 data_size 0x8(0x8830 bytes) 2024-09-17T21:30:35.658 INFO:teuthology.orchestra.run.smithi129.stdout:3754 write 0x395da2 thru 0x39a727 (0x4986 bytes) 2024-09-17T21:30:35.661 INFO:teuthology.orchestra.run.smithi129.stdout:3755 write 0x5e9b15d thru 0x5ea9cf5 (0xeb99 bytes) 2024-09-17T21:30:35.673 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.673 INFO:teuthology.orchestra.run.smithi129.stdout:3757 punch from 0x1eadaa7 to 0x1eaf7d9, (0x1d32 bytes) 2024-09-17T21:30:35.679 INFO:teuthology.orchestra.run.smithi129.stdout:3758 write 0xc2616bf thru 0xc26b5ad (0x9eef bytes) 2024-09-17T21:30:35.684 INFO:teuthology.orchestra.run.smithi129.stdout:3759 write 0x567a34e thru 0x56889ac (0xe65f bytes) 2024-09-17T21:30:35.696 INFO:teuthology.orchestra.run.smithi129.stdout:3760 trunc from 0xc26b5ae to 0x3aaba88 2024-09-17T21:30:35.708 INFO:teuthology.orchestra.run.smithi129.stdout:3761 write 0x150a4d3 thru 0x1514d4c (0xa87a bytes) 2024-09-17T21:30:35.721 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:35.718+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:35.727 INFO:teuthology.orchestra.run.smithi129.stdout:3762 writesame 0xca37cd3 thru 0xca40b81 data_size 0x1(0x8eaf bytes) 2024-09-17T21:30:35.727 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.727 INFO:teuthology.orchestra.run.smithi129.stdout:3766 read 0x899d2f8 thru 0x89a4744 (0x744d bytes) 2024-09-17T21:30:35.727 INFO:teuthology.orchestra.run.smithi129.stdout:3767 read 0x1e7d1f5 thru 0x1e8829d (0xb0a9 bytes) 2024-09-17T21:30:35.728 INFO:teuthology.orchestra.run.smithi129.stdout:3768 write 0x2a0190c thru 0x2a06dc8 (0x54bd bytes) 2024-09-17T21:30:35.729 INFO:teuthology.orchestra.run.smithi129.stdout:3769 write 0xb1985b6 thru 0xb19c5c1 (0x400c bytes) 2024-09-17T21:30:35.733 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.733 INFO:teuthology.orchestra.run.smithi129.stdout:3771 write 0x567802b thru 0x56850fb (0xd0d1 bytes) 2024-09-17T21:30:35.739 INFO:teuthology.orchestra.run.smithi129.stdout:3772 writesame 0x6277cc8 thru 0x62870a8 data_size 0x15(0xf3e1 bytes) 2024-09-17T21:30:35.742 INFO:teuthology.orchestra.run.smithi129.stdout:3774 write 0xb783d7f thru 0xb78df08 (0xa18a bytes) 2024-09-17T21:30:35.761 INFO:teuthology.orchestra.run.smithi129.stdout:3775 trunc from 0xca40b82 to 0x1af8bea 2024-09-17T21:30:35.780 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:35.780 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:35.789 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:30:35.797 INFO:teuthology.orchestra.run.smithi129.stdout:3776 writesame 0xcea0c5d thru 0xcea5429 data_size 0xb(0x47cd bytes) 2024-09-17T21:30:35.798 INFO:teuthology.orchestra.run.smithi129.stdout:3777 read 0x981ad1b thru 0x981b125 (0x40b bytes) 2024-09-17T21:30:35.798 INFO:teuthology.orchestra.run.smithi129.stdout:3778 read 0x9f55a3c thru 0x9f61bb0 (0xc175 bytes) 2024-09-17T21:30:35.798 INFO:teuthology.orchestra.run.smithi129.stdout:3779 punch from 0x68bee71 to 0x68c2032, (0x31c1 bytes) 2024-09-17T21:30:35.798 INFO:teuthology.orchestra.run.smithi129.stdout:3780 read 0xaef9964 thru 0xaefde0b (0x44a8 bytes) 2024-09-17T21:30:35.798 INFO:teuthology.orchestra.run.smithi129.stdout:3782 punch from 0x37146d6 to 0x37218b2, (0xd1dc bytes) 2024-09-17T21:30:35.798 INFO:teuthology.orchestra.run.smithi129.stdout:3783 write 0x90d7774 thru 0x90ddcc3 (0x6550 bytes) 2024-09-17T21:30:35.800 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-17T21:30:35.800 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:30:35.800 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T21:30:35.800 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T21:30:35.815 INFO:teuthology.orchestra.run.smithi129.stdout:3784 trunc from 0xcea542a to 0x1417dfb 2024-09-17T21:30:35.833 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:35.830+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:35.856 INFO:teuthology.orchestra.run.smithi129.stdout:3787 trunc from 0x1417dfb to 0x8c7de7f 2024-09-17T21:30:35.861 INFO:teuthology.orchestra.run.smithi129.stdout:3789 punch from 0x3701741 to 0x370700a, (0x58c9 bytes) 2024-09-17T21:30:35.861 INFO:teuthology.orchestra.run.smithi129.stdout:3790 punch from 0x65915b9 to 0x65983b9, (0x6e00 bytes) 2024-09-17T21:30:35.861 INFO:teuthology.orchestra.run.smithi129.stdout:3792 read 0x71a3536 thru 0x71aa8af (0x737a bytes) 2024-09-17T21:30:35.861 INFO:teuthology.orchestra.run.smithi129.stdout:3793 writesame 0xa05250 thru 0xa138b0 data_size 0x3(0xe661 bytes) 2024-09-17T21:30:35.873 INFO:teuthology.orchestra.run.smithi129.stdout:3794 punch from 0x54c2adc to 0x54d1a51, (0xef75 bytes) 2024-09-17T21:30:35.873 INFO:teuthology.orchestra.run.smithi129.stdout:3796 punch from 0x26de375 to 0x26ec5a3, (0xe22e bytes) 2024-09-17T21:30:35.873 INFO:teuthology.orchestra.run.smithi129.stdout:3798 write 0x31f947a thru 0x31f9648 (0x1cf bytes) 2024-09-17T21:30:35.876 INFO:teuthology.orchestra.run.smithi129.stdout:3800 read 0x626fa61 thru 0x627dd9b (0xe33b bytes) 2024-09-17T21:30:35.876 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:35.876 INFO:teuthology.orchestra.run.smithi129.stdout:3804 read 0x4c86a49 thru 0x4c967a7 (0xfd5f bytes) 2024-09-17T21:30:35.878 INFO:teuthology.orchestra.run.smithi129.stdout:3805 trunc from 0x8c7de7f to 0xa43d557 2024-09-17T21:30:35.892 INFO:teuthology.orchestra.run.smithi129.stdout:3808 trunc from 0xa43d557 to 0x3c8b32c 2024-09-17T21:30:35.897 INFO:teuthology.orchestra.run.smithi129.stdout:3809 write 0x222106c thru 0x222e0ac (0xd041 bytes) 2024-09-17T21:30:35.908 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:35.908 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:35.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:35.908 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:35.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.908 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:30:35.908 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:35.908 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 67.965239546999996, 2024-09-17T21:30:35.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61572000699999996, 2024-09-17T21:30:35.909 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.909 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.909 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:35.909 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.909 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:30:35.909 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:30:35.909 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T21:30:35.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.909 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:35.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:35.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939745+0000", 2024-09-17T21:30:35.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6899999999999999e-06 2024-09-17T21:30:35.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939753+0000", 2024-09-17T21:30:35.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1950000000000003e-06 2024-09-17T21:30:35.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939767+0000", 2024-09-17T21:30:35.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3429e-05 2024-09-17T21:30:35.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939776+0000", 2024-09-17T21:30:35.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:30:35.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939858+0000", 2024-09-17T21:30:35.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2496000000000004e-05 2024-09-17T21:30:35.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T21:30:35.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939868+0000", 2024-09-17T21:30:35.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1910000000000006e-06 2024-09-17T21:30:35.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940612+0000", 2024-09-17T21:30:35.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074429599999999998 2024-09-17T21:30:35.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:30:35.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940621+0000", 2024-09-17T21:30:35.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9139999999999997e-06 2024-09-17T21:30:35.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554006+0000", 2024-09-17T21:30:35.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61338505099999996 2024-09-17T21:30:35.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554166+0000", 2024-09-17T21:30:35.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015982600000000001 2024-09-17T21:30:35.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:30:35.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554296+0000", 2024-09-17T21:30:35.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013002399999999999 2024-09-17T21:30:35.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:35.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554936+0000", 2024-09-17T21:30:35.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064009400000000001 2024-09-17T21:30:35.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:35.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555389+0000", 2024-09-17T21:30:35.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045355200000000002 2024-09-17T21:30:35.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:35.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555432+0000", 2024-09-17T21:30:35.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2477e-05 2024-09-17T21:30:35.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555463+0000", 2024-09-17T21:30:35.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1612000000000002e-05 2024-09-17T21:30:35.916 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.916 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.916 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.917 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17527 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1005568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:35.917 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:35.917 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.500011716000003, 2024-09-17T21:30:35.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046452090000000001, 2024-09-17T21:30:35.917 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.917 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.917 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.917 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.917 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.918 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17527 2024-09-17T21:30:35.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.918 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:35.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:35.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405496+0000", 2024-09-17T21:30:35.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052498400000000004 2024-09-17T21:30:35.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405500+0000", 2024-09-17T21:30:35.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8410000000000002e-06 2024-09-17T21:30:35.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405511+0000", 2024-09-17T21:30:35.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0749e-05 2024-09-17T21:30:35.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405517+0000", 2024-09-17T21:30:35.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:30:35.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451275+0000", 2024-09-17T21:30:35.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045758134999999998 2024-09-17T21:30:35.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451355+0000", 2024-09-17T21:30:35.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9453e-05 2024-09-17T21:30:35.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451423+0000", 2024-09-17T21:30:35.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8782999999999995e-05 2024-09-17T21:30:35.921 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.921 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.921 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.922 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17615 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:35.922 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:35.922 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.494771227999998, 2024-09-17T21:30:35.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046741309000000002, 2024-09-17T21:30:35.922 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.922 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.922 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.922 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.922 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.922 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.922 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17615 2024-09-17T21:30:35.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.923 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:35.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:35.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:35.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:30:35.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410213+0000", 2024-09-17T21:30:35.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:30:35.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410217+0000", 2024-09-17T21:30:35.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8199999999999998e-06 2024-09-17T21:30:35.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410220+0000", 2024-09-17T21:30:35.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6900000000000001e-06 2024-09-17T21:30:35.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455264+0000", 2024-09-17T21:30:35.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045044002999999999 2024-09-17T21:30:35.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455338+0000", 2024-09-17T21:30:35.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4182000000000002e-05 2024-09-17T21:30:35.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.456953+0000", 2024-09-17T21:30:35.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016150089999999999 2024-09-17T21:30:35.926 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.926 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.926 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.926 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17616 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 872448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:35.926 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:35.926 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.494759176000002, 2024-09-17T21:30:35.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946039000000002, 2024-09-17T21:30:35.927 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.927 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.927 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.927 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17616 2024-09-17T21:30:35.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.927 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:35.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:35.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:35.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T21:30:35.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410225+0000", 2024-09-17T21:30:35.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3459999999999999e-06 2024-09-17T21:30:35.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410228+0000", 2024-09-17T21:30:35.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.083e-06 2024-09-17T21:30:35.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410232+0000", 2024-09-17T21:30:35.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4419999999999998e-06 2024-09-17T21:30:35.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457004+0000", 2024-09-17T21:30:35.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046771873999999998 2024-09-17T21:30:35.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457084+0000", 2024-09-17T21:30:35.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0506000000000005e-05 2024-09-17T21:30:35.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457170+0000", 2024-09-17T21:30:35.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5494000000000005e-05 2024-09-17T21:30:35.930 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.930 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.931 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.931 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17617 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:35.931 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:35.931 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.494747773999997, 2024-09-17T21:30:35.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047084168000000003, 2024-09-17T21:30:35.931 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.931 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.931 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.931 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17617 2024-09-17T21:30:35.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.932 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:35.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:35.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:35.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-17T21:30:35.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410237+0000", 2024-09-17T21:30:35.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1960000000000001e-06 2024-09-17T21:30:35.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410246+0000", 2024-09-17T21:30:35.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7880000000000001e-06 2024-09-17T21:30:35.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410250+0000", 2024-09-17T21:30:35.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.518e-06 2024-09-17T21:30:35.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457209+0000", 2024-09-17T21:30:35.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046959283999999997 2024-09-17T21:30:35.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457264+0000", 2024-09-17T21:30:35.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673999999999999e-05 2024-09-17T21:30:35.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457319+0000", 2024-09-17T21:30:35.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5497000000000001e-05 2024-09-17T21:30:35.935 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.935 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.935 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.935 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17618 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 899072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:35.935 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:35.935 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.494728330000001, 2024-09-17T21:30:35.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047204208999999997, 2024-09-17T21:30:35.936 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.936 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.936 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.936 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.936 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.936 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.936 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17618 2024-09-17T21:30:35.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.936 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:35.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:35.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:35.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8799999999999998e-07 2024-09-17T21:30:35.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410256+0000", 2024-09-17T21:30:35.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T21:30:35.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410260+0000", 2024-09-17T21:30:35.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6500000000000002e-06 2024-09-17T21:30:35.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410263+0000", 2024-09-17T21:30:35.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7089999999999998e-06 2024-09-17T21:30:35.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457355+0000", 2024-09-17T21:30:35.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047092756999999999 2024-09-17T21:30:35.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457407+0000", 2024-09-17T21:30:35.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1808999999999998e-05 2024-09-17T21:30:35.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457459+0000", 2024-09-17T21:30:35.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1684000000000002e-05 2024-09-17T21:30:35.939 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.939 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.940 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.940 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17619 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 913408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:35.940 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:35.940 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.494717092000002, 2024-09-17T21:30:35.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047293443999999997, 2024-09-17T21:30:35.940 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.940 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.940 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.940 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.940 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.940 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.941 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17619 2024-09-17T21:30:35.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.941 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:35.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:35.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:35.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:30:35.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410267+0000", 2024-09-17T21:30:35.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:30:35.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410270+0000", 2024-09-17T21:30:35.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.01e-06 2024-09-17T21:30:35.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410273+0000", 2024-09-17T21:30:35.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6410000000000002e-06 2024-09-17T21:30:35.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457473+0000", 2024-09-17T21:30:35.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047199829999999998 2024-09-17T21:30:35.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457517+0000", 2024-09-17T21:30:35.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4058999999999999e-05 2024-09-17T21:30:35.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457559+0000", 2024-09-17T21:30:35.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2404999999999998e-05 2024-09-17T21:30:35.944 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.944 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.944 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.944 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17620 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 927744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:35.944 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:35.945 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.494705920000001, 2024-09-17T21:30:35.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047402409, 2024-09-17T21:30:35.945 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.945 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.945 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.945 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.945 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.945 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.945 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17620 2024-09-17T21:30:35.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.945 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:35.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:35.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:35.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:30:35.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410278+0000", 2024-09-17T21:30:35.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1790000000000001e-06 2024-09-17T21:30:35.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410281+0000", 2024-09-17T21:30:35.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.061e-06 2024-09-17T21:30:35.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410284+0000", 2024-09-17T21:30:35.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.65e-06 2024-09-17T21:30:35.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457587+0000", 2024-09-17T21:30:35.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047303206 2024-09-17T21:30:35.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457633+0000", 2024-09-17T21:30:35.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6063999999999998e-05 2024-09-17T21:30:35.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457679+0000", 2024-09-17T21:30:35.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6007000000000002e-05 2024-09-17T21:30:35.949 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.949 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.949 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.949 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17621 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 940032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:35.949 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:35.949 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.494695503999999, 2024-09-17T21:30:35.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047486550000000002, 2024-09-17T21:30:35.949 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.949 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.949 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.949 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.950 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.950 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.950 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17621 2024-09-17T21:30:35.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.950 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:35.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:35.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410288+0000", 2024-09-17T21:30:35.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:30:35.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410289+0000", 2024-09-17T21:30:35.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1179999999999999e-06 2024-09-17T21:30:35.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410292+0000", 2024-09-17T21:30:35.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:30:35.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410294+0000", 2024-09-17T21:30:35.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T21:30:35.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457694+0000", 2024-09-17T21:30:35.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047399416999999999 2024-09-17T21:30:35.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457734+0000", 2024-09-17T21:30:35.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0528999999999998e-05 2024-09-17T21:30:35.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457774+0000", 2024-09-17T21:30:35.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9529e-05 2024-09-17T21:30:35.953 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.953 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.953 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.954 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17622 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:35.954 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:35.954 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.494685322000002, 2024-09-17T21:30:35.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047596619, 2024-09-17T21:30:35.954 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.954 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.954 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.954 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.954 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.954 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.954 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17622 2024-09-17T21:30:35.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.955 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:35.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:35.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:35.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:35.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410303+0000", 2024-09-17T21:30:35.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2979999999999999e-06 2024-09-17T21:30:35.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410307+0000", 2024-09-17T21:30:35.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:30:35.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410310+0000", 2024-09-17T21:30:35.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.146e-06 2024-09-17T21:30:35.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457801+0000", 2024-09-17T21:30:35.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047491322000000002 2024-09-17T21:30:35.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457850+0000", 2024-09-17T21:30:35.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8585000000000003e-05 2024-09-17T21:30:35.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457894+0000", 2024-09-17T21:30:35.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4455e-05 2024-09-17T21:30:35.958 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.958 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.958 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.958 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17623 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 968704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:35.959 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:35.959 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.494668937999997, 2024-09-17T21:30:35.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047689377999999998, 2024-09-17T21:30:35.959 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.959 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.959 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.959 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.959 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.959 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.959 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17623 2024-09-17T21:30:35.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.959 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:35.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:35.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:35.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:30:35.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410315+0000", 2024-09-17T21:30:35.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0499999999999999e-06 2024-09-17T21:30:35.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410319+0000", 2024-09-17T21:30:35.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.517e-06 2024-09-17T21:30:35.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410321+0000", 2024-09-17T21:30:35.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:30:35.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457922+0000", 2024-09-17T21:30:35.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047600638000000001 2024-09-17T21:30:35.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457964+0000", 2024-09-17T21:30:35.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2126000000000003e-05 2024-09-17T21:30:35.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458003+0000", 2024-09-17T21:30:35.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9138999999999999e-05 2024-09-17T21:30:35.963 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.963 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.963 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.963 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17624 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:35.963 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:35.963 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.494658354000002, 2024-09-17T21:30:35.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047798292999999999, 2024-09-17T21:30:35.963 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.963 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.963 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.963 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.964 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.964 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.964 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17624 2024-09-17T21:30:35.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.964 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:35.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:35.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:35.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:30:35.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410326+0000", 2024-09-17T21:30:35.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0610000000000001e-06 2024-09-17T21:30:35.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410329+0000", 2024-09-17T21:30:35.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0319999999999999e-06 2024-09-17T21:30:35.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410331+0000", 2024-09-17T21:30:35.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5749999999999999e-06 2024-09-17T21:30:35.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458033+0000", 2024-09-17T21:30:35.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047701371999999999 2024-09-17T21:30:35.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458079+0000", 2024-09-17T21:30:35.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:30:35.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458123+0000", 2024-09-17T21:30:35.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850999999999997e-05 2024-09-17T21:30:35.967 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.967 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.967 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.968 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17625 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:35.968 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:35.968 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.494648194, 2024-09-17T21:30:35.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047883051000000003, 2024-09-17T21:30:35.968 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.968 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.968 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.968 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.968 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.968 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.968 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17625 2024-09-17T21:30:35.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.969 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:35.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:35.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:35.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:35.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410336+0000", 2024-09-17T21:30:35.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1310000000000001e-06 2024-09-17T21:30:35.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410339+0000", 2024-09-17T21:30:35.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000001e-06 2024-09-17T21:30:35.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410342+0000", 2024-09-17T21:30:35.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7460000000000001e-06 2024-09-17T21:30:35.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458137+0000", 2024-09-17T21:30:35.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047794931999999998 2024-09-17T21:30:35.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458178+0000", 2024-09-17T21:30:35.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1013999999999997e-05 2024-09-17T21:30:35.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458218+0000", 2024-09-17T21:30:35.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9675999999999998e-05 2024-09-17T21:30:35.972 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.972 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.972 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.972 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17626 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1009664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:35.973 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:35.973 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.494637572999999, 2024-09-17T21:30:35.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047991601000000002, 2024-09-17T21:30:35.973 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.973 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.973 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.973 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.973 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.973 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.973 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17626 2024-09-17T21:30:35.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.973 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:35.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:35.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410346+0000", 2024-09-17T21:30:35.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:30:35.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410347+0000", 2024-09-17T21:30:35.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.114e-06 2024-09-17T21:30:35.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410350+0000", 2024-09-17T21:30:35.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.101e-06 2024-09-17T21:30:35.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410353+0000", 2024-09-17T21:30:35.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T21:30:35.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458245+0000", 2024-09-17T21:30:35.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0478921 2024-09-17T21:30:35.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458292+0000", 2024-09-17T21:30:35.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397999999999999e-05 2024-09-17T21:30:35.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458337+0000", 2024-09-17T21:30:35.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5221e-05 2024-09-17T21:30:35.977 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.977 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.977 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.978 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17627 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:35.978 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:35.978 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.494600472000002, 2024-09-17T21:30:35.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048047383999999999, 2024-09-17T21:30:35.978 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.978 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.978 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.978 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.978 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.978 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.978 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17627 2024-09-17T21:30:35.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.979 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:35.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:35.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410383+0000", 2024-09-17T21:30:35.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:30:35.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410385+0000", 2024-09-17T21:30:35.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:30:35.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410391+0000", 2024-09-17T21:30:35.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:30:35.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410395+0000", 2024-09-17T21:30:35.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6539999999999998e-06 2024-09-17T21:30:35.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458350+0000", 2024-09-17T21:30:35.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047954855999999997 2024-09-17T21:30:35.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458392+0000", 2024-09-17T21:30:35.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2094000000000001e-05 2024-09-17T21:30:35.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458430+0000", 2024-09-17T21:30:35.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7474000000000001e-05 2024-09-17T21:30:35.982 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.982 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.982 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.983 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17628 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:35.983 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:35.983 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.494580911, 2024-09-17T21:30:35.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048142676000000002, 2024-09-17T21:30:35.983 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.983 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.983 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:35.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.983 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17628 2024-09-17T21:30:35.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.984 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:35.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:35.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:35.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:30:35.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410404+0000", 2024-09-17T21:30:35.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-06 2024-09-17T21:30:35.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410411+0000", 2024-09-17T21:30:35.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6850000000000004e-06 2024-09-17T21:30:35.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410415+0000", 2024-09-17T21:30:35.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6750000000000001e-06 2024-09-17T21:30:35.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458457+0000", 2024-09-17T21:30:35.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048042045999999998 2024-09-17T21:30:35.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458501+0000", 2024-09-17T21:30:35.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3661000000000001e-05 2024-09-17T21:30:35.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458545+0000", 2024-09-17T21:30:35.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3529000000000003e-05 2024-09-17T21:30:35.988 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.988 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.988 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.988 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17687 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:35.988 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:35.988 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.492135957999999, 2024-09-17T21:30:35.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050670448, 2024-09-17T21:30:35.988 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.988 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.988 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:35.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.989 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.989 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.989 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17687 2024-09-17T21:30:35.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.989 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:35.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:35.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412848+0000", 2024-09-17T21:30:35.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9100000000000003e-07 2024-09-17T21:30:35.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413102+0000", 2024-09-17T21:30:35.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025415600000000001 2024-09-17T21:30:35.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413111+0000", 2024-09-17T21:30:35.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2010000000000005e-06 2024-09-17T21:30:35.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413116+0000", 2024-09-17T21:30:35.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:30:35.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458559+0000", 2024-09-17T21:30:35.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045442914000000001 2024-09-17T21:30:35.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458632+0000", 2024-09-17T21:30:35.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3207000000000006e-05 2024-09-17T21:30:35.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:35.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458785+0000", 2024-09-17T21:30:35.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015226700000000001 2024-09-17T21:30:35.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:35.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.461648+0000", 2024-09-17T21:30:35.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028633069999999998 2024-09-17T21:30:35.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:35.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463416+0000", 2024-09-17T21:30:35.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017680230000000001 2024-09-17T21:30:35.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:35.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463475+0000", 2024-09-17T21:30:35.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9333000000000003e-05 2024-09-17T21:30:35.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:35.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463517+0000", 2024-09-17T21:30:35.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2162000000000001e-05 2024-09-17T21:30:35.994 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.994 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:35.994 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:35.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.995 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17688 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [write 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:35.995 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:35.995 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.491857090000003, 2024-09-17T21:30:35.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050625098, 2024-09-17T21:30:35.995 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:35.995 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:35.995 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:35.995 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:35.995 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:35.995 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:35.995 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17688 2024-09-17T21:30:35.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.995 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:35.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:35.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:35.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:35.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:35.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:35.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:35.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:35.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:30:35.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:35.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413128+0000", 2024-09-17T21:30:35.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.813e-06 2024-09-17T21:30:35.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:35.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413132+0000", 2024-09-17T21:30:35.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:30:35.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:35.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413135+0000", 2024-09-17T21:30:35.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:30:35.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:35.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459756+0000", 2024-09-17T21:30:35.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046621293000000001 2024-09-17T21:30:35.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:35.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459816+0000", 2024-09-17T21:30:35.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9156000000000001e-05 2024-09-17T21:30:35.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:35.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459912+0000", 2024-09-17T21:30:35.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6302000000000001e-05 2024-09-17T21:30:35.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:35.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463231+0000", 2024-09-17T21:30:35.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033188290000000001 2024-09-17T21:30:35.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:35.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463701+0000", 2024-09-17T21:30:35.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047068199999999999 2024-09-17T21:30:35.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:35.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:35.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:36.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463728+0000", 2024-09-17T21:30:36.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6726000000000001e-05 2024-09-17T21:30:36.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:36.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463751+0000", 2024-09-17T21:30:36.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2824e-05 2024-09-17T21:30:36.000 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:36.000 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:36.000 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:36.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.001 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17691 3.d 3:b3a055f3:::rbd_data.11207744cc2c.0000000000000083:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:36.001 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:36.001 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.488312186999998, 2024-09-17T21:30:36.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049309370999999998, 2024-09-17T21:30:36.001 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:36.001 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:36.001 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:36.001 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:36.001 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:36.001 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.001 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17691 2024-09-17T21:30:36.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.001 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:36.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:36.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:36.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:36.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:36.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:36.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:36.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:36.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416672+0000", 2024-09-17T21:30:36.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.47e-07 2024-09-17T21:30:36.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:36.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416915+0000", 2024-09-17T21:30:36.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024344500000000001 2024-09-17T21:30:36.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:36.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416924+0000", 2024-09-17T21:30:36.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7739999999999993e-06 2024-09-17T21:30:36.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416929+0000", 2024-09-17T21:30:36.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6990000000000002e-06 2024-09-17T21:30:36.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:36.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460414+0000", 2024-09-17T21:30:36.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043485466 2024-09-17T21:30:36.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:36.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460540+0000", 2024-09-17T21:30:36.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012608899999999999 2024-09-17T21:30:36.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:36.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460746+0000", 2024-09-17T21:30:36.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000205577 2024-09-17T21:30:36.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:36.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463800+0000", 2024-09-17T21:30:36.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030542030000000001 2024-09-17T21:30:36.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:36.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465867+0000", 2024-09-17T21:30:36.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020665929999999998 2024-09-17T21:30:36.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:36.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465914+0000", 2024-09-17T21:30:36.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497000000000003e-05 2024-09-17T21:30:36.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:36.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465980+0000", 2024-09-17T21:30:36.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6081000000000006e-05 2024-09-17T21:30:36.006 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:36.006 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:36.006 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:36.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.006 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:70218 3.16 3:68692d8f:::rbd_data.1120b9193b9f.000000000000001b:head [stat,delete] snapc 1d=[1d] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-17T21:30:36.007 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:36.007 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0993450979999997, 2024-09-17T21:30:36.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.951400786, 2024-09-17T21:30:36.007 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:36.007 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:36.007 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:36.007 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:36.007 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:36.007 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.007 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 70218 2024-09-17T21:30:36.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.007 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:36.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:36.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:36.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:36.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:36.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:36.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:36.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:36.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:36.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4799999999999998e-07 2024-09-17T21:30:36.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:36.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805642+0000", 2024-09-17T21:30:36.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2009999999999999e-06 2024-09-17T21:30:36.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:36.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805650+0000", 2024-09-17T21:30:36.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1000000000000004e-06 2024-09-17T21:30:36.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805655+0000", 2024-09-17T21:30:36.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:30:36.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:36.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805722+0000", 2024-09-17T21:30:36.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6878000000000006e-05 2024-09-17T21:30:36.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:30:36.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805733+0000", 2024-09-17T21:30:36.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0895e-05 2024-09-17T21:30:36.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:36.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755252+0000", 2024-09-17T21:30:36.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.94951843800000002 2024-09-17T21:30:36.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:36.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755398+0000", 2024-09-17T21:30:36.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014622600000000001 2024-09-17T21:30:36.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:36.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:36.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:36.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.757039+0000", 2024-09-17T21:30:36.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001640819 2024-09-17T21:30:36.012 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:36.012 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:36.012 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:36.012 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:36.012 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:36.012 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:36.015 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:30:36.019 INFO:teuthology.orchestra.run.smithi129.stdout:3810 trunc from 0x3c8b32c to 0x67591ca 2024-09-17T21:30:36.019 INFO:teuthology.orchestra.run.smithi129.stdout:3812 read 0x65a2133 thru 0x65aa6d2 (0x85a0 bytes) 2024-09-17T21:30:36.020 INFO:teuthology.orchestra.run.smithi129.stdout:3814 writesame 0xbe29a9 thru 0xbe3431 data_size 0x3(0xa89 bytes) 2024-09-17T21:30:36.020 INFO:teuthology.orchestra.run.smithi129.stdout:3816 write 0xfc48e8 thru 0xfc7173 (0x288c bytes) 2024-09-17T21:30:36.020 INFO:teuthology.orchestra.run.smithi129.stdout:3818 write 0x1a14c78 thru 0x1a19886 (0x4c0f bytes) 2024-09-17T21:30:36.020 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.020 INFO:teuthology.orchestra.run.smithi129.stdout:3820 trunc from 0x67591ca to 0xc478fcb 2024-09-17T21:30:36.020 INFO:teuthology.orchestra.run.smithi129.stdout:3822 write 0x165f172 thru 0x166a315 (0xb1a4 bytes) 2024-09-17T21:30:36.020 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.020 INFO:teuthology.orchestra.run.smithi129.stdout:3824 read 0x6527a5b thru 0x6537642 (0xfbe8 bytes) 2024-09-17T21:30:36.020 INFO:teuthology.orchestra.run.smithi129.stdout:3825 read 0x2106942 thru 0x210fe5f (0x951e bytes) 2024-09-17T21:30:36.021 INFO:teuthology.orchestra.run.smithi129.stdout:3827 read 0x92d6116 thru 0x92d6f6d (0xe58 bytes) 2024-09-17T21:30:36.021 INFO:teuthology.orchestra.run.smithi129.stdout:3828 read 0x1b3aa1b thru 0x1b3c868 (0x1e4e bytes) 2024-09-17T21:30:36.021 INFO:teuthology.orchestra.run.smithi129.stdout:3829 read 0x9ecdd86 thru 0x9edb49b (0xd716 bytes) 2024-09-17T21:30:36.021 INFO:teuthology.orchestra.run.smithi129.stdout:3830 read 0x16b43aa thru 0x16b55bf (0x1216 bytes) 2024-09-17T21:30:36.021 INFO:teuthology.orchestra.run.smithi129.stdout:3831 trunc from 0xc478fcb to 0x39d01a8 2024-09-17T21:30:36.021 INFO:teuthology.orchestra.run.smithi129.stdout:3832 read 0x5dc0c5 thru 0x5e1d17 (0x5c53 bytes) 2024-09-17T21:30:36.021 INFO:teuthology.orchestra.run.smithi129.stdout:3833 write 0x15fec8a thru 0x160776f (0x8ae6 bytes) 2024-09-17T21:30:36.021 INFO:teuthology.orchestra.run.smithi129.stdout:3836 trunc from 0x39d01a8 to 0x74d536c 2024-09-17T21:30:36.021 INFO:teuthology.orchestra.run.smithi129.stdout:3837 read 0x655adc2 thru 0x655bd0d (0xf4c bytes) 2024-09-17T21:30:36.021 INFO:teuthology.orchestra.run.smithi129.stdout:3839 trunc from 0x74d536c to 0x822ffe2 2024-09-17T21:30:36.021 INFO:teuthology.orchestra.run.smithi129.stdout:3840 read 0x77371c7 thru 0x7737d43 (0xb7d bytes) 2024-09-17T21:30:36.021 INFO:teuthology.orchestra.run.smithi129.stdout:3841 read 0xfb3495 thru 0xfbfc5c (0xc7c8 bytes) 2024-09-17T21:30:36.021 INFO:teuthology.orchestra.run.smithi129.stdout:3842 punch from 0x3e9a19a to 0x3ea3eef, (0x9d55 bytes) 2024-09-17T21:30:36.021 INFO:teuthology.orchestra.run.smithi129.stdout:3843 trunc from 0x822ffe2 to 0x88b1331 2024-09-17T21:30:36.022 INFO:teuthology.orchestra.run.smithi129.stdout:3844 punch from 0x3d7496b to 0x3d83b12, (0xf1a7 bytes) 2024-09-17T21:30:36.022 INFO:teuthology.orchestra.run.smithi129.stdout:3845 read 0x2942ae7 thru 0x294a8ec (0x7e06 bytes) 2024-09-17T21:30:36.022 INFO:teuthology.orchestra.run.smithi129.stdout:3847 read 0x21f1878 thru 0x21f7e36 (0x65bf bytes) 2024-09-17T21:30:36.022 INFO:teuthology.orchestra.run.smithi129.stdout:3848 writesame 0xa161979 thru 0xa167156 data_size 0x17(0x57de bytes) 2024-09-17T21:30:36.022 INFO:teuthology.orchestra.run.smithi129.stdout:3850 read 0x2cb3e4e thru 0x2cc1e41 (0xdff4 bytes) 2024-09-17T21:30:36.022 INFO:teuthology.orchestra.run.smithi129.stdout:3851 writesame 0x19213cf thru 0x19311b1 data_size 0x23(0xfde3 bytes) 2024-09-17T21:30:36.022 INFO:teuthology.orchestra.run.smithi129.stdout:3852 write 0x6f8a2b1 thru 0x6f92999 (0x86e9 bytes) 2024-09-17T21:30:36.022 INFO:teuthology.orchestra.run.smithi129.stdout:3855 write 0x8436bd5 thru 0x8440959 (0x9d85 bytes) 2024-09-17T21:30:36.022 INFO:teuthology.orchestra.run.smithi129.stdout:3856 read 0x6888173 thru 0x6895f87 (0xde15 bytes) 2024-09-17T21:30:36.022 INFO:teuthology.orchestra.run.smithi129.stdout:3857 writesame 0x6b827ee thru 0x6b899f9 data_size 0xc(0x720c bytes) 2024-09-17T21:30:36.022 INFO:teuthology.orchestra.run.smithi129.stdout:3858 read 0x150d596 thru 0x151693a (0x93a5 bytes) 2024-09-17T21:30:36.023 INFO:teuthology.orchestra.run.smithi129.stdout:3859 write 0x24407f6 thru 0x2441189 (0x994 bytes) 2024-09-17T21:30:36.023 INFO:teuthology.orchestra.run.smithi129.stdout:3860 read 0x30f5800 thru 0x30fea74 (0x9275 bytes) 2024-09-17T21:30:36.023 INFO:teuthology.orchestra.run.smithi129.stdout:3861 read 0x26eee81 thru 0x26ef9ec (0xb6c bytes) 2024-09-17T21:30:36.023 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:36.012+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:36.023 INFO:teuthology.orchestra.run.smithi129.stdout:3862 trunc from 0xa167157 to 0x664a338 2024-09-17T21:30:36.042 INFO:teuthology.orchestra.run.smithi129.stdout:3863 write 0xd823836 thru 0xd82e5a1 (0xad6c bytes) 2024-09-17T21:30:36.047 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.047 INFO:teuthology.orchestra.run.smithi129.stdout:3865 punch from 0x57744f1 to 0x577fc76, (0xb785 bytes) 2024-09-17T21:30:36.047 INFO:teuthology.orchestra.run.smithi129.stdout:3866 writesame 0x56d1dd6 thru 0x56d38b9 data_size 0x2(0x1ae4 bytes) 2024-09-17T21:30:36.050 INFO:teuthology.orchestra.run.smithi129.stdout:3867 punch from 0x9c7d063 to 0x9c83998, (0x6935 bytes) 2024-09-17T21:30:36.051 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.051 INFO:teuthology.orchestra.run.smithi129.stdout:3869 write 0x872375b thru 0x87298e5 (0x618b bytes) 2024-09-17T21:30:36.054 INFO:teuthology.orchestra.run.smithi129.stdout:3870 writesame 0xaac2cc1 thru 0xaaca219 data_size 0x1(0x7559 bytes) 2024-09-17T21:30:36.064 INFO:teuthology.orchestra.run.smithi129.stdout:3871 write 0x13c47ba thru 0x13ca107 (0x594e bytes) 2024-09-17T21:30:36.066 INFO:teuthology.orchestra.run.smithi129.stdout:3872 write 0x20f0f11 thru 0x20f2584 (0x1674 bytes) 2024-09-17T21:30:36.067 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.067 INFO:teuthology.orchestra.run.smithi129.stdout:3876 punch from 0x5c32283 to 0x5c33358, (0x10d5 bytes) 2024-09-17T21:30:36.067 INFO:teuthology.orchestra.run.smithi129.stdout:3877 writesame 0xcce498 thru 0xcdb979 data_size 0x5d(0xd4e2 bytes) 2024-09-17T21:30:36.069 INFO:teuthology.orchestra.run.smithi129.stdout:3880 write 0x3c4cb4e thru 0x3c56e4f (0xa302 bytes) 2024-09-17T21:30:36.074 INFO:teuthology.orchestra.run.smithi129.stdout:3881 read 0x967a13f thru 0x9687faf (0xde71 bytes) 2024-09-17T21:30:36.074 INFO:teuthology.orchestra.run.smithi129.stdout:3882 punch from 0xa019fee to 0xa01be08, (0x1e1a bytes) 2024-09-17T21:30:36.087 INFO:teuthology.orchestra.run.smithi129.stdout:3883 trunc from 0xd82e5a2 to 0x1f75fc9 2024-09-17T21:30:36.115 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:36.111+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:36.120 INFO:teuthology.orchestra.run.smithi129.stdout:3884 write 0x9cea28e thru 0x9cf4723 (0xa496 bytes) 2024-09-17T21:30:36.130 INFO:teuthology.orchestra.run.smithi129.stdout:3885 punch from 0x9a4545c to 0x9a533ed, (0xdf91 bytes) 2024-09-17T21:30:36.130 INFO:teuthology.orchestra.run.smithi129.stdout:3886 punch from 0x942cab9 to 0x9432324, (0x586b bytes) 2024-09-17T21:30:36.130 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.137 INFO:teuthology.orchestra.run.smithi129.stdout:3888 write 0xb79728f thru 0xb79b004 (0x3d76 bytes) 2024-09-17T21:30:36.139 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:30:36.139 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:36.139 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 69, 2024-09-17T21:30:36.139 INFO:teuthology.orchestra.run.smithi129.stdout: "fsid": "f6d3a80c-8a1f-4716-b25d-e97b28890c0d", 2024-09-17T21:30:36.139 INFO:teuthology.orchestra.run.smithi129.stdout: "created": "2024-09-17T21:29:12.606637+0000", 2024-09-17T21:30:36.139 INFO:teuthology.orchestra.run.smithi129.stdout: "modified": "2024-09-17T21:30:34.769847+0000", 2024-09-17T21:30:36.139 INFO:teuthology.orchestra.run.smithi129.stdout: "last_up_change": "2024-09-17T21:30:27.721456+0000", 2024-09-17T21:30:36.140 INFO:teuthology.orchestra.run.smithi129.stdout: "last_in_change": "2024-09-17T21:29:58.530050+0000", 2024-09-17T21:30:36.140 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T21:30:36.140 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_num": 5804032, 2024-09-17T21:30:36.140 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_set": [ 2024-09-17T21:30:36.140 INFO:teuthology.orchestra.run.smithi129.stdout: "nodeep-scrub", 2024-09-17T21:30:36.140 INFO:teuthology.orchestra.run.smithi129.stdout: "pglog_hardlimit", 2024-09-17T21:30:36.140 INFO:teuthology.orchestra.run.smithi129.stdout: "purged_snapdirs", 2024-09-17T21:30:36.140 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_deletes", 2024-09-17T21:30:36.140 INFO:teuthology.orchestra.run.smithi129.stdout: "sortbitwise" 2024-09-17T21:30:36.140 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:30:36.140 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_version": 6, 2024-09-17T21:30:36.140 INFO:teuthology.orchestra.run.smithi129.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T21:30:36.140 INFO:teuthology.orchestra.run.smithi129.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T21:30:36.140 INFO:teuthology.orchestra.run.smithi129.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T21:30:36.140 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_snapshot": "", 2024-09-17T21:30:36.141 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_max": 3, 2024-09-17T21:30:36.141 INFO:teuthology.orchestra.run.smithi129.stdout: "max_osd": 8, 2024-09-17T21:30:36.141 INFO:teuthology.orchestra.run.smithi129.stdout: "require_min_compat_client": "luminous", 2024-09-17T21:30:36.141 INFO:teuthology.orchestra.run.smithi129.stdout: "min_compat_client": "jewel", 2024-09-17T21:30:36.141 INFO:teuthology.orchestra.run.smithi129.stdout: "require_osd_release": "squid", 2024-09-17T21:30:36.141 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_crimson": false, 2024-09-17T21:30:36.141 INFO:teuthology.orchestra.run.smithi129.stdout: "pools": [ 2024-09-17T21:30:36.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.141 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 1, 2024-09-17T21:30:36.141 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": ".mgr", 2024-09-17T21:30:36.141 INFO:teuthology.orchestra.run.smithi129.stdout: "create_time": "2024-09-17T21:29:26.876763+0000", 2024-09-17T21:30:36.141 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-17T21:30:36.141 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_names": "hashpspool", 2024-09-17T21:30:36.141 INFO:teuthology.orchestra.run.smithi129.stdout: "type": 1, 2024-09-17T21:30:36.142 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 2, 2024-09-17T21:30:36.142 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": 1, 2024-09-17T21:30:36.142 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_rule": 0, 2024-09-17T21:30:36.142 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:30:36.142 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:30:36.142 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:30:36.142 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:30:36.142 INFO:teuthology.orchestra.run.smithi129.stdout: "is_stretch_pool": false, 2024-09-17T21:30:36.142 INFO:teuthology.orchestra.run.smithi129.stdout: "object_hash": 2, 2024-09-17T21:30:36.142 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:30:36.142 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": 1, 2024-09-17T21:30:36.142 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num": 1, 2024-09-17T21:30:36.142 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num_target": 1, 2024-09-17T21:30:36.142 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_target": 1, 2024-09-17T21:30:36.143 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_pending": 1, 2024-09-17T21:30:36.143 INFO:teuthology.orchestra.run.smithi129.stdout: "last_pg_merge_meta": { 2024-09-17T21:30:36.143 INFO:teuthology.orchestra.run.smithi129.stdout: "source_pgid": "0.0", 2024-09-17T21:30:36.143 INFO:teuthology.orchestra.run.smithi129.stdout: "ready_epoch": 0, 2024-09-17T21:30:36.143 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_started": 0, 2024-09-17T21:30:36.143 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_clean": 0, 2024-09-17T21:30:36.143 INFO:teuthology.orchestra.run.smithi129.stdout: "source_version": "0'0", 2024-09-17T21:30:36.143 INFO:teuthology.orchestra.run.smithi129.stdout: "target_version": "0'0" 2024-09-17T21:30:36.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.143 INFO:teuthology.orchestra.run.smithi129.stdout: "last_change": "19", 2024-09-17T21:30:36.143 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend": "0", 2024-09-17T21:30:36.143 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:30:36.143 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:30:36.144 INFO:teuthology.orchestra.run.smithi129.stdout: "auid": 0, 2024-09-17T21:30:36.144 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:30:36.144 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_seq": 0, 2024-09-17T21:30:36.144 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_epoch": 0, 2024-09-17T21:30:36.144 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_snaps": [], 2024-09-17T21:30:36.144 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps": "[]", 2024-09-17T21:30:36.144 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_bytes": 0, 2024-09-17T21:30:36.144 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_objects": 0, 2024-09-17T21:30:36.144 INFO:teuthology.orchestra.run.smithi129.stdout: "tiers": [], 2024-09-17T21:30:36.144 INFO:teuthology.orchestra.run.smithi129.stdout: "tier_of": -1, 2024-09-17T21:30:36.144 INFO:teuthology.orchestra.run.smithi129.stdout: "read_tier": -1, 2024-09-17T21:30:36.144 INFO:teuthology.orchestra.run.smithi129.stdout: "write_tier": -1, 2024-09-17T21:30:36.144 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_mode": "none", 2024-09-17T21:30:36.144 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_bytes": 0, 2024-09-17T21:30:36.145 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_objects": 0, 2024-09-17T21:30:36.145 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:30:36.145 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:30:36.145 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:30:36.145 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_flush_age": 0, 2024-09-17T21:30:36.145 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_evict_age": 0, 2024-09-17T21:30:36.145 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profile": "", 2024-09-17T21:30:36.145 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_params": { 2024-09-17T21:30:36.145 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "none" 2024-09-17T21:30:36.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.145 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_period": 0, 2024-09-17T21:30:36.145 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_count": 0, 2024-09-17T21:30:36.145 INFO:teuthology.orchestra.run.smithi129.stdout: "use_gmt_hitset": true, 2024-09-17T21:30:36.145 INFO:teuthology.orchestra.run.smithi129.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:30:36.146 INFO:teuthology.orchestra.run.smithi129.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:30:36.146 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:30:36.146 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:30:36.146 INFO:teuthology.orchestra.run.smithi129.stdout: "grade_table": [], 2024-09-17T21:30:36.146 INFO:teuthology.orchestra.run.smithi129.stdout: "stripe_width": 0, 2024-09-17T21:30:36.146 INFO:teuthology.orchestra.run.smithi129.stdout: "expected_num_objects": 0, 2024-09-17T21:30:36.146 INFO:teuthology.orchestra.run.smithi129.stdout: "fast_read": false, 2024-09-17T21:30:36.146 INFO:teuthology.orchestra.run.smithi129.stdout: "options": { 2024-09-17T21:30:36.146 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_max": 32, 2024-09-17T21:30:36.146 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_min": 1 2024-09-17T21:30:36.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.146 INFO:teuthology.orchestra.run.smithi129.stdout: "application_metadata": { 2024-09-17T21:30:36.146 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr": {} 2024-09-17T21:30:36.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.147 INFO:teuthology.orchestra.run.smithi129.stdout: "read_balance": { 2024-09-17T21:30:36.147 INFO:teuthology.orchestra.run.smithi129.stdout: "score_type": "Fair distribution", 2024-09-17T21:30:36.147 INFO:teuthology.orchestra.run.smithi129.stdout: "score_acting": 8, 2024-09-17T21:30:36.147 INFO:teuthology.orchestra.run.smithi129.stdout: "score_stable": 8, 2024-09-17T21:30:36.147 INFO:teuthology.orchestra.run.smithi129.stdout: "optimal_score": 0.25, 2024-09-17T21:30:36.147 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_acting": 2, 2024-09-17T21:30:36.147 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_stable": 2, 2024-09-17T21:30:36.147 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:30:36.147 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity": 1, 2024-09-17T21:30:36.147 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:30:36.147 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.148 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 2, 2024-09-17T21:30:36.148 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": "rbd", 2024-09-17T21:30:36.148 INFO:teuthology.orchestra.run.smithi129.stdout: "create_time": "2024-09-17T21:29:31.078466+0000", 2024-09-17T21:30:36.148 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 8193, 2024-09-17T21:30:36.148 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:30:36.148 INFO:teuthology.orchestra.run.smithi129.stdout: "type": 1, 2024-09-17T21:30:36.148 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 2, 2024-09-17T21:30:36.148 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": 1, 2024-09-17T21:30:36.148 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_rule": 0, 2024-09-17T21:30:36.148 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:30:36.148 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:30:36.148 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:30:36.148 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:30:36.149 INFO:teuthology.orchestra.run.smithi129.stdout: "is_stretch_pool": false, 2024-09-17T21:30:36.149 INFO:teuthology.orchestra.run.smithi129.stdout: "object_hash": 2, 2024-09-17T21:30:36.149 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:30:36.149 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": 8, 2024-09-17T21:30:36.149 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num": 8, 2024-09-17T21:30:36.149 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num_target": 8, 2024-09-17T21:30:36.149 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_target": 8, 2024-09-17T21:30:36.149 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_pending": 8, 2024-09-17T21:30:36.149 INFO:teuthology.orchestra.run.smithi129.stdout: "last_pg_merge_meta": { 2024-09-17T21:30:36.149 INFO:teuthology.orchestra.run.smithi129.stdout: "source_pgid": "0.0", 2024-09-17T21:30:36.149 INFO:teuthology.orchestra.run.smithi129.stdout: "ready_epoch": 0, 2024-09-17T21:30:36.149 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_started": 0, 2024-09-17T21:30:36.149 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_clean": 0, 2024-09-17T21:30:36.149 INFO:teuthology.orchestra.run.smithi129.stdout: "source_version": "0'0", 2024-09-17T21:30:36.150 INFO:teuthology.orchestra.run.smithi129.stdout: "target_version": "0'0" 2024-09-17T21:30:36.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.150 INFO:teuthology.orchestra.run.smithi129.stdout: "last_change": "23", 2024-09-17T21:30:36.150 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend": "0", 2024-09-17T21:30:36.150 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:30:36.150 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:30:36.150 INFO:teuthology.orchestra.run.smithi129.stdout: "auid": 0, 2024-09-17T21:30:36.150 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:30:36.150 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_seq": 2, 2024-09-17T21:30:36.150 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_epoch": 23, 2024-09-17T21:30:36.150 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_snaps": [], 2024-09-17T21:30:36.150 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps": "[]", 2024-09-17T21:30:36.150 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_bytes": 0, 2024-09-17T21:30:36.150 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_objects": 0, 2024-09-17T21:30:36.151 INFO:teuthology.orchestra.run.smithi129.stdout: "tiers": [], 2024-09-17T21:30:36.151 INFO:teuthology.orchestra.run.smithi129.stdout: "tier_of": -1, 2024-09-17T21:30:36.151 INFO:teuthology.orchestra.run.smithi129.stdout: "read_tier": -1, 2024-09-17T21:30:36.151 INFO:teuthology.orchestra.run.smithi129.stdout: "write_tier": -1, 2024-09-17T21:30:36.151 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_mode": "none", 2024-09-17T21:30:36.151 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_bytes": 0, 2024-09-17T21:30:36.151 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_objects": 0, 2024-09-17T21:30:36.151 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:30:36.151 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:30:36.151 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:30:36.151 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_flush_age": 0, 2024-09-17T21:30:36.151 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_evict_age": 0, 2024-09-17T21:30:36.151 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profile": "", 2024-09-17T21:30:36.151 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_params": { 2024-09-17T21:30:36.151 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "none" 2024-09-17T21:30:36.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.152 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_period": 0, 2024-09-17T21:30:36.152 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_count": 0, 2024-09-17T21:30:36.152 INFO:teuthology.orchestra.run.smithi129.stdout: "use_gmt_hitset": true, 2024-09-17T21:30:36.152 INFO:teuthology.orchestra.run.smithi129.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:30:36.152 INFO:teuthology.orchestra.run.smithi129.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:30:36.152 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:30:36.152 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:30:36.152 INFO:teuthology.orchestra.run.smithi129.stdout: "grade_table": [], 2024-09-17T21:30:36.152 INFO:teuthology.orchestra.run.smithi129.stdout: "stripe_width": 0, 2024-09-17T21:30:36.152 INFO:teuthology.orchestra.run.smithi129.stdout: "expected_num_objects": 0, 2024-09-17T21:30:36.152 INFO:teuthology.orchestra.run.smithi129.stdout: "fast_read": false, 2024-09-17T21:30:36.152 INFO:teuthology.orchestra.run.smithi129.stdout: "options": {}, 2024-09-17T21:30:36.153 INFO:teuthology.orchestra.run.smithi129.stdout: "application_metadata": { 2024-09-17T21:30:36.153 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd": {} 2024-09-17T21:30:36.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.153 INFO:teuthology.orchestra.run.smithi129.stdout: "read_balance": { 2024-09-17T21:30:36.153 INFO:teuthology.orchestra.run.smithi129.stdout: "score_type": "Fair distribution", 2024-09-17T21:30:36.153 INFO:teuthology.orchestra.run.smithi129.stdout: "score_acting": 3, 2024-09-17T21:30:36.153 INFO:teuthology.orchestra.run.smithi129.stdout: "score_stable": 3, 2024-09-17T21:30:36.153 INFO:teuthology.orchestra.run.smithi129.stdout: "optimal_score": 0.75, 2024-09-17T21:30:36.153 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_acting": 2.25, 2024-09-17T21:30:36.153 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_stable": 2.25, 2024-09-17T21:30:36.153 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:30:36.153 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity": 1, 2024-09-17T21:30:36.153 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:30:36.154 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.154 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 3, 2024-09-17T21:30:36.154 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": "pool_client.0", 2024-09-17T21:30:36.154 INFO:teuthology.orchestra.run.smithi129.stdout: "create_time": "2024-09-17T21:29:39.892418+0000", 2024-09-17T21:30:36.154 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 8193, 2024-09-17T21:30:36.154 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:30:36.154 INFO:teuthology.orchestra.run.smithi129.stdout: "type": 1, 2024-09-17T21:30:36.154 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 2, 2024-09-17T21:30:36.154 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": 1, 2024-09-17T21:30:36.154 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_rule": 0, 2024-09-17T21:30:36.154 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:30:36.154 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:30:36.154 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:30:36.155 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:30:36.155 INFO:teuthology.orchestra.run.smithi129.stdout: "is_stretch_pool": false, 2024-09-17T21:30:36.155 INFO:teuthology.orchestra.run.smithi129.stdout: "object_hash": 2, 2024-09-17T21:30:36.155 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:30:36.155 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": 32, 2024-09-17T21:30:36.155 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num": 32, 2024-09-17T21:30:36.155 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num_target": 32, 2024-09-17T21:30:36.155 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_target": 32, 2024-09-17T21:30:36.155 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_pending": 32, 2024-09-17T21:30:36.155 INFO:teuthology.orchestra.run.smithi129.stdout: "last_pg_merge_meta": { 2024-09-17T21:30:36.155 INFO:teuthology.orchestra.run.smithi129.stdout: "source_pgid": "0.0", 2024-09-17T21:30:36.155 INFO:teuthology.orchestra.run.smithi129.stdout: "ready_epoch": 0, 2024-09-17T21:30:36.155 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_started": 0, 2024-09-17T21:30:36.155 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_clean": 0, 2024-09-17T21:30:36.156 INFO:teuthology.orchestra.run.smithi129.stdout: "source_version": "0'0", 2024-09-17T21:30:36.156 INFO:teuthology.orchestra.run.smithi129.stdout: "target_version": "0'0" 2024-09-17T21:30:36.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.156 INFO:teuthology.orchestra.run.smithi129.stdout: "last_change": "69", 2024-09-17T21:30:36.156 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend": "0", 2024-09-17T21:30:36.156 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:30:36.156 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:30:36.156 INFO:teuthology.orchestra.run.smithi129.stdout: "auid": 0, 2024-09-17T21:30:36.156 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:30:36.156 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_seq": 31, 2024-09-17T21:30:36.156 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_epoch": 69, 2024-09-17T21:30:36.156 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_snaps": [], 2024-09-17T21:30:36.156 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps": "[]", 2024-09-17T21:30:36.157 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_bytes": 0, 2024-09-17T21:30:36.157 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_objects": 0, 2024-09-17T21:30:36.157 INFO:teuthology.orchestra.run.smithi129.stdout: "tiers": [], 2024-09-17T21:30:36.157 INFO:teuthology.orchestra.run.smithi129.stdout: "tier_of": -1, 2024-09-17T21:30:36.157 INFO:teuthology.orchestra.run.smithi129.stdout: "read_tier": -1, 2024-09-17T21:30:36.157 INFO:teuthology.orchestra.run.smithi129.stdout: "write_tier": -1, 2024-09-17T21:30:36.157 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_mode": "none", 2024-09-17T21:30:36.157 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_bytes": 0, 2024-09-17T21:30:36.157 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_objects": 0, 2024-09-17T21:30:36.157 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:30:36.157 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:30:36.157 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:30:36.157 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_flush_age": 0, 2024-09-17T21:30:36.157 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_evict_age": 0, 2024-09-17T21:30:36.158 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profile": "", 2024-09-17T21:30:36.158 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_params": { 2024-09-17T21:30:36.158 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "none" 2024-09-17T21:30:36.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.158 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_period": 0, 2024-09-17T21:30:36.158 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_count": 0, 2024-09-17T21:30:36.158 INFO:teuthology.orchestra.run.smithi129.stdout: "use_gmt_hitset": true, 2024-09-17T21:30:36.158 INFO:teuthology.orchestra.run.smithi129.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:30:36.158 INFO:teuthology.orchestra.run.smithi129.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:30:36.158 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:30:36.158 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:30:36.158 INFO:teuthology.orchestra.run.smithi129.stdout: "grade_table": [], 2024-09-17T21:30:36.158 INFO:teuthology.orchestra.run.smithi129.stdout: "stripe_width": 0, 2024-09-17T21:30:36.158 INFO:teuthology.orchestra.run.smithi129.stdout: "expected_num_objects": 0, 2024-09-17T21:30:36.159 INFO:teuthology.orchestra.run.smithi129.stdout: "fast_read": false, 2024-09-17T21:30:36.159 INFO:teuthology.orchestra.run.smithi129.stdout: "options": {}, 2024-09-17T21:30:36.159 INFO:teuthology.orchestra.run.smithi129.stdout: "application_metadata": { 2024-09-17T21:30:36.159 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd": {} 2024-09-17T21:30:36.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.159 INFO:teuthology.orchestra.run.smithi129.stdout: "read_balance": { 2024-09-17T21:30:36.159 INFO:teuthology.orchestra.run.smithi129.stdout: "score_type": "Fair distribution", 2024-09-17T21:30:36.159 INFO:teuthology.orchestra.run.smithi129.stdout: "score_acting": 1.9900000095367432, 2024-09-17T21:30:36.159 INFO:teuthology.orchestra.run.smithi129.stdout: "score_stable": 1.9900000095367432, 2024-09-17T21:30:36.159 INFO:teuthology.orchestra.run.smithi129.stdout: "optimal_score": 0.87999999523162842, 2024-09-17T21:30:36.159 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_acting": 1.75, 2024-09-17T21:30:36.159 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_stable": 1.75, 2024-09-17T21:30:36.159 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:30:36.159 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity": 1, 2024-09-17T21:30:36.160 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:30:36.160 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.160 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.160 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:30:36.160 INFO:teuthology.orchestra.run.smithi129.stdout: "osds": [ 2024-09-17T21:30:36.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.160 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 0, 2024-09-17T21:30:36.160 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "ad067b51-9402-4f69-a0f0-08fbf068dcfc", 2024-09-17T21:30:36.160 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:30:36.160 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:30:36.160 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:30:36.160 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:30:36.160 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:30:36.161 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:30:36.161 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 13, 2024-09-17T21:30:36.161 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 66, 2024-09-17T21:30:36.161 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:30:36.161 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:30:36.161 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:30:36.161 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.161 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.161 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6800", 2024-09-17T21:30:36.161 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:30:36.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.161 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.161 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6801", 2024-09-17T21:30:36.162 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:30:36.162 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.162 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.162 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:30:36.162 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.162 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.162 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6802", 2024-09-17T21:30:36.162 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:30:36.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.162 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.163 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6803", 2024-09-17T21:30:36.163 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:30:36.163 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.163 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.163 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:30:36.163 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.163 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.163 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6806", 2024-09-17T21:30:36.163 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:30:36.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.163 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.164 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6807", 2024-09-17T21:30:36.164 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:30:36.164 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.164 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.164 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:30:36.164 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.164 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.164 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6804", 2024-09-17T21:30:36.164 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:30:36.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.165 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.165 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6805", 2024-09-17T21:30:36.165 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:30:36.165 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.165 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.165 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6801/3635367421", 2024-09-17T21:30:36.165 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6803/3635367421", 2024-09-17T21:30:36.165 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6807/3635367421", 2024-09-17T21:30:36.165 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6805/3635367421", 2024-09-17T21:30:36.165 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:30:36.165 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:30:36.165 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:30:36.166 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.166 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 1, 2024-09-17T21:30:36.166 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "30385fc0-ef5f-45f5-a3ec-37e890d4826e", 2024-09-17T21:30:36.166 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:30:36.166 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:30:36.166 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:30:36.166 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:30:36.166 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 15, 2024-09-17T21:30:36.166 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 47, 2024-09-17T21:30:36.166 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 63, 2024-09-17T21:30:36.166 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 66, 2024-09-17T21:30:36.166 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 48, 2024-09-17T21:30:36.167 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:30:36.167 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:30:36.167 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.167 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.167 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6818", 2024-09-17T21:30:36.167 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:30:36.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.167 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.167 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6819", 2024-09-17T21:30:36.167 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:30:36.167 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.167 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.168 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:30:36.168 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.168 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.168 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6820", 2024-09-17T21:30:36.168 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:30:36.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.168 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.168 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6821", 2024-09-17T21:30:36.168 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:30:36.168 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.168 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.169 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:30:36.169 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.169 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.169 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6824", 2024-09-17T21:30:36.169 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:30:36.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.169 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.169 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6825", 2024-09-17T21:30:36.169 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:30:36.169 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.169 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.170 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:30:36.170 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.170 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.170 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6822", 2024-09-17T21:30:36.170 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:30:36.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.170 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.170 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6823", 2024-09-17T21:30:36.170 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:30:36.170 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.170 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.171 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6819/842542464", 2024-09-17T21:30:36.171 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6821/842542464", 2024-09-17T21:30:36.171 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6825/842542464", 2024-09-17T21:30:36.171 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6823/842542464", 2024-09-17T21:30:36.171 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:30:36.171 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:30:36.171 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:30:36.171 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.171 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 2, 2024-09-17T21:30:36.171 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "76be9b2b-b63f-402a-9ab5-2564bd671ac3", 2024-09-17T21:30:36.171 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:30:36.171 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:30:36.172 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:30:36.172 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:30:36.172 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:30:36.172 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:30:36.172 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:30:36.172 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 65, 2024-09-17T21:30:36.172 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:30:36.172 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:30:36.172 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:30:36.172 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.172 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.172 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6808", 2024-09-17T21:30:36.172 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:30:36.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.173 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.173 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6809", 2024-09-17T21:30:36.173 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:30:36.173 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.173 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.173 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:30:36.173 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.173 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.173 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6810", 2024-09-17T21:30:36.173 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:30:36.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.174 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.174 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6811", 2024-09-17T21:30:36.174 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:30:36.174 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.174 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.174 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:30:36.174 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.174 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.174 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6814", 2024-09-17T21:30:36.174 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:30:36.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.175 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.175 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6815", 2024-09-17T21:30:36.175 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:30:36.175 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.175 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.175 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:30:36.175 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.175 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.175 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6812", 2024-09-17T21:30:36.175 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:30:36.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.176 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.176 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6813", 2024-09-17T21:30:36.176 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:30:36.176 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.176 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.176 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6809/1790706405", 2024-09-17T21:30:36.176 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6811/1790706405", 2024-09-17T21:30:36.176 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6815/1790706405", 2024-09-17T21:30:36.176 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6813/1790706405", 2024-09-17T21:30:36.176 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:30:36.176 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:30:36.176 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:30:36.177 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.177 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 3, 2024-09-17T21:30:36.177 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "d6c34e3b-1ac6-415c-9c56-38ae77a5c9b0", 2024-09-17T21:30:36.177 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:30:36.177 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:30:36.177 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:30:36.177 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:30:36.177 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:30:36.177 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:30:36.177 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:30:36.177 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 66, 2024-09-17T21:30:36.177 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:30:36.177 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:30:36.178 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:30:36.178 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.178 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.178 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6826", 2024-09-17T21:30:36.178 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:30:36.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.178 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.178 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6827", 2024-09-17T21:30:36.178 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:30:36.178 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.178 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.178 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:30:36.179 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.179 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.179 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6828", 2024-09-17T21:30:36.179 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:30:36.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.179 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.179 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6829", 2024-09-17T21:30:36.179 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:30:36.179 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.179 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.179 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:30:36.179 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.180 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.180 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6832", 2024-09-17T21:30:36.180 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:30:36.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.180 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.180 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6833", 2024-09-17T21:30:36.180 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:30:36.180 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.180 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.180 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:30:36.180 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.181 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.181 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6830", 2024-09-17T21:30:36.181 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:30:36.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.181 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.181 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6831", 2024-09-17T21:30:36.181 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:30:36.181 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.181 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.181 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6827/393361453", 2024-09-17T21:30:36.181 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6829/393361453", 2024-09-17T21:30:36.181 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6833/393361453", 2024-09-17T21:30:36.181 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6831/393361453", 2024-09-17T21:30:36.182 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:30:36.182 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:30:36.182 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:30:36.182 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.182 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 4, 2024-09-17T21:30:36.182 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "7d722e8c-c497-4f0c-9f61-d9826f8c93d8", 2024-09-17T21:30:36.182 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:30:36.182 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:30:36.182 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:30:36.182 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:30:36.182 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:30:36.183 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:30:36.183 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:30:36.183 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 63, 2024-09-17T21:30:36.183 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:30:36.183 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:30:36.183 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:30:36.183 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.183 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.183 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6808", 2024-09-17T21:30:36.183 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:30:36.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.183 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.184 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6809", 2024-09-17T21:30:36.184 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:30:36.184 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.184 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.184 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:30:36.184 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.184 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.184 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6810", 2024-09-17T21:30:36.184 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:30:36.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.185 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.185 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6811", 2024-09-17T21:30:36.185 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:30:36.185 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.185 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.185 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:30:36.185 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.185 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.185 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6814", 2024-09-17T21:30:36.185 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:30:36.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.186 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.186 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6815", 2024-09-17T21:30:36.186 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:30:36.186 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.186 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.186 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:30:36.186 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.186 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.186 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6812", 2024-09-17T21:30:36.186 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:30:36.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.187 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.187 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6813", 2024-09-17T21:30:36.187 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:30:36.187 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.187 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.187 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6809/2337546183", 2024-09-17T21:30:36.187 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6811/2337546183", 2024-09-17T21:30:36.187 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6815/2337546183", 2024-09-17T21:30:36.187 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6813/2337546183", 2024-09-17T21:30:36.187 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:30:36.187 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:30:36.187 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:30:36.187 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.188 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 5, 2024-09-17T21:30:36.188 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "954f801b-3ac9-410a-a375-eb1c533744de", 2024-09-17T21:30:36.188 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:30:36.188 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 0, 2024-09-17T21:30:36.188 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 0, 2024-09-17T21:30:36.188 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:30:36.188 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:30:36.188 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:30:36.188 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:30:36.188 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 25, 2024-09-17T21:30:36.188 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:30:36.188 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:30:36.189 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:30:36.189 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.189 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.189 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6816", 2024-09-17T21:30:36.189 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:30:36.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.189 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.189 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6817", 2024-09-17T21:30:36.189 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:30:36.189 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.189 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.189 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:30:36.190 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.190 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.190 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6818", 2024-09-17T21:30:36.190 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:30:36.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.190 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.190 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6819", 2024-09-17T21:30:36.190 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:30:36.190 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.190 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.190 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:30:36.191 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.191 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.191 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6822", 2024-09-17T21:30:36.191 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:30:36.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.191 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.191 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6823", 2024-09-17T21:30:36.191 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:30:36.191 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.191 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.191 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:30:36.191 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.192 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.192 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6820", 2024-09-17T21:30:36.192 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:30:36.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.192 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.192 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6821", 2024-09-17T21:30:36.192 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:30:36.192 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.192 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.192 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6817/1510320351", 2024-09-17T21:30:36.192 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6819/1510320351", 2024-09-17T21:30:36.193 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6823/1510320351", 2024-09-17T21:30:36.193 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6821/1510320351", 2024-09-17T21:30:36.193 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:30:36.193 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:30:36.193 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:30:36.193 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.193 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 6, 2024-09-17T21:30:36.193 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "21837515-65a2-4f21-96d6-67cfc4589358", 2024-09-17T21:30:36.193 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:30:36.193 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:30:36.193 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:30:36.193 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:30:36.194 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:30:36.194 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:30:36.194 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:30:36.194 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 48, 2024-09-17T21:30:36.194 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:30:36.194 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:30:36.194 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:30:36.194 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.194 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.194 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6824", 2024-09-17T21:30:36.194 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:30:36.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.195 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.195 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6825", 2024-09-17T21:30:36.195 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:30:36.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.195 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.195 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:30:36.195 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.195 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.195 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6826", 2024-09-17T21:30:36.195 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:30:36.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.196 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.196 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6827", 2024-09-17T21:30:36.196 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:30:36.196 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.196 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.196 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:30:36.196 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.196 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.196 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6830", 2024-09-17T21:30:36.196 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:30:36.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.197 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.197 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6831", 2024-09-17T21:30:36.197 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:30:36.197 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.197 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.197 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:30:36.197 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.197 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.197 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6828", 2024-09-17T21:30:36.197 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:30:36.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.198 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.198 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6829", 2024-09-17T21:30:36.198 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:30:36.198 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.198 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.198 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6825/471043930", 2024-09-17T21:30:36.198 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6827/471043930", 2024-09-17T21:30:36.198 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6831/471043930", 2024-09-17T21:30:36.198 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6829/471043930", 2024-09-17T21:30:36.198 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:30:36.198 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:30:36.198 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:30:36.199 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.199 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 7, 2024-09-17T21:30:36.199 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "67af7eea-8803-425d-ae50-30a6ab91a711", 2024-09-17T21:30:36.199 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:30:36.199 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:30:36.199 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:30:36.199 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:30:36.199 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:30:36.199 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:30:36.199 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:30:36.199 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 66, 2024-09-17T21:30:36.199 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:30:36.200 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:30:36.200 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:30:36.200 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.200 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.200 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6800", 2024-09-17T21:30:36.200 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:30:36.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.200 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.200 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6801", 2024-09-17T21:30:36.200 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:30:36.200 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.200 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.201 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:30:36.201 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.201 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.201 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6802", 2024-09-17T21:30:36.201 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:30:36.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.201 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.201 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6803", 2024-09-17T21:30:36.201 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:30:36.201 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.201 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.202 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:30:36.202 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.202 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.202 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6806", 2024-09-17T21:30:36.202 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:30:36.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.202 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.202 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6807", 2024-09-17T21:30:36.202 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:30:36.202 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.202 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.203 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:30:36.203 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:30:36.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.203 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:30:36.203 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6804", 2024-09-17T21:30:36.203 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:30:36.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.203 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:30:36.203 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6805", 2024-09-17T21:30:36.203 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:30:36.203 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.203 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.204 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6801/1770619177", 2024-09-17T21:30:36.204 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6803/1770619177", 2024-09-17T21:30:36.204 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6807/1770619177", 2024-09-17T21:30:36.204 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6805/1770619177", 2024-09-17T21:30:36.204 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:30:36.204 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:30:36.204 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:30:36.204 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.204 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.204 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:30:36.204 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_xinfo": [ 2024-09-17T21:30:36.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.204 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 0, 2024-09-17T21:30:36.204 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:30:36.205 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:30:36.205 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:30:36.205 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:30:36.205 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:30:36.205 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:21.445879+0000", 2024-09-17T21:30:36.205 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:30:36.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.205 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 1, 2024-09-17T21:30:36.205 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:30:36.205 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:30:36.206 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:30:36.206 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:30:36.206 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:30:36.206 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.240967+0000", 2024-09-17T21:30:36.206 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 48 2024-09-17T21:30:36.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.206 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 2, 2024-09-17T21:30:36.206 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:30:36.206 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:30:36.206 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:30:36.206 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:30:36.206 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:30:36.207 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.039097+0000", 2024-09-17T21:30:36.207 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:30:36.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.207 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 3, 2024-09-17T21:30:36.207 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:30:36.207 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:30:36.207 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:30:36.207 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:30:36.207 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:30:36.207 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.372014+0000", 2024-09-17T21:30:36.207 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:30:36.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.208 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 4, 2024-09-17T21:30:36.208 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:30:36.208 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:30:36.208 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:30:36.208 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:30:36.208 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:30:36.208 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.049543+0000", 2024-09-17T21:30:36.208 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:30:36.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.208 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 5, 2024-09-17T21:30:36.208 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:30:36.208 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:30:36.209 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:30:36.209 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:30:36.209 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 65536, 2024-09-17T21:30:36.209 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.009384+0000", 2024-09-17T21:30:36.209 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:30:36.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.209 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 6, 2024-09-17T21:30:36.209 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:30:36.209 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:30:36.209 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:30:36.209 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:30:36.209 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:30:36.209 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.048598+0000", 2024-09-17T21:30:36.210 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:30:36.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.210 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 7, 2024-09-17T21:30:36.210 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:30:36.210 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:30:36.210 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:30:36.210 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:30:36.210 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:30:36.210 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.046345+0000", 2024-09-17T21:30:36.210 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:30:36.210 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.210 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:30:36.210 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_upmap": [], 2024-09-17T21:30:36.211 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_upmap_items": [], 2024-09-17T21:30:36.211 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_upmap_primaries": [], 2024-09-17T21:30:36.211 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_temp": [], 2024-09-17T21:30:36.211 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_temp": [], 2024-09-17T21:30:36.211 INFO:teuthology.orchestra.run.smithi129.stdout: "blocklist": {}, 2024-09-17T21:30:36.211 INFO:teuthology.orchestra.run.smithi129.stdout: "range_blocklist": {}, 2024-09-17T21:30:36.211 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profiles": { 2024-09-17T21:30:36.211 INFO:teuthology.orchestra.run.smithi129.stdout: "default": { 2024-09-17T21:30:36.211 INFO:teuthology.orchestra.run.smithi129.stdout: "crush-failure-domain": "osd", 2024-09-17T21:30:36.211 INFO:teuthology.orchestra.run.smithi129.stdout: "k": "2", 2024-09-17T21:30:36.211 INFO:teuthology.orchestra.run.smithi129.stdout: "m": "1", 2024-09-17T21:30:36.211 INFO:teuthology.orchestra.run.smithi129.stdout: "plugin": "jerasure", 2024-09-17T21:30:36.211 INFO:teuthology.orchestra.run.smithi129.stdout: "technique": "reed_sol_van" 2024-09-17T21:30:36.211 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.211 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps_queue": [], 2024-09-17T21:30:36.212 INFO:teuthology.orchestra.run.smithi129.stdout: "new_removed_snaps": [], 2024-09-17T21:30:36.212 INFO:teuthology.orchestra.run.smithi129.stdout: "new_purged_snaps": [], 2024-09-17T21:30:36.212 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_node_flags": {}, 2024-09-17T21:30:36.212 INFO:teuthology.orchestra.run.smithi129.stdout: "device_class_flags": {}, 2024-09-17T21:30:36.212 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_mode": { 2024-09-17T21:30:36.212 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_mode_enabled": false, 2024-09-17T21:30:36.212 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_bucket_count": 0, 2024-09-17T21:30:36.212 INFO:teuthology.orchestra.run.smithi129.stdout: "degraded_stretch_mode": 0, 2024-09-17T21:30:36.212 INFO:teuthology.orchestra.run.smithi129.stdout: "recovering_stretch_mode": 0, 2024-09-17T21:30:36.212 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_mode_bucket": 0 2024-09-17T21:30:36.212 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.212 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:36.213 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:36.213 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:36.213 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:36.213 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:36.213 INFO:teuthology.orchestra.run.smithi129.stdout:3889 write 0xdd39071 thru 0xdd424d6 (0x9466 bytes) 2024-09-17T21:30:36.215 INFO:tasks.thrashosds.thrasher:j is {'epoch': 69, 'fsid': 'f6d3a80c-8a1f-4716-b25d-e97b28890c0d', 'created': '2024-09-17T21:29:12.606637+0000', 'modified': '2024-09-17T21:30:34.769847+0000', 'last_up_change': '2024-09-17T21:30:27.721456+0000', 'last_in_change': '2024-09-17T21:29:58.530050+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T21:29:26.876763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T21:29:31.078466+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T21:29:39.892418+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 31, 'snap_epoch': 69, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'ad067b51-9402-4f69-a0f0-08fbf068dcfc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 66, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6800', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6801', 'nonce': 3635367421}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6802', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6803', 'nonce': 3635367421}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6806', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6807', 'nonce': 3635367421}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6804', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6805', 'nonce': 3635367421}]}, 'public_addr': '172.21.15.129:6801/3635367421', 'cluster_addr': '172.21.15.129:6803/3635367421', 'heartbeat_back_addr': '172.21.15.129:6807/3635367421', 'heartbeat_front_addr': '172.21.15.129:6805/3635367421', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '30385fc0-ef5f-45f5-a3ec-37e890d4826e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 47, 'up_from': 63, 'up_thru': 66, 'down_at': 48, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6818', 'nonce': 842542464}, {'type': 'v1', 'addr': '172.21.15.129:6819', 'nonce': 842542464}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6820', 'nonce': 842542464}, {'type': 'v1', 'addr': '172.21.15.129:6821', 'nonce': 842542464}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6824', 'nonce': 842542464}, {'type': 'v1', 'addr': '172.21.15.129:6825', 'nonce': 842542464}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6822', 'nonce': 842542464}, {'type': 'v1', 'addr': '172.21.15.129:6823', 'nonce': 842542464}]}, 'public_addr': '172.21.15.129:6819/842542464', 'cluster_addr': '172.21.15.129:6821/842542464', 'heartbeat_back_addr': '172.21.15.129:6825/842542464', 'heartbeat_front_addr': '172.21.15.129:6823/842542464', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '76be9b2b-b63f-402a-9ab5-2564bd671ac3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 65, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6808', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6809', 'nonce': 1790706405}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6810', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6811', 'nonce': 1790706405}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6814', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6815', 'nonce': 1790706405}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6812', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6813', 'nonce': 1790706405}]}, 'public_addr': '172.21.15.129:6809/1790706405', 'cluster_addr': '172.21.15.129:6811/1790706405', 'heartbeat_back_addr': '172.21.15.129:6815/1790706405', 'heartbeat_front_addr': '172.21.15.129:6813/1790706405', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'd6c34e3b-1ac6-415c-9c56-38ae77a5c9b0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 66, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6826', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6827', 'nonce': 393361453}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6828', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6829', 'nonce': 393361453}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6832', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6833', 'nonce': 393361453}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6830', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6831', 'nonce': 393361453}]}, 'public_addr': '172.21.15.129:6827/393361453', 'cluster_addr': '172.21.15.129:6829/393361453', 'heartbeat_back_addr': '172.21.15.129:6833/393361453', 'heartbeat_front_addr': '172.21.15.129:6831/393361453', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '7d722e8c-c497-4f0c-9f61-d9826f8c93d8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 63, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6808', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6809', 'nonce': 2337546183}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6810', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6811', 'nonce': 2337546183}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6814', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6815', 'nonce': 2337546183}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6812', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6813', 'nonce': 2337546183}]}, 'public_addr': '172.21.15.171:6809/2337546183', 'cluster_addr': '172.21.15.171:6811/2337546183', 'heartbeat_back_addr': '172.21.15.171:6815/2337546183', 'heartbeat_front_addr': '172.21.15.171:6813/2337546183', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '954f801b-3ac9-410a-a375-eb1c533744de', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6816', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6817', 'nonce': 1510320351}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6818', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6819', 'nonce': 1510320351}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6822', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6823', 'nonce': 1510320351}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6820', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6821', 'nonce': 1510320351}]}, 'public_addr': '172.21.15.171:6817/1510320351', 'cluster_addr': '172.21.15.171:6819/1510320351', 'heartbeat_back_addr': '172.21.15.171:6823/1510320351', 'heartbeat_front_addr': '172.21.15.171:6821/1510320351', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '21837515-65a2-4f21-96d6-67cfc4589358', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6824', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6825', 'nonce': 471043930}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6826', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6827', 'nonce': 471043930}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6830', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6831', 'nonce': 471043930}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6828', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6829', 'nonce': 471043930}]}, 'public_addr': '172.21.15.171:6825/471043930', 'cluster_addr': '172.21.15.171:6827/471043930', 'heartbeat_back_addr': '172.21.15.171:6831/471043930', 'heartbeat_front_addr': '172.21.15.171:6829/471043930', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '67af7eea-8803-425d-ae50-30a6ab91a711', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 66, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6800', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6801', 'nonce': 1770619177}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6802', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6803', 'nonce': 1770619177}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6806', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6807', 'nonce': 1770619177}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6804', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6805', 'nonce': 1770619177}]}, 'public_addr': '172.21.15.171:6801/1770619177', 'cluster_addr': '172.21.15.171:6803/1770619177', 'heartbeat_back_addr': '172.21.15.171:6807/1770619177', 'heartbeat_front_addr': '172.21.15.171:6805/1770619177', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:21.445879+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.240967+0000', 'dead_epoch': 48}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.039097+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.372014+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.049543+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.009384+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.048598+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.046345+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T21:30:36.215 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:30:36.216 INFO:teuthology.orchestra.run.smithi129.stdout:3890 read 0x3a27258 thru 0x3a2b587 (0x4330 bytes) 2024-09-17T21:30:36.216 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.216 INFO:teuthology.orchestra.run.smithi129.stdout:3892 write 0xd17938f thru 0xd181475 (0x80e7 bytes) 2024-09-17T21:30:36.217 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:30:36.218 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.218 INFO:teuthology.orchestra.run.smithi129.stdout:3895 write 0x7bbce5d thru 0x7bc1243 (0x43e7 bytes) 2024-09-17T21:30:36.218 INFO:teuthology.orchestra.run.smithi129.stdout:3896 write 0x5139cf3 thru 0x5140034 (0x6342 bytes) 2024-09-17T21:30:36.218 INFO:teuthology.orchestra.run.smithi129.stdout:3899 punch from 0xd6f2b4b to 0xd6fd07d, (0xa532 bytes) 2024-09-17T21:30:36.218 INFO:teuthology.orchestra.run.smithi129.stdout:3901 writesame 0x55bf78a thru 0x55c593c data_size 0x15(0x61b3 bytes) 2024-09-17T21:30:36.218 INFO:teuthology.orchestra.run.smithi129.stdout:3903 trunc from 0xdd424d7 to 0x6d46108 2024-09-17T21:30:36.218 INFO:teuthology.orchestra.run.smithi129.stdout:3904 punch from 0x3fda1b0 to 0x3fdcdf3, (0x2c43 bytes) 2024-09-17T21:30:36.218 INFO:teuthology.orchestra.run.smithi129.stdout:3905 trunc from 0x6d46108 to 0xb97d37b 2024-09-17T21:30:36.218 INFO:teuthology.orchestra.run.smithi129.stdout:3906 writesame 0xe7b493 thru 0xe861be data_size 0x2(0xad2c bytes) 2024-09-17T21:30:36.219 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:36.215+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:36.221 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.221 INFO:teuthology.orchestra.run.smithi129.stdout:3908 write 0x52e6edb thru 0x52ee67a (0x77a0 bytes) 2024-09-17T21:30:36.223 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.231 INFO:teuthology.orchestra.run.smithi129.stdout:3911 writesame 0xe49b442 thru 0xe4a287c data_size 0xb(0x743b bytes) 2024-09-17T21:30:36.231 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.231 INFO:teuthology.orchestra.run.smithi129.stdout:3917 write 0x11c7f1f thru 0x11cdef7 (0x5fd9 bytes) 2024-09-17T21:30:36.233 INFO:teuthology.orchestra.run.smithi129.stdout:3919 write 0x1504d12 thru 0x150e3c4 (0x96b3 bytes) 2024-09-17T21:30:36.237 INFO:teuthology.orchestra.run.smithi129.stdout:3920 read 0xdea9d93 thru 0xdeb9071 (0xf2df bytes) 2024-09-17T21:30:36.237 INFO:teuthology.orchestra.run.smithi129.stdout:3922 writesame 0xbde866a thru 0xbde86d9 data_size 0x4(0x70 bytes) 2024-09-17T21:30:36.238 INFO:teuthology.orchestra.run.smithi129.stdout:3923 write 0x2a8bd8f thru 0x2a92a24 (0x6c96 bytes) 2024-09-17T21:30:36.241 INFO:teuthology.orchestra.run.smithi129.stdout:3924 read 0x795edc thru 0x79b8e7 (0x5a0c bytes) 2024-09-17T21:30:36.241 INFO:teuthology.orchestra.run.smithi129.stdout:3925 write 0x6eb1b8 thru 0x6ebf39 (0xd82 bytes) 2024-09-17T21:30:36.243 INFO:teuthology.orchestra.run.smithi129.stdout:3926 read 0xd09a89b thru 0xd0a4482 (0x9be8 bytes) 2024-09-17T21:30:36.243 INFO:teuthology.orchestra.run.smithi129.stdout:3928 write 0x9fd949 thru 0xa06d5c (0x9414 bytes) 2024-09-17T21:30:36.245 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.245 INFO:teuthology.orchestra.run.smithi129.stdout:3930 punch from 0x9fcb022 to 0x9fd9e96, (0xee74 bytes) 2024-09-17T21:30:36.247 INFO:teuthology.orchestra.run.smithi129.stdout:3931 trunc from 0xe4a287d to 0xd4b1940 2024-09-17T21:30:36.252 INFO:teuthology.orchestra.run.smithi129.stdout:3932 writesame 0x4e5501b thru 0x4e60fe1 data_size 0xf(0xbfc7 bytes) 2024-09-17T21:30:36.257 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.257 INFO:teuthology.orchestra.run.smithi129.stdout:3935 read 0x52cddb7 thru 0x52d61e8 (0x8432 bytes) 2024-09-17T21:30:36.258 INFO:teuthology.orchestra.run.smithi129.stdout:3937 punch from 0x4f84e90 to 0x4f8eabf, (0x9c2f bytes) 2024-09-17T21:30:36.258 INFO:teuthology.orchestra.run.smithi129.stdout:3938 read 0x72ee24b thru 0x72fa0de (0xbe94 bytes) 2024-09-17T21:30:36.270 INFO:teuthology.orchestra.run.smithi129.stdout:3939 trunc from 0xd4b1940 to 0x40fea70 2024-09-17T21:30:36.282 INFO:teuthology.orchestra.run.smithi129.stdout:3940 write 0x5f228d thru 0x5fd6c2 (0xb436 bytes) 2024-09-17T21:30:36.284 INFO:teuthology.orchestra.run.smithi129.stdout:3941 read 0x2fdc4d5 thru 0x2fe11d8 (0x4d04 bytes) 2024-09-17T21:30:36.293 INFO:teuthology.orchestra.run.smithi129.stdout:3942 write 0x7b98f1d thru 0x7b9d942 (0x4a26 bytes) 2024-09-17T21:30:36.297 INFO:teuthology.orchestra.run.smithi129.stdout:3943 write 0x78a8727 thru 0x78ad23e (0x4b18 bytes) 2024-09-17T21:30:36.301 INFO:teuthology.orchestra.run.smithi129.stdout:3946 read 0x730c7d5 thru 0x7310f22 (0x474e bytes) 2024-09-17T21:30:36.301 INFO:teuthology.orchestra.run.smithi129.stdout:3948 read 0x6695c57 thru 0x66997df (0x3b89 bytes) 2024-09-17T21:30:36.315 INFO:teuthology.orchestra.run.smithi129.stdout:3950 writesame 0xde6e059 thru 0xde6fc73 data_size 0x1(0x1c1b bytes) 2024-09-17T21:30:36.319 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:36.316+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:36.321 INFO:teuthology.orchestra.run.smithi129.stdout:3951 writesame 0xec9c6bf thru 0xeca519a data_size 0x2(0x8adc bytes) 2024-09-17T21:30:36.322 INFO:teuthology.orchestra.run.smithi129.stdout:3953 writesame 0xb7bafe thru 0xb8893f data_size 0x11(0xce42 bytes) 2024-09-17T21:30:36.324 INFO:teuthology.orchestra.run.smithi129.stdout:3954 read 0x4a5b7bf thru 0x4a5dc3d (0x247f bytes) 2024-09-17T21:30:36.324 INFO:teuthology.orchestra.run.smithi129.stdout:3955 write 0xe39fe77 thru 0xe3ae28b (0xe415 bytes) 2024-09-17T21:30:36.332 INFO:teuthology.orchestra.run.smithi129.stdout:3956 punch from 0xb0e1389 to 0xb0e6a6e, (0x56e5 bytes) 2024-09-17T21:30:36.332 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.332 INFO:teuthology.orchestra.run.smithi129.stdout:3960 write 0xd65cb99 thru 0xd66565e (0x8ac6 bytes) 2024-09-17T21:30:36.340 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:36.340 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:36.340 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:36.340 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:36.340 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:36.341 INFO:teuthology.orchestra.run.smithi129.stdout:3961 write 0x7a0b04b thru 0x7a1a60e (0xf5c4 bytes) 2024-09-17T21:30:36.349 INFO:teuthology.orchestra.run.smithi129.stdout:3962 writesame 0xd8a899 thru 0xd9975e data_size 0xd(0xeec6 bytes) 2024-09-17T21:30:36.350 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:30:36.352 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.352 INFO:teuthology.orchestra.run.smithi129.stdout:3964 read 0x97c7afa thru 0x97d128e (0x9795 bytes) 2024-09-17T21:30:36.353 INFO:teuthology.orchestra.run.smithi129.stdout:3965 read 0x18078f9 thru 0x180a4eb (0x2bf3 bytes) 2024-09-17T21:30:36.353 INFO:teuthology.orchestra.run.smithi129.stdout:3966 writesame 0xa0b3e08 thru 0xa0b466e data_size 0x3(0x867 bytes) 2024-09-17T21:30:36.357 INFO:teuthology.orchestra.run.smithi129.stdout:3968 write 0x3a88756 thru 0x3a97ef7 (0xf7a2 bytes) 2024-09-17T21:30:36.362 INFO:teuthology.orchestra.run.smithi129.stdout:3971 read 0xb3e720 thru 0xb4354e (0x4e2f bytes) 2024-09-17T21:30:36.363 INFO:teuthology.orchestra.run.smithi129.stdout:3974 write 0x3590d2e thru 0x3597fdf (0x72b2 bytes) 2024-09-17T21:30:36.366 INFO:teuthology.orchestra.run.smithi129.stdout:3976 read 0xbbf19fa thru 0xbbf282b (0xe32 bytes) 2024-09-17T21:30:36.373 INFO:teuthology.orchestra.run.smithi129.stdout:3977 trunc from 0xeca519b to 0x8cd2fbd 2024-09-17T21:30:36.385 INFO:teuthology.orchestra.run.smithi129.stdout:3978 write 0x5a5466e thru 0x5a5a14e (0x5ae1 bytes) 2024-09-17T21:30:36.389 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:30:36.395 INFO:teuthology.orchestra.run.smithi129.stdout:3980 trunc from 0x8cd2fbd to 0xbde6b78 2024-09-17T21:30:36.400 INFO:teuthology.orchestra.run.smithi129.stdout:3981 writesame 0x498c6b thru 0x4a4ff4 data_size 0x67(0xc38a bytes) 2024-09-17T21:30:36.403 INFO:teuthology.orchestra.run.smithi129.stdout:3982 write 0x747ac81 thru 0x74847a7 (0x9b27 bytes) 2024-09-17T21:30:36.406 INFO:teuthology.orchestra.run.smithi129.stdout:3983 read 0x3068205 thru 0x307324c (0xb048 bytes) 2024-09-17T21:30:36.407 INFO:teuthology.orchestra.run.smithi129.stdout:3984 writesame 0x1d226f7 thru 0x1d24b6a data_size 0x2(0x2474 bytes) 2024-09-17T21:30:36.411 INFO:teuthology.orchestra.run.smithi129.stdout:3985 trunc from 0xbde6b78 to 0xb5147f6 2024-09-17T21:30:36.417 INFO:teuthology.orchestra.run.smithi129.stdout:3986 read 0x4971a00 thru 0x497e579 (0xcb7a bytes) 2024-09-17T21:30:36.420 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:36.418+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:36.425 INFO:teuthology.orchestra.run.smithi129.stdout:3987 write 0xc6056da thru 0xc61163d (0xbf64 bytes) 2024-09-17T21:30:36.435 INFO:teuthology.orchestra.run.smithi129.stdout:3988 punch from 0x8b9aee to 0x8baff2, (0x1504 bytes) 2024-09-17T21:30:36.435 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.435 INFO:teuthology.orchestra.run.smithi129.stdout:3991 read 0x6002855 thru 0x60047f7 (0x1fa3 bytes) 2024-09-17T21:30:36.435 INFO:teuthology.orchestra.run.smithi129.stdout:3993 read 0x6ddb89a thru 0x6de5b9a (0xa301 bytes) 2024-09-17T21:30:36.435 INFO:teuthology.orchestra.run.smithi129.stdout:3994 read 0xaf4e646 thru 0xaf5b42d (0xcde8 bytes) 2024-09-17T21:30:36.437 INFO:teuthology.orchestra.run.smithi129.stdout:3995 trunc from 0xc61163e to 0xb9b0088 2024-09-17T21:30:36.443 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.443 INFO:teuthology.orchestra.run.smithi129.stdout:3998 read 0x34dd826 thru 0x34e67ec (0x8fc7 bytes) 2024-09-17T21:30:36.445 INFO:teuthology.orchestra.run.smithi129.stdout:3999 trunc from 0xb9b0088 to 0xabbde3f 2024-09-17T21:30:36.464 INFO:teuthology.orchestra.run.smithi129.stdout:4000 write 0xc69c486 thru 0xc6a8436 (0xbfb1 bytes) 2024-09-17T21:30:36.482 INFO:teuthology.orchestra.run.smithi129.stdout:4001 trunc from 0xc6a8437 to 0x2d0ab33 2024-09-17T21:30:36.483 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:36.484 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:36.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:36.484 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:36.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.484 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:79995 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.484 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979453+0000", 2024-09-17T21:30:36.484 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.500951288, 2024-09-17T21:30:36.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016640801, 2024-09-17T21:30:36.484 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.484 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.484 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.484 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.484 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.485 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.485 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 79995 2024-09-17T21:30:36.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.485 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979453+0000", 2024-09-17T21:30:36.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979453+0000", 2024-09-17T21:30:36.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979453+0000", 2024-09-17T21:30:36.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8500000000000002e-07 2024-09-17T21:30:36.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979455+0000", 2024-09-17T21:30:36.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0810000000000001e-06 2024-09-17T21:30:36.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979461+0000", 2024-09-17T21:30:36.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:30:36.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979466+0000", 2024-09-17T21:30:36.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5379999999999996e-06 2024-09-17T21:30:36.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.995964+0000", 2024-09-17T21:30:36.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016498228 2024-09-17T21:30:36.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996027+0000", 2024-09-17T21:30:36.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2792999999999993e-05 2024-09-17T21:30:36.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996093+0000", 2024-09-17T21:30:36.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6273999999999996e-05 2024-09-17T21:30:36.488 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.488 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.488 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.488 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:79997 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.488 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979473+0000", 2024-09-17T21:30:36.489 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.500931097, 2024-09-17T21:30:36.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016783342999999999, 2024-09-17T21:30:36.489 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.489 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.489 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.489 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.489 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.489 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.489 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 79997 2024-09-17T21:30:36.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.489 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979473+0000", 2024-09-17T21:30:36.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979473+0000", 2024-09-17T21:30:36.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979473+0000", 2024-09-17T21:30:36.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:30:36.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979475+0000", 2024-09-17T21:30:36.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-17T21:30:36.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979481+0000", 2024-09-17T21:30:36.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5930000000000002e-06 2024-09-17T21:30:36.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979486+0000", 2024-09-17T21:30:36.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7659999999999998e-06 2024-09-17T21:30:36.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996119+0000", 2024-09-17T21:30:36.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016632797000000001 2024-09-17T21:30:36.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996186+0000", 2024-09-17T21:30:36.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7390000000000004e-05 2024-09-17T21:30:36.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996256+0000", 2024-09-17T21:30:36.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9646000000000004e-05 2024-09-17T21:30:36.493 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.493 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.493 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.493 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:79999 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.493 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979493+0000", 2024-09-17T21:30:36.493 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.500911001, 2024-09-17T21:30:36.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024077517999999999, 2024-09-17T21:30:36.493 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.493 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.493 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.494 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.494 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.494 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.494 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 79999 2024-09-17T21:30:36.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.494 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979493+0000", 2024-09-17T21:30:36.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979493+0000", 2024-09-17T21:30:36.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979493+0000", 2024-09-17T21:30:36.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1199999999999998e-07 2024-09-17T21:30:36.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979495+0000", 2024-09-17T21:30:36.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8059999999999999e-06 2024-09-17T21:30:36.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979501+0000", 2024-09-17T21:30:36.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2720000000000003e-06 2024-09-17T21:30:36.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979506+0000", 2024-09-17T21:30:36.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:30:36.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996323+0000", 2024-09-17T21:30:36.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016817148000000001 2024-09-17T21:30:36.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996392+0000", 2024-09-17T21:30:36.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8295999999999998e-05 2024-09-17T21:30:36.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:35.003570+0000", 2024-09-17T21:30:36.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071786840000000003 2024-09-17T21:30:36.497 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.497 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.497 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.498 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80001 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.498 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979512+0000", 2024-09-17T21:30:36.498 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.5008915890000001, 2024-09-17T21:30:36.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017128602, 2024-09-17T21:30:36.498 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.498 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.498 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.498 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.498 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.498 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.498 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80001 2024-09-17T21:30:36.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.499 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979512+0000", 2024-09-17T21:30:36.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979512+0000", 2024-09-17T21:30:36.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979513+0000", 2024-09-17T21:30:36.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9900000000000001e-07 2024-09-17T21:30:36.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979514+0000", 2024-09-17T21:30:36.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7489999999999999e-06 2024-09-17T21:30:36.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979521+0000", 2024-09-17T21:30:36.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2190000000000001e-06 2024-09-17T21:30:36.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979526+0000", 2024-09-17T21:30:36.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9930000000000003e-06 2024-09-17T21:30:36.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996512+0000", 2024-09-17T21:30:36.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016986082999999999 2024-09-17T21:30:36.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996577+0000", 2024-09-17T21:30:36.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5258000000000004e-05 2024-09-17T21:30:36.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996641+0000", 2024-09-17T21:30:36.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3900999999999997e-05 2024-09-17T21:30:36.502 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.502 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.502 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.502 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80003 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.502 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979532+0000", 2024-09-17T21:30:36.502 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.5008713280000001, 2024-09-17T21:30:36.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017252887000000001, 2024-09-17T21:30:36.503 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.503 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.503 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.503 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.503 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.503 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.503 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80003 2024-09-17T21:30:36.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.503 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979532+0000", 2024-09-17T21:30:36.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979532+0000", 2024-09-17T21:30:36.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979533+0000", 2024-09-17T21:30:36.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.34e-07 2024-09-17T21:30:36.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979535+0000", 2024-09-17T21:30:36.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.906e-06 2024-09-17T21:30:36.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979541+0000", 2024-09-17T21:30:36.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.489e-06 2024-09-17T21:30:36.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979547+0000", 2024-09-17T21:30:36.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7010000000000001e-06 2024-09-17T21:30:36.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996665+0000", 2024-09-17T21:30:36.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017118187999999999 2024-09-17T21:30:36.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996727+0000", 2024-09-17T21:30:36.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1897999999999996e-05 2024-09-17T21:30:36.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996785+0000", 2024-09-17T21:30:36.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8270999999999998e-05 2024-09-17T21:30:36.506 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.506 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.506 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.507 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80005 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.507 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979554+0000", 2024-09-17T21:30:36.507 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.5008498029999999, 2024-09-17T21:30:36.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017377259999999999, 2024-09-17T21:30:36.507 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.507 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.507 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.507 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.507 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.507 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.508 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80005 2024-09-17T21:30:36.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.508 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979554+0000", 2024-09-17T21:30:36.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979554+0000", 2024-09-17T21:30:36.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979554+0000", 2024-09-17T21:30:36.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.58e-07 2024-09-17T21:30:36.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979556+0000", 2024-09-17T21:30:36.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8759999999999999e-06 2024-09-17T21:30:36.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979563+0000", 2024-09-17T21:30:36.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4069999999999998e-06 2024-09-17T21:30:36.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979567+0000", 2024-09-17T21:30:36.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5680000000000001e-06 2024-09-17T21:30:36.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996810+0000", 2024-09-17T21:30:36.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017242683000000002 2024-09-17T21:30:36.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996872+0000", 2024-09-17T21:30:36.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1845000000000002e-05 2024-09-17T21:30:36.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996931+0000", 2024-09-17T21:30:36.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9422999999999997e-05 2024-09-17T21:30:36.511 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.511 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.511 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.512 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80007 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3883008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.512 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979574+0000", 2024-09-17T21:30:36.512 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.500829513, 2024-09-17T21:30:36.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017501042000000001, 2024-09-17T21:30:36.512 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.512 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.512 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.512 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.512 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.512 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.512 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80007 2024-09-17T21:30:36.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.513 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979574+0000", 2024-09-17T21:30:36.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979574+0000", 2024-09-17T21:30:36.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979575+0000", 2024-09-17T21:30:36.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:30:36.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979577+0000", 2024-09-17T21:30:36.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0420000000000001e-06 2024-09-17T21:30:36.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979583+0000", 2024-09-17T21:30:36.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3110000000000002e-06 2024-09-17T21:30:36.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979588+0000", 2024-09-17T21:30:36.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0420000000000002e-06 2024-09-17T21:30:36.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996954+0000", 2024-09-17T21:30:36.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017365862999999999 2024-09-17T21:30:36.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997019+0000", 2024-09-17T21:30:36.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4838999999999994e-05 2024-09-17T21:30:36.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997075+0000", 2024-09-17T21:30:36.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6427e-05 2024-09-17T21:30:36.516 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.516 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.516 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.516 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80009 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.516 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979595+0000", 2024-09-17T21:30:36.516 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.500808613, 2024-09-17T21:30:36.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017636168000000001, 2024-09-17T21:30:36.516 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.517 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.517 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.517 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.517 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.517 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.517 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80009 2024-09-17T21:30:36.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.517 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979595+0000", 2024-09-17T21:30:36.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979595+0000", 2024-09-17T21:30:36.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979596+0000", 2024-09-17T21:30:36.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3300000000000002e-07 2024-09-17T21:30:36.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979598+0000", 2024-09-17T21:30:36.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.95e-06 2024-09-17T21:30:36.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979604+0000", 2024-09-17T21:30:36.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4579999999999998e-06 2024-09-17T21:30:36.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979609+0000", 2024-09-17T21:30:36.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7770000000000002e-06 2024-09-17T21:30:36.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997098+0000", 2024-09-17T21:30:36.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017489395000000001 2024-09-17T21:30:36.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997171+0000", 2024-09-17T21:30:36.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2410000000000006e-05 2024-09-17T21:30:36.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997231+0000", 2024-09-17T21:30:36.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0645e-05 2024-09-17T21:30:36.520 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.520 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.520 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.521 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80011 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.521 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979615+0000", 2024-09-17T21:30:36.521 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.500788292, 2024-09-17T21:30:36.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017760018999999998, 2024-09-17T21:30:36.521 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.521 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.521 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.521 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.521 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.521 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.521 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80011 2024-09-17T21:30:36.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.521 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979615+0000", 2024-09-17T21:30:36.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979615+0000", 2024-09-17T21:30:36.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979616+0000", 2024-09-17T21:30:36.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1699999999999998e-07 2024-09-17T21:30:36.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979618+0000", 2024-09-17T21:30:36.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9750000000000001e-06 2024-09-17T21:30:36.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979624+0000", 2024-09-17T21:30:36.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8699999999999997e-06 2024-09-17T21:30:36.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979629+0000", 2024-09-17T21:30:36.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.78e-06 2024-09-17T21:30:36.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997254+0000", 2024-09-17T21:30:36.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017625339 2024-09-17T21:30:36.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997315+0000", 2024-09-17T21:30:36.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0914e-05 2024-09-17T21:30:36.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997376+0000", 2024-09-17T21:30:36.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0624000000000001e-05 2024-09-17T21:30:36.525 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.525 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.525 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.525 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80224 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.525 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979990+0000", 2024-09-17T21:30:36.525 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.500413819, 2024-09-17T21:30:36.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017530202000000002, 2024-09-17T21:30:36.525 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.526 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.526 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.526 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.526 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.526 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.526 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80224 2024-09-17T21:30:36.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.526 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979990+0000", 2024-09-17T21:30:36.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979990+0000", 2024-09-17T21:30:36.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979990+0000", 2024-09-17T21:30:36.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:30:36.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979992+0000", 2024-09-17T21:30:36.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-17T21:30:36.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979999+0000", 2024-09-17T21:30:36.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:30:36.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980004+0000", 2024-09-17T21:30:36.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:30:36.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997399+0000", 2024-09-17T21:30:36.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017395342000000001 2024-09-17T21:30:36.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997462+0000", 2024-09-17T21:30:36.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2828000000000002e-05 2024-09-17T21:30:36.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997520+0000", 2024-09-17T21:30:36.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8221999999999999e-05 2024-09-17T21:30:36.529 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.530 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.530 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.530 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80225 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.530 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980011+0000", 2024-09-17T21:30:36.530 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.50039287, 2024-09-17T21:30:36.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017655349000000001, 2024-09-17T21:30:36.530 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.530 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.530 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.531 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80225 2024-09-17T21:30:36.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.531 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980011+0000", 2024-09-17T21:30:36.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980011+0000", 2024-09-17T21:30:36.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980011+0000", 2024-09-17T21:30:36.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2800000000000002e-07 2024-09-17T21:30:36.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980013+0000", 2024-09-17T21:30:36.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9420000000000002e-06 2024-09-17T21:30:36.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980019+0000", 2024-09-17T21:30:36.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.976e-06 2024-09-17T21:30:36.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980024+0000", 2024-09-17T21:30:36.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7600000000000002e-06 2024-09-17T21:30:36.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997544+0000", 2024-09-17T21:30:36.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017520134999999999 2024-09-17T21:30:36.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997607+0000", 2024-09-17T21:30:36.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2928999999999993e-05 2024-09-17T21:30:36.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997666+0000", 2024-09-17T21:30:36.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9178999999999997e-05 2024-09-17T21:30:36.534 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.534 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.534 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.534 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80226 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3227648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.534 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980031+0000", 2024-09-17T21:30:36.534 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.500372421, 2024-09-17T21:30:36.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017780114, 2024-09-17T21:30:36.535 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.535 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.535 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.535 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.535 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.535 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.535 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80226 2024-09-17T21:30:36.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.535 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980031+0000", 2024-09-17T21:30:36.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980031+0000", 2024-09-17T21:30:36.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980032+0000", 2024-09-17T21:30:36.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.01e-07 2024-09-17T21:30:36.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980034+0000", 2024-09-17T21:30:36.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8309999999999999e-06 2024-09-17T21:30:36.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980040+0000", 2024-09-17T21:30:36.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:30:36.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980045+0000", 2024-09-17T21:30:36.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7559999999999999e-06 2024-09-17T21:30:36.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997689+0000", 2024-09-17T21:30:36.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017644395 2024-09-17T21:30:36.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997751+0000", 2024-09-17T21:30:36.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2238999999999998e-05 2024-09-17T21:30:36.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997811+0000", 2024-09-17T21:30:36.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0078000000000003e-05 2024-09-17T21:30:36.538 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.538 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.539 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.539 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80227 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.539 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980051+0000", 2024-09-17T21:30:36.539 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.500352331, 2024-09-17T21:30:36.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017904265999999999, 2024-09-17T21:30:36.539 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.539 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.539 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.539 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.539 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.539 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.539 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80227 2024-09-17T21:30:36.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.540 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980051+0000", 2024-09-17T21:30:36.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980051+0000", 2024-09-17T21:30:36.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980052+0000", 2024-09-17T21:30:36.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1199999999999998e-07 2024-09-17T21:30:36.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980054+0000", 2024-09-17T21:30:36.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7659999999999999e-06 2024-09-17T21:30:36.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980060+0000", 2024-09-17T21:30:36.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5379999999999999e-06 2024-09-17T21:30:36.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980065+0000", 2024-09-17T21:30:36.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6e-06 2024-09-17T21:30:36.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997835+0000", 2024-09-17T21:30:36.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017770013000000001 2024-09-17T21:30:36.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997896+0000", 2024-09-17T21:30:36.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0803e-05 2024-09-17T21:30:36.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997956+0000", 2024-09-17T21:30:36.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0133999999999997e-05 2024-09-17T21:30:36.543 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.543 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.543 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.543 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80228 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3457024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.544 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980071+0000", 2024-09-17T21:30:36.544 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.5003323150000001, 2024-09-17T21:30:36.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018027676999999999, 2024-09-17T21:30:36.544 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.544 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.544 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.544 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.544 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.544 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.544 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80228 2024-09-17T21:30:36.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.544 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980071+0000", 2024-09-17T21:30:36.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980071+0000", 2024-09-17T21:30:36.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980072+0000", 2024-09-17T21:30:36.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:30:36.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980074+0000", 2024-09-17T21:30:36.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.728e-06 2024-09-17T21:30:36.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980080+0000", 2024-09-17T21:30:36.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3339999999999998e-06 2024-09-17T21:30:36.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980085+0000", 2024-09-17T21:30:36.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6700000000000002e-06 2024-09-17T21:30:36.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997978+0000", 2024-09-17T21:30:36.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017893678 2024-09-17T21:30:36.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998042+0000", 2024-09-17T21:30:36.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3193000000000003e-05 2024-09-17T21:30:36.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998099+0000", 2024-09-17T21:30:36.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7651000000000001e-05 2024-09-17T21:30:36.548 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.548 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.548 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.548 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80229 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.548 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980091+0000", 2024-09-17T21:30:36.548 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.5003126500000001, 2024-09-17T21:30:36.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018156578999999999, 2024-09-17T21:30:36.548 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.548 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.548 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.548 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.548 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.549 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.549 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80229 2024-09-17T21:30:36.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.549 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980091+0000", 2024-09-17T21:30:36.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980091+0000", 2024-09-17T21:30:36.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980092+0000", 2024-09-17T21:30:36.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:30:36.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980094+0000", 2024-09-17T21:30:36.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8339999999999999e-06 2024-09-17T21:30:36.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980100+0000", 2024-09-17T21:30:36.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4470000000000003e-06 2024-09-17T21:30:36.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980105+0000", 2024-09-17T21:30:36.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:30:36.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998122+0000", 2024-09-17T21:30:36.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018017610999999999 2024-09-17T21:30:36.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998189+0000", 2024-09-17T21:30:36.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6280999999999998e-05 2024-09-17T21:30:36.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998248+0000", 2024-09-17T21:30:36.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9040999999999999e-05 2024-09-17T21:30:36.552 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.552 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.552 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.552 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80230 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3686400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.553 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980111+0000", 2024-09-17T21:30:36.553 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.500292599, 2024-09-17T21:30:36.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018282363999999999, 2024-09-17T21:30:36.553 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.553 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.553 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.553 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.553 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.553 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.553 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80230 2024-09-17T21:30:36.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.553 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980111+0000", 2024-09-17T21:30:36.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980111+0000", 2024-09-17T21:30:36.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980112+0000", 2024-09-17T21:30:36.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5400000000000002e-07 2024-09-17T21:30:36.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980113+0000", 2024-09-17T21:30:36.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8199999999999999e-06 2024-09-17T21:30:36.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980120+0000", 2024-09-17T21:30:36.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3709999999999996e-06 2024-09-17T21:30:36.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980128+0000", 2024-09-17T21:30:36.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9079999999999995e-06 2024-09-17T21:30:36.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998272+0000", 2024-09-17T21:30:36.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018143920000000001 2024-09-17T21:30:36.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998334+0000", 2024-09-17T21:30:36.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1921000000000006e-05 2024-09-17T21:30:36.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998394+0000", 2024-09-17T21:30:36.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9969999999999997e-05 2024-09-17T21:30:36.557 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.557 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.557 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.557 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80231 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3801088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.557 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980135+0000", 2024-09-17T21:30:36.557 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.500269257, 2024-09-17T21:30:36.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018400430999999998, 2024-09-17T21:30:36.557 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.558 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.558 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.558 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.558 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.558 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.558 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80231 2024-09-17T21:30:36.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.558 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980135+0000", 2024-09-17T21:30:36.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980135+0000", 2024-09-17T21:30:36.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980135+0000", 2024-09-17T21:30:36.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3499999999999996e-07 2024-09-17T21:30:36.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980138+0000", 2024-09-17T21:30:36.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7970000000000001e-06 2024-09-17T21:30:36.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980144+0000", 2024-09-17T21:30:36.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.392e-06 2024-09-17T21:30:36.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980153+0000", 2024-09-17T21:30:36.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7010000000000002e-06 2024-09-17T21:30:36.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998417+0000", 2024-09-17T21:30:36.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018263643 2024-09-17T21:30:36.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998476+0000", 2024-09-17T21:30:36.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9417999999999999e-05 2024-09-17T21:30:36.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998535+0000", 2024-09-17T21:30:36.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8944999999999999e-05 2024-09-17T21:30:36.562 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.562 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.562 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.562 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80232 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.562 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980160+0000", 2024-09-17T21:30:36.562 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.5002438229999999, 2024-09-17T21:30:36.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018519280999999999, 2024-09-17T21:30:36.562 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.562 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.562 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.562 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.563 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.563 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.563 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80232 2024-09-17T21:30:36.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.563 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980160+0000", 2024-09-17T21:30:36.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980160+0000", 2024-09-17T21:30:36.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980160+0000", 2024-09-17T21:30:36.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8699999999999995e-07 2024-09-17T21:30:36.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980176+0000", 2024-09-17T21:30:36.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5529e-05 2024-09-17T21:30:36.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980183+0000", 2024-09-17T21:30:36.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0829999999999998e-06 2024-09-17T21:30:36.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980188+0000", 2024-09-17T21:30:36.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1000000000000003e-06 2024-09-17T21:30:36.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998558+0000", 2024-09-17T21:30:36.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018369897999999999 2024-09-17T21:30:36.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998620+0000", 2024-09-17T21:30:36.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1550999999999994e-05 2024-09-17T21:30:36.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998679+0000", 2024-09-17T21:30:36.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9633000000000003e-05 2024-09-17T21:30:36.566 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.566 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.566 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.566 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80234 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.566 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980197+0000", 2024-09-17T21:30:36.567 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.5002063109999999, 2024-09-17T21:30:36.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018624308999999999, 2024-09-17T21:30:36.567 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.567 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.567 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.567 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.567 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.567 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.567 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80234 2024-09-17T21:30:36.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.567 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980197+0000", 2024-09-17T21:30:36.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980197+0000", 2024-09-17T21:30:36.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980198+0000", 2024-09-17T21:30:36.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.34e-07 2024-09-17T21:30:36.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980200+0000", 2024-09-17T21:30:36.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9589999999999998e-06 2024-09-17T21:30:36.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980206+0000", 2024-09-17T21:30:36.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5250000000000002e-06 2024-09-17T21:30:36.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980211+0000", 2024-09-17T21:30:36.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:30:36.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998702+0000", 2024-09-17T21:30:36.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018490273000000002 2024-09-17T21:30:36.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998764+0000", 2024-09-17T21:30:36.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2305999999999996e-05 2024-09-17T21:30:36.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998822+0000", 2024-09-17T21:30:36.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7936000000000002e-05 2024-09-17T21:30:36.570 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.570 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.570 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.571 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80236 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 4145152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:36.571 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980218+0000", 2024-09-17T21:30:36.571 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.5001856359999999, 2024-09-17T21:30:36.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018747997999999998, 2024-09-17T21:30:36.571 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:36.571 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:36.571 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:36.571 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:36.571 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:36.571 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:36.571 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80236 2024-09-17T21:30:36.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.572 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:36.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:36.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980218+0000", 2024-09-17T21:30:36.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:36.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980218+0000", 2024-09-17T21:30:36.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:36.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:36.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980219+0000", 2024-09-17T21:30:36.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.39e-07 2024-09-17T21:30:36.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:36.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980221+0000", 2024-09-17T21:30:36.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:30:36.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:36.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980227+0000", 2024-09-17T21:30:36.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8240000000000002e-06 2024-09-17T21:30:36.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:36.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980232+0000", 2024-09-17T21:30:36.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999998e-06 2024-09-17T21:30:36.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:36.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998844+0000", 2024-09-17T21:30:36.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018612257 2024-09-17T21:30:36.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:36.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998907+0000", 2024-09-17T21:30:36.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2860999999999993e-05 2024-09-17T21:30:36.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:36.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:36.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:36.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998966+0000", 2024-09-17T21:30:36.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8897999999999998e-05 2024-09-17T21:30:36.575 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.575 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.575 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.575 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:36.575 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:36.575 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:36.577 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:30:36.578 INFO:teuthology.orchestra.run.smithi129.stdout:4002 write 0x7d4153c thru 0x7d4f5d5 (0xe09a bytes) 2024-09-17T21:30:36.578 INFO:teuthology.orchestra.run.smithi129.stdout:4003 read 0xb19c7f thru 0xb20866 (0x6be8 bytes) 2024-09-17T21:30:36.578 INFO:teuthology.orchestra.run.smithi129.stdout:4005 trunc from 0x7d4f5d6 to 0x505fa13 2024-09-17T21:30:36.578 INFO:teuthology.orchestra.run.smithi129.stdout:4006 write 0x3aa0ea1 thru 0x3aa7f84 (0x70e4 bytes) 2024-09-17T21:30:36.578 INFO:teuthology.orchestra.run.smithi129.stdout:4007 read 0x2c7d723 thru 0x2c80598 (0x2e76 bytes) 2024-09-17T21:30:36.578 INFO:teuthology.orchestra.run.smithi129.stdout:4009 punch from 0x1d75b57 to 0x1d7b162, (0x560b bytes) 2024-09-17T21:30:36.578 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.578 INFO:teuthology.orchestra.run.smithi129.stdout:4012 writesame 0x69b5adf thru 0x69c0aa8 data_size 0x1(0xafca bytes) 2024-09-17T21:30:36.578 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.578 INFO:teuthology.orchestra.run.smithi129.stdout:4014 read 0x61cd263 thru 0x61d7105 (0x9ea3 bytes) 2024-09-17T21:30:36.578 INFO:teuthology.orchestra.run.smithi129.stdout:4015 trunc from 0x69c0aa9 to 0x291b23a 2024-09-17T21:30:36.579 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:36.579 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:36.580 INFO:teuthology.orchestra.run.smithi129.stdout:4016 read 0x704ad1 thru 0x7114ec (0xca1c bytes) 2024-09-17T21:30:36.580 INFO:teuthology.orchestra.run.smithi129.stdout:4018 punch from 0xab5786 to 0xabdad0, (0x834a bytes) 2024-09-17T21:30:36.580 INFO:teuthology.orchestra.run.smithi129.stdout:4019 write 0x460a37b thru 0x460dcd5 (0x395b bytes) 2024-09-17T21:30:36.580 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:30:36.580 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:30:36.580 INFO:teuthology.orchestra.run.smithi129.stdout:4020 trunc from 0x460dcd6 to 0x3acd61c 2024-09-17T21:30:36.581 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:36.574+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:36.588 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:36.588 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:36.590 INFO:teuthology.orchestra.run.smithi129.stdout:4021 write 0x81e39f6 thru 0x81ee6fa (0xad05 bytes) 2024-09-17T21:30:36.597 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:30:36.597 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-17T21:30:36.605 INFO:teuthology.orchestra.run.smithi129.stdout:4022 write 0x89e7eaf thru 0x89e83a5 (0x4f7 bytes) 2024-09-17T21:30:36.606 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:36.606 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:36.609 INFO:teuthology.orchestra.run.smithi129.stdout:4023 read 0xfb5276 thru 0xfbe420 (0x91ab bytes) 2024-09-17T21:30:36.609 INFO:teuthology.orchestra.run.smithi129.stdout:4024 punch from 0x2777d4f to 0x2786ea5, (0xf156 bytes) 2024-09-17T21:30:36.610 INFO:teuthology.orchestra.run.smithi129.stdout:4025 read 0x1ba9adc thru 0x1bb7846 (0xdd6b bytes) 2024-09-17T21:30:36.610 INFO:teuthology.orchestra.run.smithi129.stdout:4027 read 0x1605cf7 thru 0x160a926 (0x4c30 bytes) 2024-09-17T21:30:36.611 INFO:teuthology.orchestra.run.smithi129.stdout:4028 write 0x32f9868 thru 0x3303746 (0x9edf bytes) 2024-09-17T21:30:36.612 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2024-09-17T21:30:34.870753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1009243143,"num_objects":1513,"num_object_clones":315,"num_object_copies":3026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1513,"num_whiteouts":784,"num_read":92423,"num_read_kb":435278,"num_write":11533,"num_write_kb":284665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":43917444,"num_keys_recovered":126,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8858,"ondisk_log_size":8858,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":303468,"kb_used_data":50260,"kb_used_omap":29,"kb_used_meta":253026,"kb_avail":660299412,"statfs":{"total":676457349120,"available":676146597888,"internally_reserved":0,"allocated":51466240,"data_stored":422202393,"data_compressed":8200699,"data_compressed_allocated":32301056,"data_compressed_original":405780050,"omap_allocated":30485,"internal_metadata":259098859},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42632640,"num_objects":35,"num_object_clones":3,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":9,"num_read":4960,"num_read_kb":44355,"num_write":778,"num_write_kb":2723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":40053968,"num_keys_recovered":75,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.123099"},"pg_stats":[{"pgid":"3.1f","version":"68'401","reported_seq":3041,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-17T21:30:33.822823+0000","last_change":"2024-09-17T21:29:59.559639+0000","last_active":"2024-09-17T21:30:33.822823+0000","last_peered":"2024-09-17T21:30:33.822823+0000","last_clean":"2024-09-17T21:30:33.822823+0000","last_became_active":"2024-09-17T21:29:59.541221+0000","last_became_peered":"2024-09-17T21:29:59.541221+0000","last_unstale":"2024-09-17T21:30:33.822823+0000","last_undegraded":"2024-09-17T21:30:33.822823+0000","last_fullsized":"2024-09-17T21:30:33.822823+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:03:23.374554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011306689999999999,"stat_sum":{"num_bytes":53582121,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":28,"num_read":3906,"num_read_kb":55416,"num_write":763,"num_write_kb":31245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":67819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"61'280","reported_seq":1463,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.751839+0000","last_change":"2024-09-17T21:29:43.473228+0000","last_active":"2024-09-17T21:30:30.751839+0000","last_peered":"2024-09-17T21:30:30.751839+0000","last_clean":"2024-09-17T21:30:30.751839+0000","last_became_active":"2024-09-17T21:29:41.472161+0000","last_became_peered":"2024-09-17T21:29:41.472161+0000","last_unstale":"2024-09-17T21:30:30.751839+0000","last_undegraded":"2024-09-17T21:30:30.751839+0000","last_fullsized":"2024-09-17T21:30:30.751839+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:25.032453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092644099999999996,"stat_sum":{"num_bytes":47941518,"num_objects":43,"num_object_clones":6,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":21,"num_read":1499,"num_read_kb":4484,"num_write":280,"num_write_kb":5873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"62'356","reported_seq":2875,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.750062+0000","last_change":"2024-09-17T21:30:02.564806+0000","last_active":"2024-09-17T21:30:30.750062+0000","last_peered":"2024-09-17T21:30:30.750062+0000","last_clean":"2024-09-17T21:30:30.750062+0000","last_became_active":"2024-09-17T21:30:02.564199+0000","last_became_peered":"2024-09-17T21:30:02.564199+0000","last_unstale":"2024-09-17T21:30:30.750062+0000","last_undegraded":"2024-09-17T21:30:30.750062+0000","last_fullsized":"2024-09-17T21:30:30.750062+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:27:08.561156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063012799999999998,"stat_sum":{"num_bytes":63627535,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":19,"num_read":3566,"num_read_kb":21832,"num_write":458,"num_write_kb":10274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":573415,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"66'226","reported_seq":1330,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.775815+0000","last_change":"2024-09-17T21:30:28.802285+0000","last_active":"2024-09-17T21:30:30.775815+0000","last_peered":"2024-09-17T21:30:30.775815+0000","last_clean":"2024-09-17T21:30:30.775815+0000","last_became_active":"2024-09-17T21:30:28.747704+0000","last_became_peered":"2024-09-17T21:30:28.747704+0000","last_unstale":"2024-09-17T21:30:30.775815+0000","last_undegraded":"2024-09-17T21:30:30.775815+0000","last_fullsized":"2024-09-17T21:30:30.775815+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:08:03.646805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047754999999999998,"stat_sum":{"num_bytes":23504709,"num_objects":49,"num_object_clones":7,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":31,"num_read":1665,"num_read_kb":6703,"num_write":220,"num_write_kb":1517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":259277,"num_keys_recovered":68,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"68'265","reported_seq":6036,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-17T21:30:33.855345+0000","last_change":"2024-09-17T21:29:43.486642+0000","last_active":"2024-09-17T21:30:33.855345+0000","last_peered":"2024-09-17T21:30:33.855345+0000","last_clean":"2024-09-17T21:30:33.855345+0000","last_became_active":"2024-09-17T21:29:41.476041+0000","last_became_peered":"2024-09-17T21:29:41.476041+0000","last_unstale":"2024-09-17T21:30:33.855345+0000","last_undegraded":"2024-09-17T21:30:33.855345+0000","last_fullsized":"2024-09-17T21:30:33.855345+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:00:12.232839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086460200000000001,"stat_sum":{"num_bytes":47756458,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":19,"num_read":6180,"num_read_kb":29346,"num_write":317,"num_write_kb":11794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"62'304","reported_seq":3614,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.799619+0000","last_change":"2024-09-17T21:30:28.776386+0000","last_active":"2024-09-17T21:30:30.799619+0000","last_peered":"2024-09-17T21:30:30.799619+0000","last_clean":"2024-09-17T21:30:30.799619+0000","last_became_active":"2024-09-17T21:30:28.747031+0000","last_became_peered":"2024-09-17T21:30:28.747031+0000","last_unstale":"2024-09-17T21:30:30.799619+0000","last_undegraded":"2024-09-17T21:30:30.799619+0000","last_fullsized":"2024-09-17T21:30:30.799619+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:48:37.791410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075875499999999995,"stat_sum":{"num_bytes":23460536,"num_objects":49,"num_object_clones":7,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":24,"num_read":3719,"num_read_kb":11922,"num_write":407,"num_write_kb":4707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":235969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"62'223","reported_seq":2900,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.751087+0000","last_change":"2024-09-17T21:30:28.887353+0000","last_active":"2024-09-17T21:30:30.751087+0000","last_peered":"2024-09-17T21:30:30.751087+0000","last_clean":"2024-09-17T21:30:30.751087+0000","last_became_active":"2024-09-17T21:30:28.745634+0000","last_became_peered":"2024-09-17T21:30:28.745634+0000","last_unstale":"2024-09-17T21:30:30.751087+0000","last_undegraded":"2024-09-17T21:30:30.751087+0000","last_fullsized":"2024-09-17T21:30:30.751087+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:32:00.730773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048345199999999999,"stat_sum":{"num_bytes":74184088,"num_objects":63,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":30,"num_read":3230,"num_read_kb":46027,"num_write":255,"num_write_kb":37247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":36017677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"62'281","reported_seq":2297,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.751607+0000","last_change":"2024-09-17T21:29:43.472344+0000","last_active":"2024-09-17T21:30:30.751607+0000","last_peered":"2024-09-17T21:30:30.751607+0000","last_clean":"2024-09-17T21:30:30.751607+0000","last_became_active":"2024-09-17T21:29:41.472468+0000","last_became_peered":"2024-09-17T21:29:41.472468+0000","last_unstale":"2024-09-17T21:30:30.751607+0000","last_undegraded":"2024-09-17T21:30:30.751607+0000","last_fullsized":"2024-09-17T21:30:30.751607+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:30:02.356874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061906600000000004,"stat_sum":{"num_bytes":30641646,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":16,"num_read":2955,"num_read_kb":9947,"num_write":330,"num_write_kb":9153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"68'303","reported_seq":1619,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-17T21:30:33.853235+0000","last_change":"2024-09-17T21:30:02.561153+0000","last_active":"2024-09-17T21:30:33.853235+0000","last_peered":"2024-09-17T21:30:33.853235+0000","last_clean":"2024-09-17T21:30:33.853235+0000","last_became_active":"2024-09-17T21:30:02.560707+0000","last_became_peered":"2024-09-17T21:30:02.560707+0000","last_unstale":"2024-09-17T21:30:33.853235+0000","last_undegraded":"2024-09-17T21:30:33.853235+0000","last_fullsized":"2024-09-17T21:30:33.853235+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:49:54.984867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173896,"stat_sum":{"num_bytes":43647624,"num_objects":51,"num_object_clones":10,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":23,"num_read":2753,"num_read_kb":11822,"num_write":307,"num_write_kb":24599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":602801,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"65'657","reported_seq":1321,"reported_epoch":65,"state":"peering","last_fresh":"2024-09-17T21:30:30.295447+0000","last_change":"2024-09-17T21:30:30.752138+0000","last_active":"2024-09-17T21:30:30.752138+0000","last_peered":"2024-09-17T21:30:30.295447+0000","last_clean":"2024-09-17T21:30:08.613285+0000","last_became_active":"2024-09-17T21:30:30.136233+0000","last_became_peered":"2024-09-17T21:30:30.136233+0000","last_unstale":"2024-09-17T21:30:30.295447+0000","last_undegraded":"2024-09-17T21:30:30.295447+0000","last_fullsized":"2024-09-17T21:30:29.750745+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072238100000000002,"stat_sum":{"num_bytes":30758137,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":24,"num_read":3360,"num_read_kb":6185,"num_write":1069,"num_write_kb":4515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":1297843,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"58'192","reported_seq":1862,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.751670+0000","last_change":"2024-09-17T21:29:43.564521+0000","last_active":"2024-09-17T21:30:30.751670+0000","last_peered":"2024-09-17T21:30:30.751670+0000","last_clean":"2024-09-17T21:30:30.751670+0000","last_became_active":"2024-09-17T21:29:41.478346+0000","last_became_peered":"2024-09-17T21:29:41.478346+0000","last_unstale":"2024-09-17T21:30:30.751670+0000","last_undegraded":"2024-09-17T21:30:30.751670+0000","last_fullsized":"2024-09-17T21:30:30.751670+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:38:38.343972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069290300000000003,"stat_sum":{"num_bytes":29869654,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":34,"num_read":2271,"num_read_kb":3236,"num_write":188,"num_write_kb":1929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"61'188","reported_seq":2127,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.750067+0000","last_change":"2024-09-17T21:29:43.477145+0000","last_active":"2024-09-17T21:30:30.750067+0000","last_peered":"2024-09-17T21:30:30.750067+0000","last_clean":"2024-09-17T21:30:30.750067+0000","last_became_active":"2024-09-17T21:29:41.477611+0000","last_became_peered":"2024-09-17T21:29:41.477611+0000","last_unstale":"2024-09-17T21:30:30.750067+0000","last_undegraded":"2024-09-17T21:30:30.750067+0000","last_fullsized":"2024-09-17T21:30:30.750067+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:59:03.278414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073991200000000001,"stat_sum":{"num_bytes":27780340,"num_objects":61,"num_object_clones":19,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":33,"num_read":1898,"num_read_kb":9197,"num_write":274,"num_write_kb":11008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.752446+0000","last_change":"2024-09-17T21:29:35.002656+0000","last_active":"2024-09-17T21:30:30.752446+0000","last_peered":"2024-09-17T21:30:30.752446+0000","last_clean":"2024-09-17T21:30:30.752446+0000","last_became_active":"2024-09-17T21:29:32.971677+0000","last_became_peered":"2024-09-17T21:29:32.971677+0000","last_unstale":"2024-09-17T21:30:30.752446+0000","last_undegraded":"2024-09-17T21:30:30.752446+0000","last_fullsized":"2024-09-17T21:30:30.752446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:53.781657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041121300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"68'530","reported_seq":2888,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-17T21:30:33.588127+0000","last_change":"2024-09-17T21:30:31.755784+0000","last_active":"2024-09-17T21:30:33.588127+0000","last_peered":"2024-09-17T21:30:33.588127+0000","last_clean":"2024-09-17T21:30:33.588127+0000","last_became_active":"2024-09-17T21:30:31.755039+0000","last_became_peered":"2024-09-17T21:30:31.755039+0000","last_unstale":"2024-09-17T21:30:33.588127+0000","last_undegraded":"2024-09-17T21:30:33.588127+0000","last_fullsized":"2024-09-17T21:30:33.588127+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:36:50.249299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010838759999999999,"stat_sum":{"num_bytes":22149763,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":20,"num_read":5379,"num_read_kb":10692,"num_write":751,"num_write_kb":5265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":448871,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.751178+0000","last_change":"2024-09-17T21:30:28.745759+0000","last_active":"2024-09-17T21:30:30.751178+0000","last_peered":"2024-09-17T21:30:30.751178+0000","last_clean":"2024-09-17T21:30:30.751178+0000","last_became_active":"2024-09-17T21:30:28.744742+0000","last_became_peered":"2024-09-17T21:30:28.744742+0000","last_unstale":"2024-09-17T21:30:30.751178+0000","last_undegraded":"2024-09-17T21:30:30.751178+0000","last_fullsized":"2024-09-17T21:30:30.751178+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:37:02.014566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"66'292","reported_seq":932,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.772304+0000","last_change":"2024-09-17T21:29:43.473294+0000","last_active":"2024-09-17T21:30:30.772304+0000","last_peered":"2024-09-17T21:30:30.772304+0000","last_clean":"2024-09-17T21:30:30.772304+0000","last_became_active":"2024-09-17T21:29:41.472095+0000","last_became_peered":"2024-09-17T21:29:41.472095+0000","last_unstale":"2024-09-17T21:30:30.772304+0000","last_undegraded":"2024-09-17T21:30:30.772304+0000","last_fullsized":"2024-09-17T21:30:30.772304+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:26:13.969452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074063099999999995,"stat_sum":{"num_bytes":4596263,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":23,"num_read":1538,"num_read_kb":3867,"num_write":468,"num_write_kb":1963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":106,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.751857+0000","last_change":"2024-09-17T21:29:35.078854+0000","last_active":"2024-09-17T21:30:30.751857+0000","last_peered":"2024-09-17T21:30:30.751857+0000","last_clean":"2024-09-17T21:30:30.751857+0000","last_became_active":"2024-09-17T21:29:33.071665+0000","last_became_peered":"2024-09-17T21:29:33.071665+0000","last_unstale":"2024-09-17T21:30:30.751857+0000","last_undegraded":"2024-09-17T21:30:30.751857+0000","last_fullsized":"2024-09-17T21:30:30.751857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.415687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047969899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"62'281","reported_seq":1646,"reported_epoch":66,"state":"peering","last_fresh":"2024-09-17T21:30:30.751296+0000","last_change":"2024-09-17T21:30:30.751296+0000","last_active":"2024-09-17T21:30:28.741104+0000","last_peered":"2024-09-17T21:30:26.745587+0000","last_clean":"2024-09-17T21:30:07.090802+0000","last_became_active":"2024-09-17T21:30:10.614293+0000","last_became_peered":"2024-09-17T21:30:10.614293+0000","last_unstale":"2024-09-17T21:30:30.751296+0000","last_undegraded":"2024-09-17T21:30:30.751296+0000","last_fullsized":"2024-09-17T21:30:30.751296+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045510199999999998,"stat_sum":{"num_bytes":20032401,"num_objects":51,"num_object_clones":11,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":29,"num_read":2231,"num_read_kb":8308,"num_write":356,"num_write_kb":8339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.751047+0000","last_change":"2024-09-17T21:30:28.748319+0000","last_active":"2024-09-17T21:30:30.751047+0000","last_peered":"2024-09-17T21:30:30.751047+0000","last_clean":"2024-09-17T21:30:30.751047+0000","last_became_active":"2024-09-17T21:30:28.747728+0000","last_became_peered":"2024-09-17T21:30:28.747728+0000","last_unstale":"2024-09-17T21:30:30.751047+0000","last_undegraded":"2024-09-17T21:30:30.751047+0000","last_fullsized":"2024-09-17T21:30:30.751047+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:58:18.954005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"66'171","reported_seq":2181,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.806684+0000","last_change":"2024-09-17T21:29:59.583758+0000","last_active":"2024-09-17T21:30:30.806684+0000","last_peered":"2024-09-17T21:30:30.806684+0000","last_clean":"2024-09-17T21:30:30.806684+0000","last_became_active":"2024-09-17T21:29:59.542323+0000","last_became_peered":"2024-09-17T21:29:59.542323+0000","last_unstale":"2024-09-17T21:30:30.806684+0000","last_undegraded":"2024-09-17T21:30:30.806684+0000","last_fullsized":"2024-09-17T21:30:30.806684+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:28.529108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074772900000000004,"stat_sum":{"num_bytes":23918478,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":24,"num_read":2147,"num_read_kb":11598,"num_write":165,"num_write_kb":5739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":541352,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"61'430","reported_seq":2856,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.750119+0000","last_change":"2024-09-17T21:29:43.477334+0000","last_active":"2024-09-17T21:30:30.750119+0000","last_peered":"2024-09-17T21:30:30.750119+0000","last_clean":"2024-09-17T21:30:30.750119+0000","last_became_active":"2024-09-17T21:29:41.475273+0000","last_became_peered":"2024-09-17T21:29:41.475273+0000","last_unstale":"2024-09-17T21:30:30.750119+0000","last_undegraded":"2024-09-17T21:30:30.750119+0000","last_fullsized":"2024-09-17T21:30:30.750119+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:26:41.483089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083339600000000003,"stat_sum":{"num_bytes":25629016,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":17,"num_read":4273,"num_read_kb":6763,"num_write":608,"num_write_kb":2115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":50,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.751048+0000","last_change":"2024-09-17T21:30:28.744890+0000","last_active":"2024-09-17T21:30:30.751048+0000","last_peered":"2024-09-17T21:30:30.751048+0000","last_clean":"2024-09-17T21:30:30.751048+0000","last_became_active":"2024-09-17T21:30:28.744567+0000","last_became_peered":"2024-09-17T21:30:28.744567+0000","last_unstale":"2024-09-17T21:30:30.751048+0000","last_undegraded":"2024-09-17T21:30:30.751048+0000","last_fullsized":"2024-09-17T21:30:30.751048+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:52:21.176000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"66'73","reported_seq":1291,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.751284+0000","last_change":"2024-09-17T21:30:28.895927+0000","last_active":"2024-09-17T21:30:30.751284+0000","last_peered":"2024-09-17T21:30:30.751284+0000","last_clean":"2024-09-17T21:30:30.751284+0000","last_became_active":"2024-09-17T21:30:28.746690+0000","last_became_peered":"2024-09-17T21:30:28.746690+0000","last_unstale":"2024-09-17T21:30:30.751284+0000","last_undegraded":"2024-09-17T21:30:30.751284+0000","last_fullsized":"2024-09-17T21:30:30.751284+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:00:07.337429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060580400000000004,"stat_sum":{"num_bytes":9540869,"num_objects":33,"num_object_clones":5,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":21,"num_read":1182,"num_read_kb":2861,"num_write":61,"num_write_kb":1220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1513,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.749040+0000","last_change":"2024-09-17T21:29:34.981930+0000","last_active":"2024-09-17T21:30:30.749040+0000","last_peered":"2024-09-17T21:30:30.749040+0000","last_clean":"2024-09-17T21:30:30.749040+0000","last_became_active":"2024-09-17T21:29:33.071313+0000","last_became_peered":"2024-09-17T21:29:33.071313+0000","last_unstale":"2024-09-17T21:30:30.749040+0000","last_undegraded":"2024-09-17T21:30:30.749040+0000","last_fullsized":"2024-09-17T21:30:30.749040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:11:51.460374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000527295,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"68'261","reported_seq":2501,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-17T21:30:33.822474+0000","last_change":"2024-09-17T21:29:43.486638+0000","last_active":"2024-09-17T21:30:33.822474+0000","last_peered":"2024-09-17T21:30:33.822474+0000","last_clean":"2024-09-17T21:30:33.822474+0000","last_became_active":"2024-09-17T21:29:41.473964+0000","last_became_peered":"2024-09-17T21:29:41.473964+0000","last_unstale":"2024-09-17T21:30:33.822474+0000","last_undegraded":"2024-09-17T21:30:33.822474+0000","last_fullsized":"2024-09-17T21:30:33.822474+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:34:27.096694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087607199999999998,"stat_sum":{"num_bytes":61977600,"num_objects":59,"num_object_clones":13,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":31,"num_read":2437,"num_read_kb":51081,"num_write":456,"num_write_kb":32529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":115,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.751509+0000","last_change":"2024-09-17T21:30:28.746557+0000","last_active":"2024-09-17T21:30:30.751509+0000","last_peered":"2024-09-17T21:30:30.751509+0000","last_clean":"2024-09-17T21:30:30.751509+0000","last_became_active":"2024-09-17T21:30:28.746261+0000","last_became_peered":"2024-09-17T21:30:28.746261+0000","last_unstale":"2024-09-17T21:30:30.751509+0000","last_undegraded":"2024-09-17T21:30:30.751509+0000","last_fullsized":"2024-09-17T21:30:30.751509+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:42.591689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044115199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"59'124","reported_seq":2528,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.751627+0000","last_change":"2024-09-17T21:29:59.600040+0000","last_active":"2024-09-17T21:30:30.751627+0000","last_peered":"2024-09-17T21:30:30.751627+0000","last_clean":"2024-09-17T21:30:30.751627+0000","last_became_active":"2024-09-17T21:29:59.544470+0000","last_became_peered":"2024-09-17T21:29:59.544470+0000","last_unstale":"2024-09-17T21:30:30.751627+0000","last_undegraded":"2024-09-17T21:30:30.751627+0000","last_fullsized":"2024-09-17T21:30:30.751627+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:16:06.143105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062140600000000002,"stat_sum":{"num_bytes":10334695,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":23,"num_read":2853,"num_read_kb":11069,"num_write":146,"num_write_kb":3403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":504918,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":79,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.748884+0000","last_change":"2024-09-17T21:29:59.544306+0000","last_active":"2024-09-17T21:30:30.748884+0000","last_peered":"2024-09-17T21:30:30.748884+0000","last_clean":"2024-09-17T21:30:30.748884+0000","last_became_active":"2024-09-17T21:29:59.540943+0000","last_became_peered":"2024-09-17T21:29:59.540943+0000","last_unstale":"2024-09-17T21:30:30.748884+0000","last_undegraded":"2024-09-17T21:30:30.748884+0000","last_fullsized":"2024-09-17T21:30:30.748884+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:17.491167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":197,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.751573+0000","last_change":"2024-09-17T21:29:28.554672+0000","last_active":"2024-09-17T21:30:30.751573+0000","last_peered":"2024-09-17T21:30:30.751573+0000","last_clean":"2024-09-17T21:30:30.751573+0000","last_became_active":"2024-09-17T21:29:28.553733+0000","last_became_peered":"2024-09-17T21:29:28.553733+0000","last_unstale":"2024-09-17T21:30:30.751573+0000","last_undegraded":"2024-09-17T21:30:30.751573+0000","last_fullsized":"2024-09-17T21:30:30.751573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_clean_scrub_stamp":"2024-09-17T21:29:26.925225+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:36:42.585477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"65'383","reported_seq":2465,"reported_epoch":65,"state":"peering","last_fresh":"2024-09-17T21:30:30.406598+0000","last_change":"2024-09-17T21:30:30.752162+0000","last_active":"2024-09-17T21:30:30.752162+0000","last_peered":"2024-09-17T21:30:30.406598+0000","last_clean":"2024-09-17T21:30:08.710776+0000","last_became_active":"2024-09-17T21:30:30.132058+0000","last_became_peered":"2024-09-17T21:30:30.132058+0000","last_unstale":"2024-09-17T21:30:30.406598+0000","last_undegraded":"2024-09-17T21:30:30.406598+0000","last_fullsized":"2024-09-17T21:30:29.744129+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082324299999999998,"stat_sum":{"num_bytes":27179434,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":2914,"num_read_kb":10978,"num_write":549,"num_write_kb":9116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2410412,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"66'135","reported_seq":573,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.806921+0000","last_change":"2024-09-17T21:29:43.476642+0000","last_active":"2024-09-17T21:30:30.806921+0000","last_peered":"2024-09-17T21:30:30.806921+0000","last_clean":"2024-09-17T21:30:30.806921+0000","last_became_active":"2024-09-17T21:29:41.477965+0000","last_became_peered":"2024-09-17T21:29:41.477965+0000","last_unstale":"2024-09-17T21:30:30.806921+0000","last_undegraded":"2024-09-17T21:30:30.806921+0000","last_fullsized":"2024-09-17T21:30:30.806921+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:21.386185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035185700000000002,"stat_sum":{"num_bytes":17190883,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":18,"num_read":372,"num_read_kb":2202,"num_write":191,"num_write_kb":6821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"66'235","reported_seq":2339,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.807183+0000","last_change":"2024-09-17T21:29:43.563885+0000","last_active":"2024-09-17T21:30:30.807183+0000","last_peered":"2024-09-17T21:30:30.807183+0000","last_clean":"2024-09-17T21:30:30.807183+0000","last_became_active":"2024-09-17T21:29:41.473420+0000","last_became_peered":"2024-09-17T21:29:41.473420+0000","last_unstale":"2024-09-17T21:30:30.807183+0000","last_undegraded":"2024-09-17T21:30:30.807183+0000","last_fullsized":"2024-09-17T21:30:30.807183+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:32.204391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046561200000000001,"stat_sum":{"num_bytes":50090793,"num_objects":48,"num_object_clones":8,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":28,"num_read":2205,"num_read_kb":9970,"num_write":259,"num_write_kb":3919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"66'200","reported_seq":4545,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.796885+0000","last_change":"2024-09-17T21:29:43.472555+0000","last_active":"2024-09-17T21:30:30.796885+0000","last_peered":"2024-09-17T21:30:30.796885+0000","last_clean":"2024-09-17T21:30:30.796885+0000","last_became_active":"2024-09-17T21:29:41.472354+0000","last_became_peered":"2024-09-17T21:29:41.472354+0000","last_unstale":"2024-09-17T21:30:30.796885+0000","last_undegraded":"2024-09-17T21:30:30.796885+0000","last_fullsized":"2024-09-17T21:30:30.796885+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:17:31.663727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073818799999999997,"stat_sum":{"num_bytes":23840566,"num_objects":56,"num_object_clones":13,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":26,"num_read":4766,"num_read_kb":14247,"num_write":211,"num_write_kb":1031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"65'433","reported_seq":3218,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.779697+0000","last_change":"2024-09-17T21:30:02.564126+0000","last_active":"2024-09-17T21:30:30.779697+0000","last_peered":"2024-09-17T21:30:30.779697+0000","last_clean":"2024-09-17T21:30:30.779697+0000","last_became_active":"2024-09-17T21:30:02.563793+0000","last_became_peered":"2024-09-17T21:30:02.563793+0000","last_unstale":"2024-09-17T21:30:30.779697+0000","last_undegraded":"2024-09-17T21:30:30.779697+0000","last_fullsized":"2024-09-17T21:30:30.779697+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:10:54.350888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047685499999999999,"stat_sum":{"num_bytes":78544342,"num_objects":51,"num_object_clones":10,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":26,"num_read":5343,"num_read_kb":11188,"num_write":402,"num_write_kb":3646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":285875,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"62'299","reported_seq":2395,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.751694+0000","last_change":"2024-09-17T21:29:59.632891+0000","last_active":"2024-09-17T21:30:30.751694+0000","last_peered":"2024-09-17T21:30:30.751694+0000","last_clean":"2024-09-17T21:30:30.751694+0000","last_became_active":"2024-09-17T21:29:59.544069+0000","last_became_peered":"2024-09-17T21:29:59.544069+0000","last_unstale":"2024-09-17T21:30:30.751694+0000","last_undegraded":"2024-09-17T21:30:30.751694+0000","last_fullsized":"2024-09-17T21:30:30.751694+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:58:44.836569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065633800000000002,"stat_sum":{"num_bytes":11356007,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":23,"num_read":2668,"num_read_kb":4505,"num_write":424,"num_write_kb":2058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":669702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"61'262","reported_seq":3015,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.751729+0000","last_change":"2024-09-17T21:29:43.563860+0000","last_active":"2024-09-17T21:30:30.751729+0000","last_peered":"2024-09-17T21:30:30.751729+0000","last_clean":"2024-09-17T21:30:30.751729+0000","last_became_active":"2024-09-17T21:29:41.473736+0000","last_became_peered":"2024-09-17T21:29:41.473736+0000","last_unstale":"2024-09-17T21:30:30.751729+0000","last_undegraded":"2024-09-17T21:30:30.751729+0000","last_fullsized":"2024-09-17T21:30:30.751729+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:51.692155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039426499999999998,"stat_sum":{"num_bytes":14966602,"num_objects":42,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":24,"num_read":3248,"num_read_kb":12980,"num_write":387,"num_write_kb":3146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"65'218","reported_seq":3001,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.751759+0000","last_change":"2024-09-17T21:29:43.564285+0000","last_active":"2024-09-17T21:30:30.751759+0000","last_peered":"2024-09-17T21:30:30.751759+0000","last_clean":"2024-09-17T21:30:30.751759+0000","last_became_active":"2024-09-17T21:29:41.473394+0000","last_became_peered":"2024-09-17T21:29:41.473394+0000","last_unstale":"2024-09-17T21:30:30.751759+0000","last_undegraded":"2024-09-17T21:30:30.751759+0000","last_fullsized":"2024-09-17T21:30:30.751759+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:06:23.634804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073387300000000001,"stat_sum":{"num_bytes":30062250,"num_objects":53,"num_object_clones":12,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":32,"num_read":3021,"num_read_kb":4706,"num_write":228,"num_write_kb":17632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"62'281","reported_seq":824,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.752423+0000","last_change":"2024-09-17T21:29:43.564107+0000","last_active":"2024-09-17T21:30:30.752423+0000","last_peered":"2024-09-17T21:30:30.752423+0000","last_clean":"2024-09-17T21:30:30.752423+0000","last_became_active":"2024-09-17T21:29:41.471342+0000","last_became_peered":"2024-09-17T21:29:41.471342+0000","last_unstale":"2024-09-17T21:30:30.752423+0000","last_undegraded":"2024-09-17T21:30:30.752423+0000","last_fullsized":"2024-09-17T21:30:30.752423+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:00:03.057673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062934700000000004,"stat_sum":{"num_bytes":8936267,"num_objects":38,"num_object_clones":5,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":21,"num_read":1184,"num_read_kb":3265,"num_write":404,"num_write_kb":7039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"62'202","reported_seq":3316,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.751710+0000","last_change":"2024-09-17T21:29:43.563795+0000","last_active":"2024-09-17T21:30:30.751710+0000","last_peered":"2024-09-17T21:30:30.751710+0000","last_clean":"2024-09-17T21:30:30.751710+0000","last_became_active":"2024-09-17T21:29:41.473252+0000","last_became_peered":"2024-09-17T21:29:41.473252+0000","last_unstale":"2024-09-17T21:30:30.751710+0000","last_undegraded":"2024-09-17T21:30:30.751710+0000","last_fullsized":"2024-09-17T21:30:30.751710+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:42:21.655170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044419899999999999,"stat_sum":{"num_bytes":37328252,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":3705,"num_read_kb":23673,"num_write":207,"num_write_kb":10610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"68'138","reported_seq":1568,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-17T21:30:33.675981+0000","last_change":"2024-09-17T21:29:43.486918+0000","last_active":"2024-09-17T21:30:33.675981+0000","last_peered":"2024-09-17T21:30:33.675981+0000","last_clean":"2024-09-17T21:30:33.675981+0000","last_became_active":"2024-09-17T21:29:41.477019+0000","last_became_peered":"2024-09-17T21:29:41.477019+0000","last_unstale":"2024-09-17T21:30:33.675981+0000","last_undegraded":"2024-09-17T21:30:33.675981+0000","last_fullsized":"2024-09-17T21:30:33.675981+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:53:05.233480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010796479999999999,"stat_sum":{"num_bytes":16702817,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":22,"num_read":1395,"num_read_kb":4086,"num_write":115,"num_write_kb":1151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"66'159","reported_seq":2392,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-17T21:30:30.807157+0000","last_change":"2024-09-17T21:29:43.564292+0000","last_active":"2024-09-17T21:30:30.807157+0000","last_peered":"2024-09-17T21:30:30.807157+0000","last_clean":"2024-09-17T21:30:30.807157+0000","last_became_active":"2024-09-17T21:29:41.473182+0000","last_became_peered":"2024-09-17T21:29:41.473182+0000","last_unstale":"2024-09-17T21:30:30.807157+0000","last_undegraded":"2024-09-17T21:30:30.807157+0000","last_fullsized":"2024-09-17T21:30:30.807157+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:37.192419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052844700000000003,"stat_sum":{"num_bytes":17521092,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":32,"num_read":2494,"num_read_kb":11067,"num_write":174,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1008652756,"num_objects":1509,"num_object_clones":315,"num_object_copies":3018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1509,"num_whiteouts":784,"num_read":92357,"num_read_kb":435223,"num_write":11430,"num_write_kb":283035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":43917444,"num_keys_recovered":126,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":46600192,"data_stored":420603804,"data_compressed":8433834,"data_compressed_allocated":32555008,"data_compressed_original":406451794,"omap_allocated":10476,"internal_metadata":0},"log_size":8783,"ondisk_log_size":8783,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45664,"kb_used_data":6160,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326176,"statfs":{"total":96636764160,"available":96590004224,"internally_reserved":0,"allocated":6307840,"data_stored":30342549,"data_compressed":950559,"data_compressed_allocated":3272704,"data_compressed_original":27684152,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39508,"kb_used_data":4152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332332,"statfs":{"total":96636764160,"available":96596307968,"internally_reserved":0,"allocated":4251648,"data_stored":24115344,"data_compressed":969392,"data_compressed_allocated":2400256,"data_compressed_original":22764247,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":0,"seq":64424509457,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49408,"kb_used_data":9896,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322432,"statfs":{"total":96636764160,"available":96586170368,"internally_reserved":0,"allocated":10133504,"data_stored":80287355,"data_compressed":1659019,"data_compressed_allocated":6492160,"data_compressed_original":77037113,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47952,"kb_used_data":8444,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323888,"statfs":{"total":96636764160,"available":96587661312,"internally_reserved":0,"allocated":8646656,"data_stored":67754757,"data_compressed":1604556,"data_compressed_allocated":5636096,"data_compressed_original":65395733,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36008,"kb_used_data":4820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335832,"statfs":{"total":96636764160,"available":96599891968,"internally_reserved":0,"allocated":4935680,"data_stored":50281640,"data_compressed":563060,"data_compressed_allocated":2977792,"data_compressed_original":48482231,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":63,"seq":270582939650,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35752,"kb_used_data":7120,"kb_used_omap":20,"kb_used_meta":28587,"kb_avail":94336088,"statfs":{"total":96636764160,"available":96600154112,"internally_reserved":0,"allocated":7290880,"data_stored":64271290,"data_compressed":799714,"data_compressed_allocated":4931584,"data_compressed_original":62464000,"omap_allocated":20925,"internal_metadata":29273667},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49176,"kb_used_data":9668,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322664,"statfs":{"total":96636764160,"available":96586407936,"internally_reserved":0,"allocated":9900032,"data_stored":105149458,"data_compressed":1654399,"data_compressed_allocated":6590464,"data_compressed_original":101952574,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9027584,"data_stored":104412909,"data_compressed":1643628,"data_compressed_allocated":6553600,"data_compressed_original":101604414,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6524928,"data_stored":63930186,"data_compressed":799714,"data_compressed_allocated":4931584,"data_compressed_original":62464000,"omap_allocated":10476,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":49946845,"data_compressed":563060,"data_compressed_allocated":2977792,"data_compressed_original":48482231,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7905280,"data_stored":67419956,"data_compressed":1604556,"data_compressed_allocated":5636096,"data_compressed_original":65395733,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":9383936,"data_stored":79952497,"data_compressed":1659019,"data_compressed_allocated":6492160,"data_compressed_original":77037113,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":1737628,"data_compressed":248875,"data_compressed_allocated":327680,"data_compressed_original":1593344,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3514368,"data_stored":23786419,"data_compressed":969392,"data_compressed_allocated":2400256,"data_compressed_original":22764247,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5505024,"data_stored":29417364,"data_compressed":945590,"data_compressed_allocated":3235840,"data_compressed_original":27110712,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:30:36.614 INFO:tasks.thrashosds.thrasher:Setting 2.0 to [0, 5, 3, 1] 2024-09-17T21:30:36.614 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.0', '0', '5', '3', '1'] 2024-09-17T21:30:36.615 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.0 0 5 3 1 2024-09-17T21:30:36.617 INFO:teuthology.orchestra.run.smithi129.stdout:4029 write 0x3016a5f thru 0x301bc00 (0x51a2 bytes) 2024-09-17T21:30:36.624 INFO:teuthology.orchestra.run.smithi129.stdout:4030 writesame 0x1199362 thru 0x119be06 data_size 0x3(0x2aa5 bytes) 2024-09-17T21:30:36.626 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:36.626 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:36.628 INFO:teuthology.orchestra.run.smithi129.stdout:4031 trunc from 0x89e83a6 to 0x9d964e6 2024-09-17T21:30:36.633 INFO:teuthology.orchestra.run.smithi129.stdout:4032 read 0x1fd26ea thru 0x1fd5496 (0x2dad bytes) 2024-09-17T21:30:36.633 INFO:teuthology.orchestra.run.smithi129.stdout:4033 read 0x11704cc thru 0x117ddac (0xd8e1 bytes) 2024-09-17T21:30:36.634 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.634 INFO:teuthology.orchestra.run.smithi129.stdout:4037 read 0x4e4d50d thru 0x4e56368 (0x8e5c bytes) 2024-09-17T21:30:36.634 INFO:teuthology.orchestra.run.smithi129.stdout:4038 write 0x111774b thru 0x1125db0 (0xe666 bytes) 2024-09-17T21:30:36.636 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.636 INFO:teuthology.orchestra.run.smithi129.stdout:4040 writesame 0x458d75c thru 0x4596f7b data_size 0x4(0x9820 bytes) 2024-09-17T21:30:36.644 INFO:teuthology.orchestra.run.smithi129.stdout:4043 write 0x29393e6 thru 0x2946255 (0xce70 bytes) 2024-09-17T21:30:36.645 INFO:teuthology.orchestra.run.smithi129.stdout:4045 write 0x446e846 thru 0x4478175 (0x9930 bytes) 2024-09-17T21:30:36.649 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:36.650 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:36.652 INFO:teuthology.orchestra.run.smithi129.stdout:4046 trunc from 0x9d964e6 to 0x861f302 2024-09-17T21:30:36.658 INFO:teuthology.orchestra.run.smithi129.stdout:4047 punch from 0x7eedf6 to 0x7efec3, (0x10cd bytes) 2024-09-17T21:30:36.658 INFO:teuthology.orchestra.run.smithi129.stdout:4048 read 0x32a096 thru 0x33336f (0x92da bytes) 2024-09-17T21:30:36.659 INFO:teuthology.orchestra.run.smithi129.stdout:4049 read 0x592563e thru 0x592c371 (0x6d34 bytes) 2024-09-17T21:30:36.665 INFO:teuthology.orchestra.run.smithi129.stdout:4050 writesame 0x9dfe17e thru 0x9dfe1bf data_size 0x1(0x42 bytes) 2024-09-17T21:30:36.672 INFO:teuthology.orchestra.run.smithi129.stdout:4051 trunc from 0x9dfe1c0 to 0x4442b35 2024-09-17T21:30:36.677 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:36.675+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:36.681 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:36.681 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:36.684 INFO:teuthology.orchestra.run.smithi129.stdout:4053 write 0x291b7d3 thru 0x292432b (0x8b59 bytes) 2024-09-17T21:30:36.690 INFO:teuthology.orchestra.run.smithi129.stdout:4054 trunc from 0x4442b35 to 0x6512331 2024-09-17T21:30:36.696 INFO:teuthology.orchestra.run.smithi129.stdout:4055 read 0x57c9ef9 thru 0x57d5780 (0xb888 bytes) 2024-09-17T21:30:36.696 INFO:teuthology.orchestra.run.smithi129.stdout:4057 writesame 0x16e005d thru 0x16e9296 data_size 0x11(0x923a bytes) 2024-09-17T21:30:36.698 INFO:teuthology.orchestra.run.smithi129.stdout:4058 read 0x4ee8036 thru 0x4eec0ac (0x4077 bytes) 2024-09-17T21:30:36.700 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:36.701 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:36.712 INFO:teuthology.orchestra.run.smithi129.stdout:4059 write 0xbe2df89 thru 0xbe3cc25 (0xec9d bytes) 2024-09-17T21:30:36.718 INFO:teuthology.orchestra.run.smithi129.stdout:4060 read 0x738ebb2 thru 0x739383d (0x4c8c bytes) 2024-09-17T21:30:36.719 INFO:teuthology.orchestra.run.smithi129.stdout:4061 read 0x5ee7adf thru 0x5ee7d3b (0x25d bytes) 2024-09-17T21:30:36.719 INFO:teuthology.orchestra.run.smithi129.stdout:4062 punch from 0x45fd6fd to 0x4604d50, (0x7653 bytes) 2024-09-17T21:30:36.719 INFO:teuthology.orchestra.run.smithi129.stdout:4063 writesame 0x3665714 thru 0x366fdf1 data_size 0x3e(0xa6de bytes) 2024-09-17T21:30:36.720 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.720 INFO:teuthology.orchestra.run.smithi129.stdout:4066 write 0xb9aa3b5 thru 0xb9aa565 (0x1b1 bytes) 2024-09-17T21:30:36.722 INFO:teuthology.orchestra.run.smithi129.stdout:4067 write 0xb003624 thru 0xb011255 (0xdc32 bytes) 2024-09-17T21:30:36.729 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:36.729 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:36.730 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.730 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.730 INFO:teuthology.orchestra.run.smithi129.stdout:4071 read 0x9414e3f thru 0x942056a (0xb72c bytes) 2024-09-17T21:30:36.730 INFO:teuthology.orchestra.run.smithi129.stdout:4072 read 0x1a28a93 thru 0x1a38013 (0xf581 bytes) 2024-09-17T21:30:36.731 INFO:teuthology.orchestra.run.smithi129.stdout:4074 read 0x73f8e43 thru 0x7401e8c (0x904a bytes) 2024-09-17T21:30:36.731 INFO:teuthology.orchestra.run.smithi129.stdout:4075 read 0xa833c2 thru 0xa8df53 (0xab92 bytes) 2024-09-17T21:30:36.732 INFO:teuthology.orchestra.run.smithi129.stdout:4078 read 0x6075d95 thru 0x607f368 (0x95d4 bytes) 2024-09-17T21:30:36.732 INFO:teuthology.orchestra.run.smithi129.stdout:4080 read 0x68580e5 thru 0x685c14e (0x406a bytes) 2024-09-17T21:30:36.732 INFO:teuthology.orchestra.run.smithi129.stdout:4081 read 0xb42e2fd thru 0xb43218f (0x3e93 bytes) 2024-09-17T21:30:36.736 INFO:teuthology.orchestra.run.smithi129.stdout:4082 write 0xbf47315 thru 0xbf4dabc (0x67a8 bytes) 2024-09-17T21:30:36.740 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:30:36.749 INFO:teuthology.orchestra.run.smithi129.stdout:4083 write 0xc3b71e4 thru 0xc3c4df7 (0xdc14 bytes) 2024-09-17T21:30:36.758 INFO:teuthology.orchestra.run.smithi129.stdout:4084 write 0xd7cbd6d thru 0xd7d9508 (0xd79c bytes) 2024-09-17T21:30:36.764 INFO:teuthology.orchestra.run.smithi129.stdout:4085 punch from 0xce2c4ff to 0xce3b4cd, (0xefce bytes) 2024-09-17T21:30:36.765 INFO:teuthology.orchestra.run.smithi129.stdout:4086 read 0xb4eb1ac thru 0xb4f7747 (0xc59c bytes) 2024-09-17T21:30:36.765 INFO:teuthology.orchestra.run.smithi129.stdout:4087 read 0x8765044 thru 0x8773aba (0xea77 bytes) 2024-09-17T21:30:36.765 INFO:teuthology.orchestra.run.smithi129.stdout:4088 write 0x9ced7af thru 0x9cf96bd (0xbf0f bytes) 2024-09-17T21:30:36.768 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.768 INFO:teuthology.orchestra.run.smithi129.stdout:4092 punch from 0x296cbd2 to 0x2972ffb, (0x6429 bytes) 2024-09-17T21:30:36.768 INFO:teuthology.orchestra.run.smithi129.stdout:4094 write 0x64a6db8 thru 0x64adcb1 (0x6efa bytes) 2024-09-17T21:30:36.774 INFO:teuthology.orchestra.run.smithi129.stdout:4095 read 0x36e4d61 thru 0x36eb8d9 (0x6b79 bytes) 2024-09-17T21:30:36.774 INFO:teuthology.orchestra.run.smithi129.stdout:4096 writesame 0x1ae4df thru 0x1ba365 data_size 0x5(0xbe87 bytes) 2024-09-17T21:30:36.778 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:36.775+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:36.778 INFO:teuthology.orchestra.run.smithi129.stdout:4097 write 0x3fcce81 thru 0x3fd8de6 (0xbf66 bytes) 2024-09-17T21:30:36.784 INFO:teuthology.orchestra.run.smithi129.stdout:4099 read 0x4c52225 thru 0x4c54c0e (0x29ea bytes) 2024-09-17T21:30:36.784 INFO:teuthology.orchestra.run.smithi129.stdout:4101 read 0x40badd9 thru 0x40c6ef5 (0xc11d bytes) 2024-09-17T21:30:36.785 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.785 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:36.785 INFO:teuthology.orchestra.run.smithi129.stdout:4105 read 0x107ab22 thru 0x1086c9b (0xc17a bytes) 2024-09-17T21:30:36.785 INFO:teuthology.orchestra.run.smithi129.stdout:4107 clone 30 order 25 su 2097152 sc 12 2024-09-17T21:30:36.792 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:30:36.861 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:36.861 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:36.870 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:30:36.879 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:36.876+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:36.977 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:36.977 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:36.980 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:36.977+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:36.987 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:30:37.081 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:37.079+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:37.132 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:37.132 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:37.140 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:30:37.182 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:37.180+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:37.283 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:37.283 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:37.284 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:37.280+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:37.292 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:30:37.385 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:37.383+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:37.396 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:37.396 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:37.405 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:30:37.486 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:37.483+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:37.509 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:37.509 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:37.518 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:30:37.587 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:37.585+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:37.620 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:37.620 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:37.629 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:30:37.688 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:37.686+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:37.773 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:37.773 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:37.781 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:30:37.788 INFO:teuthology.orchestra.run.smithi129.stderr:set 2.0 pg_upmap_items mapping to [0->5,3->1] 2024-09-17T21:30:37.788 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:37.786+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:37.791 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is unset 2024-09-17T21:30:37.874 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone29 from 0xd7d9509 (overlap 0x1417dfb) to 0xc0e530 2024-09-17T21:30:37.889 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:37.886+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:37.912 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:37.912 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:37.924 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:30:37.990 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:37.988+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:38.001 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-17T21:30:38.040 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.040 INFO:teuthology.orchestra.run.smithi129.stdout:4110 read 0xadd1530 thru 0xadd69ad (0x547e bytes) 2024-09-17T21:30:38.041 INFO:teuthology.orchestra.run.smithi129.stdout:4111 punch from 0x9785c49 to 0x9792b8c, (0xcf43 bytes) 2024-09-17T21:30:38.044 INFO:teuthology.orchestra.run.smithi129.stdout:4112 write 0xaa32753 thru 0xaa35b85 (0x3433 bytes) 2024-09-17T21:30:38.059 INFO:teuthology.orchestra.run.smithi129.stdout:4113 read 0xd5310b6 thru 0xd53c85f (0xb7aa bytes) 2024-09-17T21:30:38.061 INFO:teuthology.orchestra.run.smithi129.stdout:4114 trunc from 0xd7d9509 to 0xc220541 2024-09-17T21:30:38.086 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:38.086 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:38.091 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:38.089+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:38.095 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:30:38.111 INFO:teuthology.orchestra.run.smithi129.stdout:4115 write 0x241b999 thru 0x242ac4b (0xf2b3 bytes) 2024-09-17T21:30:38.115 INFO:teuthology.orchestra.run.smithi129.stdout:4117 trunc from 0xc220541 to 0xe56933b 2024-09-17T21:30:38.120 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.120 INFO:teuthology.orchestra.run.smithi129.stdout:4119 read 0xa4a31a3 thru 0xa4ab53e (0x839c bytes) 2024-09-17T21:30:38.120 INFO:teuthology.orchestra.run.smithi129.stdout:4120 write 0xbfaa44a thru 0xbface60 (0x2a17 bytes) 2024-09-17T21:30:38.122 INFO:teuthology.orchestra.run.smithi129.stdout:4121 writesame 0xde1be0a thru 0xde1f248 data_size 0x5(0x343f bytes) 2024-09-17T21:30:38.124 INFO:teuthology.orchestra.run.smithi129.stdout:4122 punch from 0x3a54e9d to 0x3a631d3, (0xe336 bytes) 2024-09-17T21:30:38.124 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.124 INFO:teuthology.orchestra.run.smithi129.stdout:4125 punch from 0x7fa0b51 to 0x7fb0275, (0xf724 bytes) 2024-09-17T21:30:38.124 INFO:teuthology.orchestra.run.smithi129.stdout:4126 read 0x6bf12a6 thru 0x6bf8fe7 (0x7d42 bytes) 2024-09-17T21:30:38.124 INFO:teuthology.orchestra.run.smithi129.stdout:4129 read 0x5e9be9a thru 0x5ea1819 (0x5980 bytes) 2024-09-17T21:30:38.125 INFO:teuthology.orchestra.run.smithi129.stdout:4130 write 0x1a01a2a thru 0x1a0c5a5 (0xab7c bytes) 2024-09-17T21:30:38.126 INFO:teuthology.orchestra.run.smithi129.stdout:4132 read 0xe9c0bc thru 0xea677e (0xa6c3 bytes) 2024-09-17T21:30:38.127 INFO:teuthology.orchestra.run.smithi129.stdout:4134 write 0x38679ec thru 0x3868227 (0x83c bytes) 2024-09-17T21:30:38.130 INFO:teuthology.orchestra.run.smithi129.stdout:4135 trunc from 0xe56933b to 0xcf8bc78 2024-09-17T21:30:38.149 INFO:teuthology.orchestra.run.smithi129.stdout:4136 write 0x65f5f26 thru 0x65ffe3a (0x9f15 bytes) 2024-09-17T21:30:38.151 INFO:teuthology.orchestra.run.smithi129.stdout:4137 write 0x2c014b5 thru 0x2c0d6ca (0xc216 bytes) 2024-09-17T21:30:38.153 INFO:teuthology.orchestra.run.smithi129.stdout:4140 write 0x7df8400 thru 0x7e034d2 (0xb0d3 bytes) 2024-09-17T21:30:38.154 INFO:teuthology.orchestra.run.smithi129.stdout:4141 writesame 0x9f5560a thru 0x9f60f9a data_size 0x5(0xb991 bytes) 2024-09-17T21:30:38.159 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.159 INFO:teuthology.orchestra.run.smithi129.stdout:4143 punch from 0x936e86e to 0x9375244, (0x69d6 bytes) 2024-09-17T21:30:38.159 INFO:teuthology.orchestra.run.smithi129.stdout:4144 punch from 0x71c677d to 0x71c6bcc, (0x44f bytes) 2024-09-17T21:30:38.159 INFO:teuthology.orchestra.run.smithi129.stdout:4145 writesame 0x70fe26c thru 0x71035a8 data_size 0x1(0x533d bytes) 2024-09-17T21:30:38.167 INFO:teuthology.orchestra.run.smithi129.stdout:4147 punch from 0x9ea26c7 to 0x9ea5983, (0x32bc bytes) 2024-09-17T21:30:38.168 INFO:teuthology.orchestra.run.smithi129.stdout:4148 read 0x5c47286 thru 0x5c4b17a (0x3ef5 bytes) 2024-09-17T21:30:38.168 INFO:teuthology.orchestra.run.smithi129.stdout:4149 read 0xc0daf7b thru 0xc0e7fc9 (0xd04f bytes) 2024-09-17T21:30:38.168 INFO:teuthology.orchestra.run.smithi129.stdout:4150 read 0xbf36897 thru 0xbf41fd8 (0xb742 bytes) 2024-09-17T21:30:38.169 INFO:teuthology.orchestra.run.smithi129.stdout:4151 read 0x84e7fbb thru 0x84f38b3 (0xb8f9 bytes) 2024-09-17T21:30:38.169 INFO:teuthology.orchestra.run.smithi129.stdout:4152 punch from 0x7176ffe to 0x71850a0, (0xe0a2 bytes) 2024-09-17T21:30:38.169 INFO:teuthology.orchestra.run.smithi129.stdout:4153 read 0x3fd967b thru 0x3fe377b (0xa101 bytes) 2024-09-17T21:30:38.170 INFO:teuthology.orchestra.run.smithi129.stdout:4154 writesame 0xa7d14d7 thru 0xa7dfb04 data_size 0x45(0xe62e bytes) 2024-09-17T21:30:38.171 INFO:teuthology.orchestra.run.smithi129.stdout:4155 read 0x9275efc thru 0x927d96f (0x7a74 bytes) 2024-09-17T21:30:38.172 INFO:teuthology.orchestra.run.smithi129.stdout:4157 read 0x34014f thru 0x34c6fb (0xc5ad bytes) 2024-09-17T21:30:38.172 INFO:teuthology.orchestra.run.smithi129.stdout:4159 punch from 0x5f2e0a9 to 0x5f37a9e, (0x99f5 bytes) 2024-09-17T21:30:38.172 INFO:teuthology.orchestra.run.smithi129.stdout:4160 read 0xc38b4f1 thru 0xc39318f (0x7c9f bytes) 2024-09-17T21:30:38.173 INFO:teuthology.orchestra.run.smithi129.stdout:4161 write 0x2902e8f thru 0x290a94a (0x7abc bytes) 2024-09-17T21:30:38.174 INFO:teuthology.orchestra.run.smithi129.stdout:4162 punch from 0xc7262f9 to 0xc72b20a, (0x4f11 bytes) 2024-09-17T21:30:38.174 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.175 INFO:teuthology.orchestra.run.smithi129.stdout:4165 write 0x2817d86 thru 0x28252d7 (0xd552 bytes) 2024-09-17T21:30:38.177 INFO:teuthology.orchestra.run.smithi129.stdout:4166 punch from 0xc117857 to 0xc11f612, (0x7dbb bytes) 2024-09-17T21:30:38.177 INFO:teuthology.orchestra.run.smithi129.stdout:4168 punch from 0x1dc89b to 0x1e7724, (0xae89 bytes) 2024-09-17T21:30:38.177 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.183 INFO:teuthology.orchestra.run.smithi129.stdout:4170 write 0xe36a79a thru 0xe3780af (0xd916 bytes) 2024-09-17T21:30:38.185 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.185 INFO:teuthology.orchestra.run.smithi129.stdout:4172 write 0x78ac2be thru 0x78b6500 (0xa243 bytes) 2024-09-17T21:30:38.187 INFO:teuthology.orchestra.run.smithi129.stdout:4173 write 0x32483e thru 0x32d08c (0x884f bytes) 2024-09-17T21:30:38.188 INFO:teuthology.orchestra.run.smithi129.stdout:4174 write 0x7af14dd thru 0x7af536d (0x3e91 bytes) 2024-09-17T21:30:38.193 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:38.190+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:38.203 INFO:teuthology.orchestra.run.smithi129.stdout:4175 trunc from 0xe3780b0 to 0x2df9370 2024-09-17T21:30:38.203 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:38.203 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:38.211 INFO:teuthology.orchestra.run.smithi129.stdout:4176 read 0x16f2c3b thru 0x16f7d31 (0x50f7 bytes) 2024-09-17T21:30:38.213 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:30:38.217 INFO:teuthology.orchestra.run.smithi129.stdout:4177 trunc from 0x2df9370 to 0x81e6786 2024-09-17T21:30:38.222 INFO:teuthology.orchestra.run.smithi129.stdout:4178 read 0xb82612 thru 0xb92484 (0xfe73 bytes) 2024-09-17T21:30:38.227 INFO:teuthology.orchestra.run.smithi129.stdout:4179 trunc from 0x81e6786 to 0xb666a9c 2024-09-17T21:30:38.233 INFO:teuthology.orchestra.run.smithi129.stdout:4180 read 0x574cc5 thru 0x575240 (0x57c bytes) 2024-09-17T21:30:38.233 INFO:teuthology.orchestra.run.smithi129.stdout:4182 write 0x9016816 thru 0x9024001 (0xd7ec bytes) 2024-09-17T21:30:38.236 INFO:teuthology.orchestra.run.smithi129.stdout:4183 write 0x74f57f4 thru 0x750118c (0xb999 bytes) 2024-09-17T21:30:38.237 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.245 INFO:teuthology.orchestra.run.smithi129.stdout:4185 write 0xd05a98a thru 0xd05cda9 (0x2420 bytes) 2024-09-17T21:30:38.246 INFO:teuthology.orchestra.run.smithi129.stdout:4186 read 0xbf151f5 thru 0xbf19d75 (0x4b81 bytes) 2024-09-17T21:30:38.252 INFO:teuthology.orchestra.run.smithi129.stdout:4188 writesame 0xd833374 thru 0xd841ccf data_size 0x91(0xe95c bytes) 2024-09-17T21:30:38.254 INFO:teuthology.orchestra.run.smithi129.stdout:4189 trunc from 0xd841cd0 to 0xe276cfa 2024-09-17T21:30:38.259 INFO:teuthology.orchestra.run.smithi129.stdout:4191 write 0xdf52bb4 thru 0xdf5bfb6 (0x9403 bytes) 2024-09-17T21:30:38.261 INFO:teuthology.orchestra.run.smithi129.stdout:4192 writesame 0xd37ebf6 thru 0xd388230 data_size 0x1(0x963b bytes) 2024-09-17T21:30:38.279 INFO:teuthology.orchestra.run.smithi129.stdout:4193 write 0xd6daa2c thru 0xd6e6c93 (0xc268 bytes) 2024-09-17T21:30:38.282 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.282 INFO:teuthology.orchestra.run.smithi129.stdout:4196 write 0x99342a3 thru 0x993adc9 (0x6b27 bytes) 2024-09-17T21:30:38.283 INFO:teuthology.orchestra.run.smithi129.stdout:4197 punch from 0x713c611 to 0x714977a, (0xd169 bytes) 2024-09-17T21:30:38.283 INFO:teuthology.orchestra.run.smithi129.stdout:4198 read 0x1513cfd thru 0x15150fa (0x13fe bytes) 2024-09-17T21:30:38.283 INFO:teuthology.orchestra.run.smithi129.stdout:4199 writesame 0x67dff93 thru 0x67eb3f9 data_size 0x1(0xb467 bytes) 2024-09-17T21:30:38.294 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:38.292+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:38.304 INFO:teuthology.orchestra.run.smithi129.stdout:4200 punch from 0xd1670c0 to 0xd16c103, (0x5043 bytes) 2024-09-17T21:30:38.304 INFO:teuthology.orchestra.run.smithi129.stdout:4201 read 0x283938a thru 0x283a0b3 (0xd2a bytes) 2024-09-17T21:30:38.305 INFO:teuthology.orchestra.run.smithi129.stdout:4202 write 0x31b6cf0 thru 0x31c2957 (0xbc68 bytes) 2024-09-17T21:30:38.307 INFO:teuthology.orchestra.run.smithi129.stdout:4203 read 0xa8d937 thru 0xa8e80f (0xed9 bytes) 2024-09-17T21:30:38.307 INFO:teuthology.orchestra.run.smithi129.stdout:4204 read 0x9132b79 thru 0x913919f (0x6627 bytes) 2024-09-17T21:30:38.307 INFO:teuthology.orchestra.run.smithi129.stdout:4205 writesame 0x4a6035b thru 0x4a6517c data_size 0x49(0x4e22 bytes) 2024-09-17T21:30:38.309 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.309 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.321 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:38.321 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:38.324 INFO:teuthology.orchestra.run.smithi129.stdout:4209 trunc from 0xe276cfa to 0x2211b82 2024-09-17T21:30:38.330 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:30:38.353 INFO:teuthology.orchestra.run.smithi129.stdout:4210 write 0xdb92f6a thru 0xdb96534 (0x35cb bytes) 2024-09-17T21:30:38.354 INFO:teuthology.orchestra.run.smithi129.stdout:4211 read 0xcb0b8d1 thru 0xcb0d995 (0x20c5 bytes) 2024-09-17T21:30:38.354 INFO:teuthology.orchestra.run.smithi129.stdout:4213 read 0x7754c02 thru 0x7763a2e (0xee2d bytes) 2024-09-17T21:30:38.356 INFO:teuthology.orchestra.run.smithi129.stdout:4215 trunc from 0xdb96535 to 0xe819b4b 2024-09-17T21:30:38.360 INFO:teuthology.orchestra.run.smithi129.stdout:4216 punch from 0x8a1c9a2 to 0x8a231a0, (0x67fe bytes) 2024-09-17T21:30:38.361 INFO:teuthology.orchestra.run.smithi129.stdout:4217 read 0x37aa6de thru 0x37aceda (0x27fd bytes) 2024-09-17T21:30:38.361 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.361 INFO:teuthology.orchestra.run.smithi129.stdout:4223 punch from 0x9db3f0d to 0x9dba409, (0x64fc bytes) 2024-09-17T21:30:38.369 INFO:teuthology.orchestra.run.smithi129.stdout:4224 trunc from 0xe819b4b to 0x7ca6bf2 2024-09-17T21:30:38.390 INFO:teuthology.orchestra.run.smithi129.stdout:4227 write 0xe6ddad1 thru 0xe6e1d5e (0x428e bytes) 2024-09-17T21:30:38.392 INFO:teuthology.orchestra.run.smithi129.stdout:4228 read 0x77d6fda thru 0x77da9e7 (0x3a0e bytes) 2024-09-17T21:30:38.392 INFO:teuthology.orchestra.run.smithi129.stdout:4229 punch from 0x97532df to 0x975f8a7, (0xc5c8 bytes) 2024-09-17T21:30:38.392 INFO:teuthology.orchestra.run.smithi129.stdout:4231 read 0x94e114a thru 0x94e8e04 (0x7cbb bytes) 2024-09-17T21:30:38.392 INFO:teuthology.orchestra.run.smithi129.stdout:4232 read 0xc2f0859 thru 0xc2fef76 (0xe71e bytes) 2024-09-17T21:30:38.393 INFO:teuthology.orchestra.run.smithi129.stdout:4234 writesame 0x61d8c58 thru 0x61dbb87 data_size 0x4(0x2f30 bytes) 2024-09-17T21:30:38.395 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.395 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.395 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.395 INFO:teuthology.orchestra.run.smithi129.stdout:4239 write 0xb12106 thru 0xb1489f (0x279a bytes) 2024-09-17T21:30:38.396 INFO:teuthology.orchestra.run.smithi129.stdout:4240 writesame 0xbfaab3f thru 0xbfab84a data_size 0xa(0xd0c bytes) 2024-09-17T21:30:38.398 INFO:teuthology.orchestra.run.smithi129.stdout:4241 read 0x83e75e7 thru 0x83ef6f8 (0x8112 bytes) 2024-09-17T21:30:38.398 INFO:teuthology.orchestra.run.smithi129.stdout:4242 writesame 0xcecb67a thru 0xced84b6 data_size 0x1(0xce3d bytes) 2024-09-17T21:30:38.414 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:38.412+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:38.423 INFO:teuthology.orchestra.run.smithi129.stdout:4243 write 0xe32e794 thru 0xe333734 (0x4fa1 bytes) 2024-09-17T21:30:38.424 INFO:teuthology.orchestra.run.smithi129.stdout:4244 read 0xce35dab thru 0xce3d8d0 (0x7b26 bytes) 2024-09-17T21:30:38.425 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.425 INFO:teuthology.orchestra.run.smithi129.stdout:4247 write 0x3954b57 thru 0x3954c14 (0xbe bytes) 2024-09-17T21:30:38.426 INFO:teuthology.orchestra.run.smithi129.stdout:4248 write 0x6ca5c68 thru 0x6caf746 (0x9adf bytes) 2024-09-17T21:30:38.428 INFO:teuthology.orchestra.run.smithi129.stdout:4249 punch from 0xd17a332 to 0xd184ce9, (0xa9b7 bytes) 2024-09-17T21:30:38.428 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.428 INFO:teuthology.orchestra.run.smithi129.stdout:4251 punch from 0x53f1edc to 0x53fc4d9, (0xa5fd bytes) 2024-09-17T21:30:38.428 INFO:teuthology.orchestra.run.smithi129.stdout:4252 writesame 0x50d1f43 thru 0x50e0ff1 data_size 0x1(0xf0af bytes) 2024-09-17T21:30:38.456 INFO:teuthology.orchestra.run.smithi129.stdout:4253 trunc from 0xe6e1d5f to 0xabc164a 2024-09-17T21:30:38.464 INFO:teuthology.orchestra.run.smithi129.stdout:4254 read 0x410804d thru 0x410bab3 (0x3a67 bytes) 2024-09-17T21:30:38.464 INFO:teuthology.orchestra.run.smithi129.stdout:4255 read 0x89ba7fb thru 0x89c0ffd (0x6803 bytes) 2024-09-17T21:30:38.464 INFO:teuthology.orchestra.run.smithi129.stdout:4256 read 0x5a7ab6f thru 0x5a81243 (0x66d5 bytes) 2024-09-17T21:30:38.464 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.465 INFO:teuthology.orchestra.run.smithi129.stdout:4258 read 0x5d92d46 thru 0x5d9f94c (0xcc07 bytes) 2024-09-17T21:30:38.471 INFO:teuthology.orchestra.run.smithi129.stdout:4260 writesame 0xaf2acc4 thru 0xaf2cabc data_size 0x1(0x1df9 bytes) 2024-09-17T21:30:38.472 INFO:teuthology.orchestra.run.smithi129.stdout:4261 punch from 0x5d1dfb2 to 0x5d2331b, (0x5369 bytes) 2024-09-17T21:30:38.472 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.473 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.473 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.475 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:38.475 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:38.480 INFO:teuthology.orchestra.run.smithi129.stdout:4267 write 0xaf7e2bf thru 0xaf8156b (0x32ad bytes) 2024-09-17T21:30:38.481 INFO:teuthology.orchestra.run.smithi129.stdout:4268 flatten 2024-09-17T21:30:38.485 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:30:38.488 INFO:teuthology.orchestra.run.smithi129.stdout:4270 read 0x5fe284c thru 0x5fe5c88 (0x343d bytes) 2024-09-17T21:30:38.490 INFO:teuthology.orchestra.run.smithi129.stdout:4271 trunc from 0xaf8156c to 0xce2678a 2024-09-17T21:30:38.495 INFO:teuthology.orchestra.run.smithi129.stdout:4273 writesame 0x3f4954e thru 0x3f51aab data_size 0x2(0x855e bytes) 2024-09-17T21:30:38.496 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:38.493+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:38.508 INFO:teuthology.orchestra.run.smithi129.stdout:4275 trunc from 0xce2678a to 0x89cdef3 2024-09-17T21:30:38.522 INFO:teuthology.orchestra.run.smithi129.stdout:4278 trunc from 0x89cdef3 to 0xe85cd37 2024-09-17T21:30:38.529 INFO:teuthology.orchestra.run.smithi129.stdout:4279 punch from 0x7fffbbb to 0x800559e, (0x59e3 bytes) 2024-09-17T21:30:38.529 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.529 INFO:teuthology.orchestra.run.smithi129.stdout:4281 read 0x5fbc61c thru 0x5fc7b93 (0xb578 bytes) 2024-09-17T21:30:38.529 INFO:teuthology.orchestra.run.smithi129.stdout:4282 punch from 0xbab0200 to 0xbab342c, (0x322c bytes) 2024-09-17T21:30:38.529 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.529 INFO:teuthology.orchestra.run.smithi129.stdout:4284 punch from 0x5b85c64 to 0x5b868c6, (0xc62 bytes) 2024-09-17T21:30:38.529 INFO:teuthology.orchestra.run.smithi129.stdout:4286 write 0x43c8366 thru 0x43ce2c6 (0x5f61 bytes) 2024-09-17T21:30:38.540 INFO:teuthology.orchestra.run.smithi129.stdout:4287 trunc from 0xe85cd37 to 0x64cd8c5 2024-09-17T21:30:38.547 INFO:teuthology.orchestra.run.smithi129.stdout:4288 punch from 0x17d8145 to 0x17e28be, (0xa779 bytes) 2024-09-17T21:30:38.547 INFO:teuthology.orchestra.run.smithi129.stdout:4290 write 0x13cf2fd thru 0x13ddbaf (0xe8b3 bytes) 2024-09-17T21:30:38.555 INFO:teuthology.orchestra.run.smithi129.stdout:4292 write 0x6aadc3e thru 0x6ab60d2 (0x8495 bytes) 2024-09-17T21:30:38.556 INFO:teuthology.orchestra.run.smithi129.stdout:4293 write 0x2cfb7b4 thru 0x2d06771 (0xafbe bytes) 2024-09-17T21:30:38.558 INFO:teuthology.orchestra.run.smithi129.stdout:4294 punch from 0x10e3984 to 0x10ec449, (0x8ac5 bytes) 2024-09-17T21:30:38.568 INFO:teuthology.orchestra.run.smithi129.stdout:4295 write 0xaa3e845 thru 0xaa404d0 (0x1c8c bytes) 2024-09-17T21:30:38.569 INFO:teuthology.orchestra.run.smithi129.stdout:4296 read 0x839fb2 thru 0x84581d (0xb86c bytes) 2024-09-17T21:30:38.570 INFO:teuthology.orchestra.run.smithi129.stdout:4300 write 0x9449a34 thru 0x94592a8 (0xf875 bytes) 2024-09-17T21:30:38.571 INFO:teuthology.orchestra.run.smithi129.stdout:4301 read 0x5ad51bf thru 0x5ae2246 (0xd088 bytes) 2024-09-17T21:30:38.581 INFO:teuthology.orchestra.run.smithi129.stdout:4303 writesame 0xd728cc1 thru 0xd73530e data_size 0x3(0xc64e bytes) 2024-09-17T21:30:38.583 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.583 INFO:teuthology.orchestra.run.smithi129.stdout:4305 read 0x180c2c9 thru 0x1819970 (0xd6a8 bytes) 2024-09-17T21:30:38.583 INFO:teuthology.orchestra.run.smithi129.stdout:4307 write 0x9108d9a thru 0x9113426 (0xa68d bytes) 2024-09-17T21:30:38.585 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.585 INFO:teuthology.orchestra.run.smithi129.stdout:4310 writesame 0xb2b633c thru 0xb2b8f46 data_size 0x29(0x2c0b bytes) 2024-09-17T21:30:38.593 INFO:teuthology.orchestra.run.smithi129.stdout:4311 writesame 0xe7bdec0 thru 0xe7c40b7 data_size 0x3c(0x61f8 bytes) 2024-09-17T21:30:38.594 INFO:teuthology.orchestra.run.smithi129.stdout:4312 write 0x5163c51 thru 0x5168445 (0x47f5 bytes) 2024-09-17T21:30:38.596 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:38.594+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:38.612 INFO:teuthology.orchestra.run.smithi129.stdout:4313 trunc from 0xe7c40b8 to 0x135ef8f 2024-09-17T21:30:38.632 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:38.633 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:38.639 INFO:teuthology.orchestra.run.smithi129.stdout:4314 write 0x9503f46 thru 0x950421f (0x2da bytes) 2024-09-17T21:30:38.640 INFO:teuthology.orchestra.run.smithi129.stdout:4315 write 0x7893ac3 thru 0x789ccdd (0x921b bytes) 2024-09-17T21:30:38.641 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:30:38.642 INFO:teuthology.orchestra.run.smithi129.stdout:4316 read 0x1ba9b46 thru 0x1baaec5 (0x1380 bytes) 2024-09-17T21:30:38.642 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.642 INFO:teuthology.orchestra.run.smithi129.stdout:4318 read 0x91b7af1 thru 0x91b805e (0x56e bytes) 2024-09-17T21:30:38.642 INFO:teuthology.orchestra.run.smithi129.stdout:4319 write 0x2c25ed6 thru 0x2c28862 (0x298d bytes) 2024-09-17T21:30:38.645 INFO:teuthology.orchestra.run.smithi129.stdout:4320 write 0x393394 thru 0x39c7a1 (0x940e bytes) 2024-09-17T21:30:38.647 INFO:teuthology.orchestra.run.smithi129.stdout:4321 write 0x8eca124 thru 0x8ed35b3 (0x9490 bytes) 2024-09-17T21:30:38.657 INFO:teuthology.orchestra.run.smithi129.stdout:4323 writesame 0xaea8e09 thru 0xaeaa7b8 data_size 0x18(0x19b0 bytes) 2024-09-17T21:30:38.658 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.669 INFO:teuthology.orchestra.run.smithi129.stdout:4326 trunc from 0xaeaa7b9 to 0x125b460 2024-09-17T21:30:38.680 INFO:teuthology.orchestra.run.smithi129.stdout:4327 read 0x11f59c3 thru 0x11f75da (0x1c18 bytes) 2024-09-17T21:30:38.684 INFO:teuthology.orchestra.run.smithi129.stdout:4328 trunc from 0x125b460 to 0x19af42b 2024-09-17T21:30:38.686 INFO:teuthology.orchestra.run.smithi129.stdout:4329 punch from 0x867c3c to 0x86bc6d, (0x4031 bytes) 2024-09-17T21:30:38.686 INFO:teuthology.orchestra.run.smithi129.stdout:4330 read 0x151bd6 thru 0x155bae (0x3fd9 bytes) 2024-09-17T21:30:38.686 INFO:teuthology.orchestra.run.smithi129.stdout:4331 read 0x607b1e thru 0x60a016 (0x24f9 bytes) 2024-09-17T21:30:38.694 INFO:teuthology.orchestra.run.smithi129.stdout:4333 write 0x338677b thru 0x338d9eb (0x7271 bytes) 2024-09-17T21:30:38.696 INFO:teuthology.orchestra.run.smithi129.stdout:4334 read 0x1d1a0e thru 0x1dc24e (0xa841 bytes) 2024-09-17T21:30:38.696 INFO:teuthology.orchestra.run.smithi129.stdout:4335 read 0x14505f1 thru 0x145c8f2 (0xc302 bytes) 2024-09-17T21:30:38.697 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.697 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:38.695+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:38.711 INFO:teuthology.orchestra.run.smithi129.stdout:4339 write 0xb0f070f thru 0xb0f983d (0x912f bytes) 2024-09-17T21:30:38.712 INFO:teuthology.orchestra.run.smithi129.stdout:4340 punch from 0x3966e95 to 0x39671af, (0x31a bytes) 2024-09-17T21:30:38.712 INFO:teuthology.orchestra.run.smithi129.stdout:4341 writesame 0x42b75ae thru 0x42c2d80 data_size 0x1(0xb7d3 bytes) 2024-09-17T21:30:38.717 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:30:38.735 INFO:teuthology.orchestra.run.smithi129.stdout:4342 read 0x71c335b thru 0x71cff31 (0xcbd7 bytes) 2024-09-17T21:30:38.735 INFO:teuthology.orchestra.run.smithi129.stdout:4343 read 0x143633a thru 0x1439b23 (0x37ea bytes) 2024-09-17T21:30:38.735 INFO:teuthology.orchestra.run.smithi129.stdout:4344 writesame 0xf59d42 thru 0xf60815 data_size 0x81(0x6ad4 bytes) 2024-09-17T21:30:38.737 INFO:teuthology.orchestra.run.smithi129.stdout:4346 trunc from 0xb0f983e to 0xb2082d2 2024-09-17T21:30:38.743 INFO:teuthology.orchestra.run.smithi129.stdout:4347 read 0xac0cf13 thru 0xac1013e (0x322c bytes) 2024-09-17T21:30:38.743 INFO:teuthology.orchestra.run.smithi129.stdout:4348 write 0xe29913 thru 0xe31409 (0x7af7 bytes) 2024-09-17T21:30:38.749 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:38.749 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:38.756 INFO:teuthology.orchestra.run.smithi129.stdout:4350 write 0xec7a8cb thru 0xec885b8 (0xdcee bytes) 2024-09-17T21:30:38.758 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:30:38.759 INFO:teuthology.orchestra.run.smithi129.stdout:4352 punch from 0x1128439 to 0x113563f, (0xd206 bytes) 2024-09-17T21:30:38.759 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.759 INFO:teuthology.orchestra.run.smithi129.stdout:4354 punch from 0x1e92038 to 0x1e9464f, (0x2617 bytes) 2024-09-17T21:30:38.759 INFO:teuthology.orchestra.run.smithi129.stdout:4355 read 0x17ffc56 thru 0x1803485 (0x3830 bytes) 2024-09-17T21:30:38.759 INFO:teuthology.orchestra.run.smithi129.stdout:4356 punch from 0x333f261 to 0x334c759, (0xd4f8 bytes) 2024-09-17T21:30:38.770 INFO:teuthology.orchestra.run.smithi129.stdout:4358 trunc from 0xec885b9 to 0x5c09ea4 2024-09-17T21:30:38.785 INFO:teuthology.orchestra.run.smithi129.stdout:4359 writesame 0x6dff63e thru 0x6e0e66d data_size 0xb7(0xf030 bytes) 2024-09-17T21:30:38.786 INFO:teuthology.orchestra.run.smithi129.stdout:4360 read 0x1ad7f65 thru 0x1ae6769 (0xe805 bytes) 2024-09-17T21:30:38.787 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.787 INFO:teuthology.orchestra.run.smithi129.stdout:4363 write 0x347c933 thru 0x348a767 (0xde35 bytes) 2024-09-17T21:30:38.789 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.789 INFO:teuthology.orchestra.run.smithi129.stdout:4366 write 0x5644ac3 thru 0x5650307 (0xb845 bytes) 2024-09-17T21:30:38.791 INFO:teuthology.orchestra.run.smithi129.stdout:4368 punch from 0x3e0a9fb to 0x3e190ee, (0xe6f3 bytes) 2024-09-17T21:30:38.792 INFO:teuthology.orchestra.run.smithi129.stdout:4369 trunc from 0x6e0e66e to 0x5fef2b0 2024-09-17T21:30:38.799 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:38.797+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:38.801 INFO:teuthology.orchestra.run.smithi129.stdout:4370 trunc from 0x5fef2b0 to 0x6f0a7f4 2024-09-17T21:30:38.807 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:38.807 INFO:teuthology.orchestra.run.smithi129.stdout:4373 write 0x6512757 thru 0x6521dc5 (0xf66f bytes) 2024-09-17T21:30:38.824 INFO:teuthology.orchestra.run.smithi129.stdout:4374 write 0xee4abcb thru 0xee4e694 (0x3aca bytes) 2024-09-17T21:30:38.852 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:38.852 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:38.868 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:30:38.869 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:30:38.884 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:38.885 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:38.899 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:38.897+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:38.900 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:38.900 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:38.901 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:38.901 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:38.910 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:30:38.917 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:38.917 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:38.933 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:38.933 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:38.950 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:38.950 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:38.999 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:38.997+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:39.001 INFO:teuthology.orchestra.run.smithi129.stdout:4376 write 0x7999c30 thru 0x79a5d2e (0xc0ff bytes) 2024-09-17T21:30:39.003 INFO:teuthology.orchestra.run.smithi129.stdout:4378 punch from 0x41fb6b to 0x426503, (0x6998 bytes) 2024-09-17T21:30:39.003 INFO:teuthology.orchestra.run.smithi129.stdout:4380 read 0x25d34ab thru 0x25e0365 (0xcebb bytes) 2024-09-17T21:30:39.003 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:39.003 INFO:teuthology.orchestra.run.smithi129.stdout:4383 write 0x3e53e23 thru 0x3e5e2e4 (0xa4c2 bytes) 2024-09-17T21:30:39.014 INFO:teuthology.orchestra.run.smithi129.stdout:4384 trunc from 0xee4e695 to 0x7584e2d 2024-09-17T21:30:39.017 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:39.017 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:39.022 INFO:teuthology.orchestra.run.smithi129.stdout:4385 punch from 0x12b3e3b to 0x12b457c, (0x741 bytes) 2024-09-17T21:30:39.022 INFO:teuthology.orchestra.run.smithi129.stdout:4387 punch from 0x5b3373b to 0x5b356b1, (0x1f76 bytes) 2024-09-17T21:30:39.022 INFO:teuthology.orchestra.run.smithi129.stdout:4388 punch from 0x2773450 to 0x277de4e, (0xa9fe bytes) 2024-09-17T21:30:39.023 INFO:teuthology.orchestra.run.smithi129.stdout:4389 write 0x1366073 thru 0x136c20b (0x6199 bytes) 2024-09-17T21:30:39.026 INFO:teuthology.orchestra.run.smithi129.stdout:4390 trunc from 0x7584e2d to 0x57aa7be 2024-09-17T21:30:39.035 INFO:teuthology.orchestra.run.smithi129.stdout:4393 punch from 0x10f43c6 to 0x10fcc30, (0x886a bytes) 2024-09-17T21:30:39.035 INFO:teuthology.orchestra.run.smithi129.stdout:4394 write 0x43bec28 thru 0x43c8fd8 (0xa3b1 bytes) 2024-09-17T21:30:39.036 INFO:teuthology.orchestra.run.smithi129.stdout:4395 punch from 0x5501162 to 0x5501a05, (0x8a3 bytes) 2024-09-17T21:30:39.036 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:39.036 INFO:teuthology.orchestra.run.smithi129.stdout:4397 writesame 0xf15a48 thru 0xf18772 data_size 0x1(0x2d2b bytes) 2024-09-17T21:30:39.042 INFO:teuthology.orchestra.run.smithi129.stdout:4398 read 0x22b2840 thru 0x22c00af (0xd870 bytes) 2024-09-17T21:30:39.043 INFO:teuthology.orchestra.run.smithi129.stdout:4399 write 0x447a100 thru 0x4489381 (0xf282 bytes) 2024-09-17T21:30:39.059 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:36935 3.a 3:54c8759b:::rbd_data.1120e62eea89.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-17T21:30:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.097787597, 2024-09-17T21:30:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020540151999999999, 2024-09-17T21:30:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 36935 2024-09-17T21:30:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958910+0000", 2024-09-17T21:30:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:30:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958918+0000", 2024-09-17T21:30:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6469999999999998e-06 2024-09-17T21:30:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958933+0000", 2024-09-17T21:30:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4791e-05 2024-09-17T21:30:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958940+0000", 2024-09-17T21:30:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-17T21:30:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958990+0000", 2024-09-17T21:30:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9759000000000003e-05 2024-09-17T21:30:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.959039+0000", 2024-09-17T21:30:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9487000000000002e-05 2024-09-17T21:30:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.962098+0000", 2024-09-17T21:30:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003059258 2024-09-17T21:30:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979325+0000", 2024-09-17T21:30:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017226841 2024-09-17T21:30:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979343+0000", 2024-09-17T21:30:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7486000000000002e-05 2024-09-17T21:30:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979401+0000", 2024-09-17T21:30:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7980999999999999e-05 2024-09-17T21:30:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979449+0000", 2024-09-17T21:30:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8639e-05 2024-09-17T21:30:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39447 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 909312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088849697000001, 2024-09-17T21:30:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017419153999999999, 2024-09-17T21:30:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39447 2024-09-17T21:30:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:30:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967849+0000", 2024-09-17T21:30:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4339999999999999e-06 2024-09-17T21:30:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967853+0000", 2024-09-17T21:30:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:30:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967857+0000", 2024-09-17T21:30:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:30:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985193+0000", 2024-09-17T21:30:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017335973000000001 2024-09-17T21:30:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985228+0000", 2024-09-17T21:30:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5154999999999998e-05 2024-09-17T21:30:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985266+0000", 2024-09-17T21:30:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8448000000000002e-05 2024-09-17T21:30:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39448 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 974848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088836880999999, 2024-09-17T21:30:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017517066000000001, 2024-09-17T21:30:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39448 2024-09-17T21:30:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.72e-07 2024-09-17T21:30:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967861+0000", 2024-09-17T21:30:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.096e-06 2024-09-17T21:30:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967865+0000", 2024-09-17T21:30:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9339999999999999e-06 2024-09-17T21:30:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967868+0000", 2024-09-17T21:30:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.508e-06 2024-09-17T21:30:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985285+0000", 2024-09-17T21:30:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017417208 2024-09-17T21:30:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985339+0000", 2024-09-17T21:30:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3931000000000003e-05 2024-09-17T21:30:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985377+0000", 2024-09-17T21:30:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8016999999999999e-05 2024-09-17T21:30:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39449 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088824770999999, 2024-09-17T21:30:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017595137, 2024-09-17T21:30:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39449 2024-09-17T21:30:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2700000000000001e-07 2024-09-17T21:30:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967873+0000", 2024-09-17T21:30:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.164e-06 2024-09-17T21:30:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967877+0000", 2024-09-17T21:30:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.23e-06 2024-09-17T21:30:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967880+0000", 2024-09-17T21:30:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.286e-06 2024-09-17T21:30:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985397+0000", 2024-09-17T21:30:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017516982 2024-09-17T21:30:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985431+0000", 2024-09-17T21:30:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3899999999999997e-05 2024-09-17T21:30:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985467+0000", 2024-09-17T21:30:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6347999999999998e-05 2024-09-17T21:30:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39450 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1105920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088804514000003, 2024-09-17T21:30:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017677824000000002, 2024-09-17T21:30:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39450 2024-09-17T21:30:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967893+0000", 2024-09-17T21:30:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9100000000000004e-07 2024-09-17T21:30:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967895+0000", 2024-09-17T21:30:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:30:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967901+0000", 2024-09-17T21:30:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.083e-06 2024-09-17T21:30:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967906+0000", 2024-09-17T21:30:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:30:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985493+0000", 2024-09-17T21:30:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017587565999999999 2024-09-17T21:30:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985531+0000", 2024-09-17T21:30:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7490000000000002e-05 2024-09-17T21:30:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985570+0000", 2024-09-17T21:30:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9205000000000002e-05 2024-09-17T21:30:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39451 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1171456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088784134000001, 2024-09-17T21:30:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017752337999999999, 2024-09-17T21:30:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39451 2024-09-17T21:30:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:30:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967915+0000", 2024-09-17T21:30:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780000000000001e-06 2024-09-17T21:30:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967922+0000", 2024-09-17T21:30:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7020000000000004e-06 2024-09-17T21:30:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967926+0000", 2024-09-17T21:30:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.809e-06 2024-09-17T21:30:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985590+0000", 2024-09-17T21:30:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017663913999999999 2024-09-17T21:30:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985627+0000", 2024-09-17T21:30:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6896e-05 2024-09-17T21:30:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985665+0000", 2024-09-17T21:30:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7793999999999998e-05 2024-09-17T21:30:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39452 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1236992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088764191999999, 2024-09-17T21:30:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017829602999999999, 2024-09-17T21:30:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39452 2024-09-17T21:30:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:30:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967947+0000", 2024-09-17T21:30:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4280999999999999e-05 2024-09-17T21:30:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967954+0000", 2024-09-17T21:30:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3249999999999996e-06 2024-09-17T21:30:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967959+0000", 2024-09-17T21:30:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2299999999999999e-06 2024-09-17T21:30:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985687+0000", 2024-09-17T21:30:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017727844999999999 2024-09-17T21:30:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985725+0000", 2024-09-17T21:30:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8243e-05 2024-09-17T21:30:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985762+0000", 2024-09-17T21:30:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7023000000000001e-05 2024-09-17T21:30:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39453 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088727704, 2024-09-17T21:30:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017893793000000002, 2024-09-17T21:30:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39453 2024-09-17T21:30:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967970+0000", 2024-09-17T21:30:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:30:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967972+0000", 2024-09-17T21:30:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0229999999999999e-06 2024-09-17T21:30:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967978+0000", 2024-09-17T21:30:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0430000000000004e-06 2024-09-17T21:30:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967982+0000", 2024-09-17T21:30:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2570000000000001e-06 2024-09-17T21:30:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985782+0000", 2024-09-17T21:30:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017799683 2024-09-17T21:30:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985818+0000", 2024-09-17T21:30:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6304999999999999e-05 2024-09-17T21:30:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985863+0000", 2024-09-17T21:30:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4953999999999997e-05 2024-09-17T21:30:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39454 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1368064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088708376, 2024-09-17T21:30:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017968886, 2024-09-17T21:30:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39454 2024-09-17T21:30:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967989+0000", 2024-09-17T21:30:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.15e-07 2024-09-17T21:30:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967991+0000", 2024-09-17T21:30:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.945e-06 2024-09-17T21:30:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967997+0000", 2024-09-17T21:30:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0190000000000003e-06 2024-09-17T21:30:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968001+0000", 2024-09-17T21:30:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6650000000000002e-06 2024-09-17T21:30:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985883+0000", 2024-09-17T21:30:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017881971999999999 2024-09-17T21:30:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985920+0000", 2024-09-17T21:30:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-05 2024-09-17T21:30:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985957+0000", 2024-09-17T21:30:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7370000000000003e-05 2024-09-17T21:30:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39455 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088689201999998, 2024-09-17T21:30:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018044437999999999, 2024-09-17T21:30:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39455 2024-09-17T21:30:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:30:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968010+0000", 2024-09-17T21:30:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.886e-06 2024-09-17T21:30:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968017+0000", 2024-09-17T21:30:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:30:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968021+0000", 2024-09-17T21:30:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850000000000004e-06 2024-09-17T21:30:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985977+0000", 2024-09-17T21:30:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017956368 2024-09-17T21:30:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986015+0000", 2024-09-17T21:30:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7610000000000001e-05 2024-09-17T21:30:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986052+0000", 2024-09-17T21:30:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7109e-05 2024-09-17T21:30:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39456 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1499136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088668902000002, 2024-09-17T21:30:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018114514000000002, 2024-09-17T21:30:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39456 2024-09-17T21:30:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-17T21:30:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968030+0000", 2024-09-17T21:30:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0049999999999999e-06 2024-09-17T21:30:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968036+0000", 2024-09-17T21:30:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0129999999999999e-06 2024-09-17T21:30:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968041+0000", 2024-09-17T21:30:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8300000000000003e-06 2024-09-17T21:30:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986071+0000", 2024-09-17T21:30:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018029620999999999 2024-09-17T21:30:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986106+0000", 2024-09-17T21:30:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4867999999999999e-05 2024-09-17T21:30:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986142+0000", 2024-09-17T21:30:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6566000000000002e-05 2024-09-17T21:30:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39457 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1564672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088649085999997, 2024-09-17T21:30:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018188644, 2024-09-17T21:30:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39457 2024-09-17T21:30:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7199999999999999e-07 2024-09-17T21:30:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968050+0000", 2024-09-17T21:30:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1110000000000002e-06 2024-09-17T21:30:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968057+0000", 2024-09-17T21:30:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6170000000000003e-06 2024-09-17T21:30:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968062+0000", 2024-09-17T21:30:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:30:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986163+0000", 2024-09-17T21:30:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018101435999999999 2024-09-17T21:30:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986199+0000", 2024-09-17T21:30:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.591e-05 2024-09-17T21:30:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986236+0000", 2024-09-17T21:30:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7416999999999998e-05 2024-09-17T21:30:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39458 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1630208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088622145999999, 2024-09-17T21:30:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018255856000000001, 2024-09-17T21:30:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39458 2024-09-17T21:30:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:30:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968077+0000", 2024-09-17T21:30:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.779e-06 2024-09-17T21:30:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968081+0000", 2024-09-17T21:30:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.532e-06 2024-09-17T21:30:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968085+0000", 2024-09-17T21:30:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999999e-06 2024-09-17T21:30:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986256+0000", 2024-09-17T21:30:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018171273000000002 2024-09-17T21:30:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986293+0000", 2024-09-17T21:30:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6534e-05 2024-09-17T21:30:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986330+0000", 2024-09-17T21:30:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7536000000000001e-05 2024-09-17T21:30:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39459 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1695744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088607568, 2024-09-17T21:30:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334493, 2024-09-17T21:30:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39459 2024-09-17T21:30:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968090+0000", 2024-09-17T21:30:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:30:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968091+0000", 2024-09-17T21:30:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3319999999999999e-06 2024-09-17T21:30:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968094+0000", 2024-09-17T21:30:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2590000000000001e-06 2024-09-17T21:30:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968097+0000", 2024-09-17T21:30:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:30:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986350+0000", 2024-09-17T21:30:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018253364000000001 2024-09-17T21:30:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986387+0000", 2024-09-17T21:30:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6350999999999998e-05 2024-09-17T21:30:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986424+0000", 2024-09-17T21:30:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6897999999999998e-05 2024-09-17T21:30:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39460 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1761280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088596432000003, 2024-09-17T21:30:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018415918999999999, 2024-09-17T21:30:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39460 2024-09-17T21:30:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968101+0000", 2024-09-17T21:30:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T21:30:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968102+0000", 2024-09-17T21:30:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1689999999999999e-06 2024-09-17T21:30:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968105+0000", 2024-09-17T21:30:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:30:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968108+0000", 2024-09-17T21:30:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6630000000000002e-06 2024-09-17T21:30:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986442+0000", 2024-09-17T21:30:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334487 2024-09-17T21:30:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986479+0000", 2024-09-17T21:30:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6557000000000003e-05 2024-09-17T21:30:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986516+0000", 2024-09-17T21:30:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7648999999999998e-05 2024-09-17T21:30:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39461 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1826816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088585672000001, 2024-09-17T21:30:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018499768, 2024-09-17T21:30:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39461 2024-09-17T21:30:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4699999999999998e-07 2024-09-17T21:30:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968112+0000", 2024-09-17T21:30:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-06 2024-09-17T21:30:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968116+0000", 2024-09-17T21:30:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1080000000000001e-06 2024-09-17T21:30:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968122+0000", 2024-09-17T21:30:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1070000000000001e-06 2024-09-17T21:30:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986537+0000", 2024-09-17T21:30:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018414834000000001 2024-09-17T21:30:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986572+0000", 2024-09-17T21:30:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5726000000000003e-05 2024-09-17T21:30:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986611+0000", 2024-09-17T21:30:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8584000000000002e-05 2024-09-17T21:30:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39462 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1892352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088571307999999, 2024-09-17T21:30:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018583252000000001, 2024-09-17T21:30:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39462 2024-09-17T21:30:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968126+0000", 2024-09-17T21:30:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968127+0000", 2024-09-17T21:30:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.229e-06 2024-09-17T21:30:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968131+0000", 2024-09-17T21:30:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8010000000000001e-06 2024-09-17T21:30:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968134+0000", 2024-09-17T21:30:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8880000000000001e-06 2024-09-17T21:30:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986634+0000", 2024-09-17T21:30:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018500501999999999 2024-09-17T21:30:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986671+0000", 2024-09-17T21:30:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6881000000000001e-05 2024-09-17T21:30:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986709+0000", 2024-09-17T21:30:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7546999999999998e-05 2024-09-17T21:30:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39463 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1957888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088559693000001, 2024-09-17T21:30:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018667186999999998, 2024-09-17T21:30:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39463 2024-09-17T21:30:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:30:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968138+0000", 2024-09-17T21:30:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:30:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968142+0000", 2024-09-17T21:30:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.145e-06 2024-09-17T21:30:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968144+0000", 2024-09-17T21:30:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6869999999999999e-06 2024-09-17T21:30:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986728+0000", 2024-09-17T21:30:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018584163000000001 2024-09-17T21:30:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986766+0000", 2024-09-17T21:30:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7987000000000001e-05 2024-09-17T21:30:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986804+0000", 2024-09-17T21:30:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7892000000000002e-05 2024-09-17T21:30:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39464 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088549383999997, 2024-09-17T21:30:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018753723999999999, 2024-09-17T21:30:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39464 2024-09-17T21:30:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968148+0000", 2024-09-17T21:30:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:30:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968152+0000", 2024-09-17T21:30:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3749999999999996e-06 2024-09-17T21:30:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968155+0000", 2024-09-17T21:30:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.371e-06 2024-09-17T21:30:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968158+0000", 2024-09-17T21:30:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T21:30:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986824+0000", 2024-09-17T21:30:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018665779 2024-09-17T21:30:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986863+0000", 2024-09-17T21:30:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9273000000000002e-05 2024-09-17T21:30:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986901+0000", 2024-09-17T21:30:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7978000000000001e-05 2024-09-17T21:30:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39465 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.088533984999998, 2024-09-17T21:30:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018825341999999998, 2024-09-17T21:30:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39465 2024-09-17T21:30:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-17T21:30:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968164+0000", 2024-09-17T21:30:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1289999999999999e-06 2024-09-17T21:30:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968167+0000", 2024-09-17T21:30:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1379999999999999e-06 2024-09-17T21:30:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968170+0000", 2024-09-17T21:30:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:30:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986914+0000", 2024-09-17T21:30:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018743829 2024-09-17T21:30:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986950+0000", 2024-09-17T21:30:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5819999999999999e-05 2024-09-17T21:30:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986988+0000", 2024-09-17T21:30:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8362000000000003e-05 2024-09-17T21:30:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.149 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.149 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.149 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:39.151 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:30:39.151 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:39.148+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:39.153 INFO:teuthology.orchestra.run.smithi129.stdout:4400 write 0xea8efde thru 0xea9eb36 (0xfb59 bytes) 2024-09-17T21:30:39.153 INFO:teuthology.orchestra.run.smithi129.stdout:4401 punch from 0x1369be0 to 0x136c39d, (0x27bd bytes) 2024-09-17T21:30:39.153 INFO:teuthology.orchestra.run.smithi129.stdout:4402 punch from 0xd954cfd to 0xd963c2c, (0xef2f bytes) 2024-09-17T21:30:39.153 INFO:teuthology.orchestra.run.smithi129.stdout:4403 read 0x116090b thru 0x116175a (0xe50 bytes) 2024-09-17T21:30:39.153 INFO:teuthology.orchestra.run.smithi129.stdout:4404 read 0xab281dd thru 0xab2ecfe (0x6b22 bytes) 2024-09-17T21:30:39.153 INFO:teuthology.orchestra.run.smithi129.stdout:4405 writesame 0x7ac8f02 thru 0x7ace4c3 data_size 0x3(0x55c2 bytes) 2024-09-17T21:30:39.153 INFO:teuthology.orchestra.run.smithi129.stdout:4406 writesame 0x342d5d4 thru 0x3430c00 data_size 0x17(0x362d bytes) 2024-09-17T21:30:39.154 INFO:teuthology.orchestra.run.smithi129.stdout:4407 write 0x68174c9 thru 0x681a01f (0x2b57 bytes) 2024-09-17T21:30:39.154 INFO:teuthology.orchestra.run.smithi129.stdout:4409 punch from 0x977f6a to 0x98645b, (0xe4f1 bytes) 2024-09-17T21:30:39.154 INFO:teuthology.orchestra.run.smithi129.stdout:4410 trunc from 0xea9eb37 to 0x97b1db7 2024-09-17T21:30:39.154 INFO:teuthology.orchestra.run.smithi129.stdout:4411 write 0xc751a60 thru 0xc75ea22 (0xcfc3 bytes) 2024-09-17T21:30:39.154 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:39.154 INFO:teuthology.orchestra.run.smithi129.stdout:4415 write 0x7b1ea9f thru 0x7b2dd90 (0xf2f2 bytes) 2024-09-17T21:30:39.154 INFO:teuthology.orchestra.run.smithi129.stdout:4417 write 0x89a6ce7 thru 0x89a9fa8 (0x32c2 bytes) 2024-09-17T21:30:39.154 INFO:teuthology.orchestra.run.smithi129.stdout:4419 punch from 0x51af512 to 0x51b901c, (0x9b0a bytes) 2024-09-17T21:30:39.154 INFO:teuthology.orchestra.run.smithi129.stdout:4422 writesame 0x6d40ac8 thru 0x6d4b360 data_size 0x1(0xa899 bytes) 2024-09-17T21:30:39.154 INFO:teuthology.orchestra.run.smithi129.stdout:4424 write 0xe7e98cc thru 0xe7ef7a2 (0x5ed7 bytes) 2024-09-17T21:30:39.154 INFO:teuthology.orchestra.run.smithi129.stdout:4426 write 0x70b8cd3 thru 0x70c734c (0xe67a bytes) 2024-09-17T21:30:39.154 INFO:teuthology.orchestra.run.smithi129.stdout:4427 write 0x3c222a3 thru 0x3c2f3b8 (0xd116 bytes) 2024-09-17T21:30:39.154 INFO:teuthology.orchestra.run.smithi129.stdout:4429 read 0xe5dc340 thru 0xe5dfeeb (0x3bac bytes) 2024-09-17T21:30:39.155 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:39.155 INFO:teuthology.orchestra.run.smithi129.stdout:4431 punch from 0xc3d4e to 0xcbcb0, (0x7f62 bytes) 2024-09-17T21:30:39.155 INFO:teuthology.orchestra.run.smithi129.stdout:4433 write 0x9e66586 thru 0x9e667f3 (0x26e bytes) 2024-09-17T21:30:39.155 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:39.155 INFO:teuthology.orchestra.run.smithi129.stdout:4438 writesame 0xb40bc57 thru 0xb41247f data_size 0x1(0x6829 bytes) 2024-09-17T21:30:39.155 INFO:teuthology.orchestra.run.smithi129.stdout:4439 writesame 0x4291a85 thru 0x42a0bd3 data_size 0xaf(0xf14f bytes) 2024-09-17T21:30:39.155 INFO:teuthology.orchestra.run.smithi129.stdout:4440 write 0x4f10e6 thru 0x4f13d9 (0x2f4 bytes) 2024-09-17T21:30:39.155 INFO:teuthology.orchestra.run.smithi129.stdout:4441 read 0xc94b80a thru 0xc9564ff (0xacf6 bytes) 2024-09-17T21:30:39.164 INFO:teuthology.orchestra.run.smithi129.stdout:4442 trunc from 0xe7ef7a3 to 0x2b27e75 2024-09-17T21:30:39.172 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:39.189 INFO:teuthology.orchestra.run.smithi129.stdout:4444 write 0xc2fc1d8 thru 0xc3068a3 (0xa6cc bytes) 2024-09-17T21:30:39.198 INFO:teuthology.orchestra.run.smithi129.stdout:4445 trunc from 0xc3068a4 to 0x5736e14 2024-09-17T21:30:39.206 INFO:teuthology.orchestra.run.smithi129.stdout:4446 read 0x454808e thru 0x455629a (0xe20d bytes) 2024-09-17T21:30:39.206 INFO:teuthology.orchestra.run.smithi129.stdout:4447 read 0x1247937 thru 0x12525cc (0xac96 bytes) 2024-09-17T21:30:39.211 INFO:teuthology.orchestra.run.smithi129.stdout:4448 trunc from 0x5736e14 to 0xef61d3 2024-09-17T21:30:39.225 INFO:teuthology.orchestra.run.smithi129.stdout:4450 punch from 0x608619 to 0x60decf, (0x58b6 bytes) 2024-09-17T21:30:39.233 INFO:teuthology.orchestra.run.smithi129.stdout:4452 trunc from 0xef61d3 to 0x6e8b05a 2024-09-17T21:30:39.238 INFO:teuthology.orchestra.run.smithi129.stdout:4453 read 0x1391cd6 thru 0x1394e45 (0x3170 bytes) 2024-09-17T21:30:39.248 INFO:teuthology.orchestra.run.smithi129.stdout:4454 writesame 0xb251b1a thru 0xb253389 data_size 0x2e(0x1870 bytes) 2024-09-17T21:30:39.251 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:39.249+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:39.257 INFO:teuthology.orchestra.run.smithi129.stdout:4455 write 0xe1f81b0 thru 0xe1fa536 (0x2387 bytes) 2024-09-17T21:30:39.259 INFO:teuthology.orchestra.run.smithi129.stdout:4456 punch from 0xbdfe6f7 to 0xbe0156f, (0x2e78 bytes) 2024-09-17T21:30:39.259 INFO:teuthology.orchestra.run.smithi129.stdout:4457 write 0x8f1167d thru 0x8f12697 (0x101b bytes) 2024-09-17T21:30:39.261 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:39.262 INFO:teuthology.orchestra.run.smithi129.stdout:4459 writesame 0x137e00d thru 0x138ce0a data_size 0x29(0xedfe bytes) 2024-09-17T21:30:39.265 INFO:teuthology.orchestra.run.smithi129.stdout:4460 read 0xe19939d thru 0xe19fd62 (0x69c6 bytes) 2024-09-17T21:30:39.265 INFO:teuthology.orchestra.run.smithi129.stdout:4461 writesame 0xa434a62 thru 0xa43eef4 data_size 0x1(0xa493 bytes) 2024-09-17T21:30:39.297 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:39.297 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:39.297 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:39.297 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:39.302 INFO:teuthology.orchestra.run.smithi129.stdout:4464 trunc from 0xe1fa537 to 0x10de77c 2024-09-17T21:30:39.306 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:30:39.322 INFO:teuthology.orchestra.run.smithi129.stdout:4465 clone 31 order 24 su 2097152 sc 9 2024-09-17T21:30:39.352 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:39.349+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:39.409 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:39.409 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:39.409 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:39.409 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:39.410 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:39.418 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:30:39.462 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:39.460+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:39.522 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:39.522 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:39.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:39.523 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:39.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.523 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:30:39.523 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:39.523 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 71.579759022000005, 2024-09-17T21:30:39.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61572000699999996, 2024-09-17T21:30:39.523 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.523 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.523 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:39.523 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.523 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:30:39.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:30:39.524 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T21:30:39.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.524 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:39.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:39.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939745+0000", 2024-09-17T21:30:39.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6899999999999999e-06 2024-09-17T21:30:39.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939753+0000", 2024-09-17T21:30:39.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1950000000000003e-06 2024-09-17T21:30:39.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939767+0000", 2024-09-17T21:30:39.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3429e-05 2024-09-17T21:30:39.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939776+0000", 2024-09-17T21:30:39.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:30:39.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939858+0000", 2024-09-17T21:30:39.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2496000000000004e-05 2024-09-17T21:30:39.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T21:30:39.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939868+0000", 2024-09-17T21:30:39.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1910000000000006e-06 2024-09-17T21:30:39.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940612+0000", 2024-09-17T21:30:39.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074429599999999998 2024-09-17T21:30:39.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:30:39.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940621+0000", 2024-09-17T21:30:39.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9139999999999997e-06 2024-09-17T21:30:39.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554006+0000", 2024-09-17T21:30:39.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61338505099999996 2024-09-17T21:30:39.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554166+0000", 2024-09-17T21:30:39.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015982600000000001 2024-09-17T21:30:39.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:30:39.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554296+0000", 2024-09-17T21:30:39.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013002399999999999 2024-09-17T21:30:39.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:39.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554936+0000", 2024-09-17T21:30:39.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064009400000000001 2024-09-17T21:30:39.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:39.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555389+0000", 2024-09-17T21:30:39.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045355200000000002 2024-09-17T21:30:39.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:39.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555432+0000", 2024-09-17T21:30:39.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2477e-05 2024-09-17T21:30:39.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555463+0000", 2024-09-17T21:30:39.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1612000000000002e-05 2024-09-17T21:30:39.530 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.530 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.530 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.531 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17527 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1005568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:39.531 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:39.531 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.114531190999998, 2024-09-17T21:30:39.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046452090000000001, 2024-09-17T21:30:39.531 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.531 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.531 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.531 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17527 2024-09-17T21:30:39.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.531 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:39.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:39.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405496+0000", 2024-09-17T21:30:39.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052498400000000004 2024-09-17T21:30:39.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405500+0000", 2024-09-17T21:30:39.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8410000000000002e-06 2024-09-17T21:30:39.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405511+0000", 2024-09-17T21:30:39.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0749e-05 2024-09-17T21:30:39.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405517+0000", 2024-09-17T21:30:39.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:30:39.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451275+0000", 2024-09-17T21:30:39.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045758134999999998 2024-09-17T21:30:39.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451355+0000", 2024-09-17T21:30:39.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9453e-05 2024-09-17T21:30:39.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451423+0000", 2024-09-17T21:30:39.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8782999999999995e-05 2024-09-17T21:30:39.535 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.535 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.535 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.535 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17615 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:39.535 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:39.535 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.109290702999999, 2024-09-17T21:30:39.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046741309000000002, 2024-09-17T21:30:39.535 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.535 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.535 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.536 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17615 2024-09-17T21:30:39.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.536 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:39.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:39.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:39.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:30:39.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410213+0000", 2024-09-17T21:30:39.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:30:39.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410217+0000", 2024-09-17T21:30:39.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8199999999999998e-06 2024-09-17T21:30:39.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410220+0000", 2024-09-17T21:30:39.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6900000000000001e-06 2024-09-17T21:30:39.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455264+0000", 2024-09-17T21:30:39.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045044002999999999 2024-09-17T21:30:39.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455338+0000", 2024-09-17T21:30:39.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4182000000000002e-05 2024-09-17T21:30:39.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.456953+0000", 2024-09-17T21:30:39.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016150089999999999 2024-09-17T21:30:39.539 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.539 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.540 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.540 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17616 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 872448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:39.540 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:39.540 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.109278650999997, 2024-09-17T21:30:39.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946039000000002, 2024-09-17T21:30:39.540 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.540 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.540 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.540 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.540 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.540 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.541 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17616 2024-09-17T21:30:39.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.541 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:39.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:39.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:39.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T21:30:39.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410225+0000", 2024-09-17T21:30:39.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3459999999999999e-06 2024-09-17T21:30:39.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410228+0000", 2024-09-17T21:30:39.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.083e-06 2024-09-17T21:30:39.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410232+0000", 2024-09-17T21:30:39.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4419999999999998e-06 2024-09-17T21:30:39.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457004+0000", 2024-09-17T21:30:39.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046771873999999998 2024-09-17T21:30:39.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457084+0000", 2024-09-17T21:30:39.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0506000000000005e-05 2024-09-17T21:30:39.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457170+0000", 2024-09-17T21:30:39.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5494000000000005e-05 2024-09-17T21:30:39.544 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.544 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.544 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.544 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17617 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:39.544 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:39.544 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.109267248999998, 2024-09-17T21:30:39.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047084168000000003, 2024-09-17T21:30:39.545 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.545 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.545 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.545 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.545 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.545 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.545 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17617 2024-09-17T21:30:39.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.545 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:39.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:39.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:39.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-17T21:30:39.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410237+0000", 2024-09-17T21:30:39.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1960000000000001e-06 2024-09-17T21:30:39.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410246+0000", 2024-09-17T21:30:39.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7880000000000001e-06 2024-09-17T21:30:39.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410250+0000", 2024-09-17T21:30:39.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.518e-06 2024-09-17T21:30:39.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457209+0000", 2024-09-17T21:30:39.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046959283999999997 2024-09-17T21:30:39.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457264+0000", 2024-09-17T21:30:39.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673999999999999e-05 2024-09-17T21:30:39.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457319+0000", 2024-09-17T21:30:39.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5497000000000001e-05 2024-09-17T21:30:39.548 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.548 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.548 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.549 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17618 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 899072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:39.549 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:39.549 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.109247805000003, 2024-09-17T21:30:39.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047204208999999997, 2024-09-17T21:30:39.549 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.549 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.549 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.549 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.549 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.549 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.549 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17618 2024-09-17T21:30:39.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.549 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:39.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:39.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:39.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8799999999999998e-07 2024-09-17T21:30:39.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410256+0000", 2024-09-17T21:30:39.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T21:30:39.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410260+0000", 2024-09-17T21:30:39.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6500000000000002e-06 2024-09-17T21:30:39.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410263+0000", 2024-09-17T21:30:39.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7089999999999998e-06 2024-09-17T21:30:39.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457355+0000", 2024-09-17T21:30:39.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047092756999999999 2024-09-17T21:30:39.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457407+0000", 2024-09-17T21:30:39.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1808999999999998e-05 2024-09-17T21:30:39.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457459+0000", 2024-09-17T21:30:39.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1684000000000002e-05 2024-09-17T21:30:39.553 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.553 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.553 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.553 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17619 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 913408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:39.553 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:39.553 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.109236567000004, 2024-09-17T21:30:39.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047293443999999997, 2024-09-17T21:30:39.554 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.554 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.554 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.554 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17619 2024-09-17T21:30:39.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.554 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:39.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:39.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:39.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:30:39.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410267+0000", 2024-09-17T21:30:39.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:30:39.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410270+0000", 2024-09-17T21:30:39.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.01e-06 2024-09-17T21:30:39.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410273+0000", 2024-09-17T21:30:39.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6410000000000002e-06 2024-09-17T21:30:39.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457473+0000", 2024-09-17T21:30:39.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047199829999999998 2024-09-17T21:30:39.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457517+0000", 2024-09-17T21:30:39.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4058999999999999e-05 2024-09-17T21:30:39.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457559+0000", 2024-09-17T21:30:39.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2404999999999998e-05 2024-09-17T21:30:39.557 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.557 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.557 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.558 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17620 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 927744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:39.558 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:39.558 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.109225395000003, 2024-09-17T21:30:39.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047402409, 2024-09-17T21:30:39.558 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.558 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.558 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.558 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17620 2024-09-17T21:30:39.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.558 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:39.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:39.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:39.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:30:39.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410278+0000", 2024-09-17T21:30:39.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1790000000000001e-06 2024-09-17T21:30:39.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410281+0000", 2024-09-17T21:30:39.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.061e-06 2024-09-17T21:30:39.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410284+0000", 2024-09-17T21:30:39.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.65e-06 2024-09-17T21:30:39.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457587+0000", 2024-09-17T21:30:39.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047303206 2024-09-17T21:30:39.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457633+0000", 2024-09-17T21:30:39.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6063999999999998e-05 2024-09-17T21:30:39.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457679+0000", 2024-09-17T21:30:39.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6007000000000002e-05 2024-09-17T21:30:39.561 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.561 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.562 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.562 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17621 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 940032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:39.562 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:39.562 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.109214979000001, 2024-09-17T21:30:39.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047486550000000002, 2024-09-17T21:30:39.562 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.562 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.562 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.562 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.562 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.562 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.563 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17621 2024-09-17T21:30:39.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.563 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:39.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:39.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410288+0000", 2024-09-17T21:30:39.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:30:39.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410289+0000", 2024-09-17T21:30:39.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1179999999999999e-06 2024-09-17T21:30:39.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410292+0000", 2024-09-17T21:30:39.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:30:39.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410294+0000", 2024-09-17T21:30:39.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T21:30:39.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457694+0000", 2024-09-17T21:30:39.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047399416999999999 2024-09-17T21:30:39.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457734+0000", 2024-09-17T21:30:39.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0528999999999998e-05 2024-09-17T21:30:39.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457774+0000", 2024-09-17T21:30:39.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9529e-05 2024-09-17T21:30:39.566 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.566 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.566 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.566 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17622 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:39.566 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:39.566 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.109204796999997, 2024-09-17T21:30:39.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047596619, 2024-09-17T21:30:39.566 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.566 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.567 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.567 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17622 2024-09-17T21:30:39.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.567 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:39.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:39.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:39.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:39.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410303+0000", 2024-09-17T21:30:39.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2979999999999999e-06 2024-09-17T21:30:39.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410307+0000", 2024-09-17T21:30:39.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:30:39.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410310+0000", 2024-09-17T21:30:39.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.146e-06 2024-09-17T21:30:39.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457801+0000", 2024-09-17T21:30:39.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047491322000000002 2024-09-17T21:30:39.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457850+0000", 2024-09-17T21:30:39.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8585000000000003e-05 2024-09-17T21:30:39.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457894+0000", 2024-09-17T21:30:39.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4455e-05 2024-09-17T21:30:39.570 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.570 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.570 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.571 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17623 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 968704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:39.571 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:39.571 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.109188412999998, 2024-09-17T21:30:39.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047689377999999998, 2024-09-17T21:30:39.571 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.571 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.571 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.571 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.571 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.571 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.571 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17623 2024-09-17T21:30:39.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.571 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:39.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:39.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:39.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:30:39.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410315+0000", 2024-09-17T21:30:39.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0499999999999999e-06 2024-09-17T21:30:39.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410319+0000", 2024-09-17T21:30:39.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.517e-06 2024-09-17T21:30:39.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410321+0000", 2024-09-17T21:30:39.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:30:39.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457922+0000", 2024-09-17T21:30:39.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047600638000000001 2024-09-17T21:30:39.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457964+0000", 2024-09-17T21:30:39.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2126000000000003e-05 2024-09-17T21:30:39.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458003+0000", 2024-09-17T21:30:39.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9138999999999999e-05 2024-09-17T21:30:39.575 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.575 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.575 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.575 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17624 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:39.575 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:39.575 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.109177828999997, 2024-09-17T21:30:39.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047798292999999999, 2024-09-17T21:30:39.575 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.575 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.575 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.575 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.575 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.576 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17624 2024-09-17T21:30:39.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.576 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:39.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:39.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:39.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:30:39.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410326+0000", 2024-09-17T21:30:39.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0610000000000001e-06 2024-09-17T21:30:39.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410329+0000", 2024-09-17T21:30:39.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0319999999999999e-06 2024-09-17T21:30:39.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410331+0000", 2024-09-17T21:30:39.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5749999999999999e-06 2024-09-17T21:30:39.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458033+0000", 2024-09-17T21:30:39.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047701371999999999 2024-09-17T21:30:39.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458079+0000", 2024-09-17T21:30:39.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:30:39.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458123+0000", 2024-09-17T21:30:39.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850999999999997e-05 2024-09-17T21:30:39.579 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.579 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.579 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.579 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17625 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:39.579 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:39.579 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.109167669000001, 2024-09-17T21:30:39.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047883051000000003, 2024-09-17T21:30:39.580 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.580 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.580 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.580 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.580 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.580 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.580 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17625 2024-09-17T21:30:39.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.580 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:39.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:39.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:39.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:39.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410336+0000", 2024-09-17T21:30:39.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1310000000000001e-06 2024-09-17T21:30:39.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410339+0000", 2024-09-17T21:30:39.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000001e-06 2024-09-17T21:30:39.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410342+0000", 2024-09-17T21:30:39.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7460000000000001e-06 2024-09-17T21:30:39.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458137+0000", 2024-09-17T21:30:39.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047794931999999998 2024-09-17T21:30:39.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458178+0000", 2024-09-17T21:30:39.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1013999999999997e-05 2024-09-17T21:30:39.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458218+0000", 2024-09-17T21:30:39.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9675999999999998e-05 2024-09-17T21:30:39.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.583 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.584 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17626 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1009664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:39.584 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:39.584 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.109157048, 2024-09-17T21:30:39.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047991601000000002, 2024-09-17T21:30:39.584 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.584 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.584 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.584 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.584 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.584 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.584 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17626 2024-09-17T21:30:39.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.585 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:39.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:39.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410346+0000", 2024-09-17T21:30:39.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:30:39.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410347+0000", 2024-09-17T21:30:39.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.114e-06 2024-09-17T21:30:39.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410350+0000", 2024-09-17T21:30:39.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.101e-06 2024-09-17T21:30:39.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410353+0000", 2024-09-17T21:30:39.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T21:30:39.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458245+0000", 2024-09-17T21:30:39.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0478921 2024-09-17T21:30:39.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458292+0000", 2024-09-17T21:30:39.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397999999999999e-05 2024-09-17T21:30:39.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458337+0000", 2024-09-17T21:30:39.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5221e-05 2024-09-17T21:30:39.588 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.588 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.588 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.588 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17627 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:39.588 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:39.588 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.109119947000003, 2024-09-17T21:30:39.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048047383999999999, 2024-09-17T21:30:39.588 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.588 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.589 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.589 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.589 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.589 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.589 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17627 2024-09-17T21:30:39.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.589 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:39.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:39.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410383+0000", 2024-09-17T21:30:39.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:30:39.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410385+0000", 2024-09-17T21:30:39.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:30:39.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410391+0000", 2024-09-17T21:30:39.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:30:39.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410395+0000", 2024-09-17T21:30:39.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6539999999999998e-06 2024-09-17T21:30:39.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458350+0000", 2024-09-17T21:30:39.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047954855999999997 2024-09-17T21:30:39.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458392+0000", 2024-09-17T21:30:39.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2094000000000001e-05 2024-09-17T21:30:39.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458430+0000", 2024-09-17T21:30:39.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7474000000000001e-05 2024-09-17T21:30:39.592 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.592 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.592 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.593 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17628 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:39.593 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:39.593 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.109100386000001, 2024-09-17T21:30:39.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048142676000000002, 2024-09-17T21:30:39.593 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.593 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.593 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.593 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.593 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.593 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.593 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17628 2024-09-17T21:30:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:39.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:30:39.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410404+0000", 2024-09-17T21:30:39.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-06 2024-09-17T21:30:39.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410411+0000", 2024-09-17T21:30:39.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6850000000000004e-06 2024-09-17T21:30:39.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410415+0000", 2024-09-17T21:30:39.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6750000000000001e-06 2024-09-17T21:30:39.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458457+0000", 2024-09-17T21:30:39.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048042045999999998 2024-09-17T21:30:39.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458501+0000", 2024-09-17T21:30:39.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3661000000000001e-05 2024-09-17T21:30:39.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458545+0000", 2024-09-17T21:30:39.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3529000000000003e-05 2024-09-17T21:30:39.597 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.597 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.597 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.597 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17687 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:39.597 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:39.597 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.106655433, 2024-09-17T21:30:39.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050670448, 2024-09-17T21:30:39.597 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.597 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.597 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:39.598 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.598 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.598 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.598 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17687 2024-09-17T21:30:39.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.598 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:39.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:39.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412848+0000", 2024-09-17T21:30:39.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9100000000000003e-07 2024-09-17T21:30:39.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413102+0000", 2024-09-17T21:30:39.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025415600000000001 2024-09-17T21:30:39.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413111+0000", 2024-09-17T21:30:39.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2010000000000005e-06 2024-09-17T21:30:39.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413116+0000", 2024-09-17T21:30:39.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:30:39.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458559+0000", 2024-09-17T21:30:39.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045442914000000001 2024-09-17T21:30:39.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458632+0000", 2024-09-17T21:30:39.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3207000000000006e-05 2024-09-17T21:30:39.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:39.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458785+0000", 2024-09-17T21:30:39.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015226700000000001 2024-09-17T21:30:39.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:39.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.461648+0000", 2024-09-17T21:30:39.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028633069999999998 2024-09-17T21:30:39.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:39.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463416+0000", 2024-09-17T21:30:39.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017680230000000001 2024-09-17T21:30:39.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:39.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463475+0000", 2024-09-17T21:30:39.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9333000000000003e-05 2024-09-17T21:30:39.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463517+0000", 2024-09-17T21:30:39.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2162000000000001e-05 2024-09-17T21:30:39.603 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.603 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.603 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.603 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17688 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [write 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:39.603 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:39.603 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.106376564999998, 2024-09-17T21:30:39.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050625098, 2024-09-17T21:30:39.603 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.603 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.603 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:39.603 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.604 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.604 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.604 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17688 2024-09-17T21:30:39.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.604 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:39.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:39.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:39.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:30:39.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413128+0000", 2024-09-17T21:30:39.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.813e-06 2024-09-17T21:30:39.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413132+0000", 2024-09-17T21:30:39.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:30:39.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413135+0000", 2024-09-17T21:30:39.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:30:39.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459756+0000", 2024-09-17T21:30:39.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046621293000000001 2024-09-17T21:30:39.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459816+0000", 2024-09-17T21:30:39.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9156000000000001e-05 2024-09-17T21:30:39.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:39.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459912+0000", 2024-09-17T21:30:39.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6302000000000001e-05 2024-09-17T21:30:39.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:39.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463231+0000", 2024-09-17T21:30:39.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033188290000000001 2024-09-17T21:30:39.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:39.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463701+0000", 2024-09-17T21:30:39.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047068199999999999 2024-09-17T21:30:39.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:39.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463728+0000", 2024-09-17T21:30:39.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6726000000000001e-05 2024-09-17T21:30:39.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463751+0000", 2024-09-17T21:30:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2824e-05 2024-09-17T21:30:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17691 3.d 3:b3a055f3:::rbd_data.11207744cc2c.0000000000000083:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.102831662, 2024-09-17T21:30:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049309370999999998, 2024-09-17T21:30:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.610 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.610 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.610 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17691 2024-09-17T21:30:39.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.610 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:39.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:39.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416672+0000", 2024-09-17T21:30:39.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.47e-07 2024-09-17T21:30:39.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416915+0000", 2024-09-17T21:30:39.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024344500000000001 2024-09-17T21:30:39.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416924+0000", 2024-09-17T21:30:39.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7739999999999993e-06 2024-09-17T21:30:39.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416929+0000", 2024-09-17T21:30:39.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6990000000000002e-06 2024-09-17T21:30:39.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460414+0000", 2024-09-17T21:30:39.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043485466 2024-09-17T21:30:39.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460540+0000", 2024-09-17T21:30:39.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012608899999999999 2024-09-17T21:30:39.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:39.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460746+0000", 2024-09-17T21:30:39.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000205577 2024-09-17T21:30:39.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:39.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463800+0000", 2024-09-17T21:30:39.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030542030000000001 2024-09-17T21:30:39.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:39.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465867+0000", 2024-09-17T21:30:39.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020665929999999998 2024-09-17T21:30:39.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:39.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465914+0000", 2024-09-17T21:30:39.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497000000000003e-05 2024-09-17T21:30:39.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465980+0000", 2024-09-17T21:30:39.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6081000000000006e-05 2024-09-17T21:30:39.614 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.614 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.614 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.615 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:70218 3.16 3:68692d8f:::rbd_data.1120b9193b9f.000000000000001b:head [stat,delete] snapc 1d=[1d] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-17T21:30:39.615 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:39.615 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7138645730000004, 2024-09-17T21:30:39.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.951400786, 2024-09-17T21:30:39.615 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:39.615 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:39.615 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:39.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:39.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:39.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:39.615 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 70218 2024-09-17T21:30:39.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.616 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:39.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:39.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:39.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:39.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:39.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:39.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:39.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:39.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4799999999999998e-07 2024-09-17T21:30:39.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:39.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805642+0000", 2024-09-17T21:30:39.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2009999999999999e-06 2024-09-17T21:30:39.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:39.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805650+0000", 2024-09-17T21:30:39.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1000000000000004e-06 2024-09-17T21:30:39.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:39.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805655+0000", 2024-09-17T21:30:39.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:30:39.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805722+0000", 2024-09-17T21:30:39.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6878000000000006e-05 2024-09-17T21:30:39.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:30:39.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805733+0000", 2024-09-17T21:30:39.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0895e-05 2024-09-17T21:30:39.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:39.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755252+0000", 2024-09-17T21:30:39.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.94951843800000002 2024-09-17T21:30:39.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:39.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755398+0000", 2024-09-17T21:30:39.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014622600000000001 2024-09-17T21:30:39.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:39.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:39.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:39.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.757039+0000", 2024-09-17T21:30:39.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001640819 2024-09-17T21:30:39.620 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.620 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.620 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.620 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:39.620 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:39.620 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:39.622 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:30:39.622 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:39.619+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:39.722 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:39.719+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:39.728 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:39.728 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:39.728 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:39.728 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:39.737 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:30:39.807 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:30:39.823 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:39.820+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:39.897 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:39.897 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:39.898 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:39.898 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:39.898 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:39.909 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:30:39.923 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:39.921+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:39.925 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-17T21:30:39.952 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:39.957 INFO:teuthology.orchestra.run.smithi129.stdout:4468 trunc from 0x10de77c to 0x4a3d77d 2024-09-17T21:30:39.979 INFO:teuthology.orchestra.run.smithi129.stdout:4469 writesame 0xaf54b85 thru 0xaf59c1a data_size 0x5(0x5096 bytes) 2024-09-17T21:30:39.980 INFO:teuthology.orchestra.run.smithi129.stdout:4470 punch from 0x88b1195 to 0x88bc6a2, (0xb50d bytes) 2024-09-17T21:30:39.980 INFO:teuthology.orchestra.run.smithi129.stdout:4471 punch from 0x22844ec to 0x228bd9b, (0x78af bytes) 2024-09-17T21:30:39.985 INFO:teuthology.orchestra.run.smithi129.stdout:4473 write 0xbdb09dd thru 0xbdb8cee (0x8312 bytes) 2024-09-17T21:30:39.997 INFO:teuthology.orchestra.run.smithi129.stdout:4474 write 0xe5d92e6 thru 0xe5e85bb (0xf2d6 bytes) 2024-09-17T21:30:40.012 INFO:teuthology.orchestra.run.smithi129.stdout:4475 trunc from 0xe5e85bc to 0x45ad871 2024-09-17T21:30:40.024 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:40.022+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:40.038 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:40.038 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:40.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:40.038 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:40.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.039 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:79995 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.039 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979453+0000", 2024-09-17T21:30:40.039 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0557368609999997, 2024-09-17T21:30:40.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016640801, 2024-09-17T21:30:40.039 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.039 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.039 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.039 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.039 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.039 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.039 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 79995 2024-09-17T21:30:40.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.039 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979453+0000", 2024-09-17T21:30:40.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979453+0000", 2024-09-17T21:30:40.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979453+0000", 2024-09-17T21:30:40.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8500000000000002e-07 2024-09-17T21:30:40.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979455+0000", 2024-09-17T21:30:40.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0810000000000001e-06 2024-09-17T21:30:40.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979461+0000", 2024-09-17T21:30:40.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:30:40.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979466+0000", 2024-09-17T21:30:40.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5379999999999996e-06 2024-09-17T21:30:40.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.995964+0000", 2024-09-17T21:30:40.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016498228 2024-09-17T21:30:40.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996027+0000", 2024-09-17T21:30:40.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2792999999999993e-05 2024-09-17T21:30:40.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996093+0000", 2024-09-17T21:30:40.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6273999999999996e-05 2024-09-17T21:30:40.043 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.043 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.043 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.043 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:79997 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.043 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979473+0000", 2024-09-17T21:30:40.043 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0557166699999998, 2024-09-17T21:30:40.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016783342999999999, 2024-09-17T21:30:40.043 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.043 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.044 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.044 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.044 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.044 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.044 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 79997 2024-09-17T21:30:40.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.044 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979473+0000", 2024-09-17T21:30:40.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979473+0000", 2024-09-17T21:30:40.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979473+0000", 2024-09-17T21:30:40.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:30:40.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979475+0000", 2024-09-17T21:30:40.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-17T21:30:40.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979481+0000", 2024-09-17T21:30:40.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5930000000000002e-06 2024-09-17T21:30:40.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979486+0000", 2024-09-17T21:30:40.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7659999999999998e-06 2024-09-17T21:30:40.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996119+0000", 2024-09-17T21:30:40.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016632797000000001 2024-09-17T21:30:40.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996186+0000", 2024-09-17T21:30:40.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7390000000000004e-05 2024-09-17T21:30:40.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996256+0000", 2024-09-17T21:30:40.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9646000000000004e-05 2024-09-17T21:30:40.048 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.048 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.048 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.048 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:79999 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.048 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979493+0000", 2024-09-17T21:30:40.048 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0556965739999997, 2024-09-17T21:30:40.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024077517999999999, 2024-09-17T21:30:40.048 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.049 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.049 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.049 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.049 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.049 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.049 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 79999 2024-09-17T21:30:40.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.049 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979493+0000", 2024-09-17T21:30:40.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979493+0000", 2024-09-17T21:30:40.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979493+0000", 2024-09-17T21:30:40.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1199999999999998e-07 2024-09-17T21:30:40.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979495+0000", 2024-09-17T21:30:40.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8059999999999999e-06 2024-09-17T21:30:40.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979501+0000", 2024-09-17T21:30:40.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2720000000000003e-06 2024-09-17T21:30:40.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979506+0000", 2024-09-17T21:30:40.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:30:40.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996323+0000", 2024-09-17T21:30:40.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016817148000000001 2024-09-17T21:30:40.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996392+0000", 2024-09-17T21:30:40.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8295999999999998e-05 2024-09-17T21:30:40.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:35.003570+0000", 2024-09-17T21:30:40.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071786840000000003 2024-09-17T21:30:40.052 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.052 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.052 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.052 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80001 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.053 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979512+0000", 2024-09-17T21:30:40.053 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0556771620000003, 2024-09-17T21:30:40.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017128602, 2024-09-17T21:30:40.053 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.053 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.053 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.053 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.053 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.053 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.053 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80001 2024-09-17T21:30:40.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.053 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979512+0000", 2024-09-17T21:30:40.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979512+0000", 2024-09-17T21:30:40.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979513+0000", 2024-09-17T21:30:40.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9900000000000001e-07 2024-09-17T21:30:40.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979514+0000", 2024-09-17T21:30:40.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7489999999999999e-06 2024-09-17T21:30:40.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979521+0000", 2024-09-17T21:30:40.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2190000000000001e-06 2024-09-17T21:30:40.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979526+0000", 2024-09-17T21:30:40.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9930000000000003e-06 2024-09-17T21:30:40.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996512+0000", 2024-09-17T21:30:40.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016986082999999999 2024-09-17T21:30:40.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996577+0000", 2024-09-17T21:30:40.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5258000000000004e-05 2024-09-17T21:30:40.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996641+0000", 2024-09-17T21:30:40.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3900999999999997e-05 2024-09-17T21:30:40.057 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.057 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.057 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.057 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80003 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.057 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979532+0000", 2024-09-17T21:30:40.057 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0556569009999999, 2024-09-17T21:30:40.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017252887000000001, 2024-09-17T21:30:40.057 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.057 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.057 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.058 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.058 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.058 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.058 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80003 2024-09-17T21:30:40.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.058 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979532+0000", 2024-09-17T21:30:40.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979532+0000", 2024-09-17T21:30:40.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979533+0000", 2024-09-17T21:30:40.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.34e-07 2024-09-17T21:30:40.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979535+0000", 2024-09-17T21:30:40.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.906e-06 2024-09-17T21:30:40.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979541+0000", 2024-09-17T21:30:40.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.489e-06 2024-09-17T21:30:40.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979547+0000", 2024-09-17T21:30:40.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7010000000000001e-06 2024-09-17T21:30:40.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996665+0000", 2024-09-17T21:30:40.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017118187999999999 2024-09-17T21:30:40.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996727+0000", 2024-09-17T21:30:40.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1897999999999996e-05 2024-09-17T21:30:40.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996785+0000", 2024-09-17T21:30:40.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8270999999999998e-05 2024-09-17T21:30:40.061 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.061 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.062 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.062 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80005 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.062 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979554+0000", 2024-09-17T21:30:40.062 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0556353759999997, 2024-09-17T21:30:40.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017377259999999999, 2024-09-17T21:30:40.062 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.062 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.062 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.062 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80005 2024-09-17T21:30:40.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.063 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979554+0000", 2024-09-17T21:30:40.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979554+0000", 2024-09-17T21:30:40.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979554+0000", 2024-09-17T21:30:40.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.58e-07 2024-09-17T21:30:40.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979556+0000", 2024-09-17T21:30:40.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8759999999999999e-06 2024-09-17T21:30:40.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979563+0000", 2024-09-17T21:30:40.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4069999999999998e-06 2024-09-17T21:30:40.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979567+0000", 2024-09-17T21:30:40.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5680000000000001e-06 2024-09-17T21:30:40.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996810+0000", 2024-09-17T21:30:40.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017242683000000002 2024-09-17T21:30:40.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996872+0000", 2024-09-17T21:30:40.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1845000000000002e-05 2024-09-17T21:30:40.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996931+0000", 2024-09-17T21:30:40.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9422999999999997e-05 2024-09-17T21:30:40.066 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.066 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.066 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.066 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80007 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3883008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.066 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979574+0000", 2024-09-17T21:30:40.066 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0556150860000004, 2024-09-17T21:30:40.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017501042000000001, 2024-09-17T21:30:40.066 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.066 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.067 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.067 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80007 2024-09-17T21:30:40.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.068 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979574+0000", 2024-09-17T21:30:40.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979574+0000", 2024-09-17T21:30:40.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979575+0000", 2024-09-17T21:30:40.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:30:40.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979577+0000", 2024-09-17T21:30:40.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0420000000000001e-06 2024-09-17T21:30:40.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979583+0000", 2024-09-17T21:30:40.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3110000000000002e-06 2024-09-17T21:30:40.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979588+0000", 2024-09-17T21:30:40.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0420000000000002e-06 2024-09-17T21:30:40.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.996954+0000", 2024-09-17T21:30:40.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017365862999999999 2024-09-17T21:30:40.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997019+0000", 2024-09-17T21:30:40.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4838999999999994e-05 2024-09-17T21:30:40.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997075+0000", 2024-09-17T21:30:40.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6427e-05 2024-09-17T21:30:40.072 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.072 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.072 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.072 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80009 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.072 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979595+0000", 2024-09-17T21:30:40.072 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0555941860000004, 2024-09-17T21:30:40.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017636168000000001, 2024-09-17T21:30:40.072 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.072 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.072 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.073 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.073 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.073 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.073 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80009 2024-09-17T21:30:40.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.073 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979595+0000", 2024-09-17T21:30:40.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979595+0000", 2024-09-17T21:30:40.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979596+0000", 2024-09-17T21:30:40.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3300000000000002e-07 2024-09-17T21:30:40.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979598+0000", 2024-09-17T21:30:40.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.95e-06 2024-09-17T21:30:40.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979604+0000", 2024-09-17T21:30:40.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4579999999999998e-06 2024-09-17T21:30:40.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979609+0000", 2024-09-17T21:30:40.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7770000000000002e-06 2024-09-17T21:30:40.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997098+0000", 2024-09-17T21:30:40.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017489395000000001 2024-09-17T21:30:40.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997171+0000", 2024-09-17T21:30:40.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2410000000000006e-05 2024-09-17T21:30:40.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997231+0000", 2024-09-17T21:30:40.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0645e-05 2024-09-17T21:30:40.076 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.077 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.077 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.077 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80011 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.077 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979615+0000", 2024-09-17T21:30:40.077 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0555738650000004, 2024-09-17T21:30:40.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017760018999999998, 2024-09-17T21:30:40.077 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.077 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.077 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.077 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.077 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.078 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.078 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80011 2024-09-17T21:30:40.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.078 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979615+0000", 2024-09-17T21:30:40.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979615+0000", 2024-09-17T21:30:40.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979616+0000", 2024-09-17T21:30:40.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1699999999999998e-07 2024-09-17T21:30:40.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979618+0000", 2024-09-17T21:30:40.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9750000000000001e-06 2024-09-17T21:30:40.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979624+0000", 2024-09-17T21:30:40.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8699999999999997e-06 2024-09-17T21:30:40.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979629+0000", 2024-09-17T21:30:40.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.78e-06 2024-09-17T21:30:40.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997254+0000", 2024-09-17T21:30:40.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017625339 2024-09-17T21:30:40.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997315+0000", 2024-09-17T21:30:40.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0914e-05 2024-09-17T21:30:40.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997376+0000", 2024-09-17T21:30:40.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0624000000000001e-05 2024-09-17T21:30:40.081 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.081 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.081 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.081 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80224 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.081 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.979990+0000", 2024-09-17T21:30:40.081 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0551993919999996, 2024-09-17T21:30:40.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017530202000000002, 2024-09-17T21:30:40.082 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.082 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.082 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.082 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.082 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.082 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.082 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80224 2024-09-17T21:30:40.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.082 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979990+0000", 2024-09-17T21:30:40.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979990+0000", 2024-09-17T21:30:40.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979990+0000", 2024-09-17T21:30:40.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:30:40.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979992+0000", 2024-09-17T21:30:40.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-17T21:30:40.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.979999+0000", 2024-09-17T21:30:40.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5100000000000004e-06 2024-09-17T21:30:40.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980004+0000", 2024-09-17T21:30:40.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:30:40.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997399+0000", 2024-09-17T21:30:40.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017395342000000001 2024-09-17T21:30:40.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997462+0000", 2024-09-17T21:30:40.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2828000000000002e-05 2024-09-17T21:30:40.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997520+0000", 2024-09-17T21:30:40.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8221999999999999e-05 2024-09-17T21:30:40.085 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.086 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.086 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.086 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80225 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.086 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980011+0000", 2024-09-17T21:30:40.086 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.055178443, 2024-09-17T21:30:40.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017655349000000001, 2024-09-17T21:30:40.086 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.086 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.086 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.086 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.086 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.087 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.087 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80225 2024-09-17T21:30:40.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.087 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980011+0000", 2024-09-17T21:30:40.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980011+0000", 2024-09-17T21:30:40.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980011+0000", 2024-09-17T21:30:40.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2800000000000002e-07 2024-09-17T21:30:40.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980013+0000", 2024-09-17T21:30:40.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9420000000000002e-06 2024-09-17T21:30:40.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980019+0000", 2024-09-17T21:30:40.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.976e-06 2024-09-17T21:30:40.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980024+0000", 2024-09-17T21:30:40.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7600000000000002e-06 2024-09-17T21:30:40.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997544+0000", 2024-09-17T21:30:40.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017520134999999999 2024-09-17T21:30:40.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997607+0000", 2024-09-17T21:30:40.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2928999999999993e-05 2024-09-17T21:30:40.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997666+0000", 2024-09-17T21:30:40.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9178999999999997e-05 2024-09-17T21:30:40.090 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.090 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.090 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.090 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80226 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3227648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.091 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980031+0000", 2024-09-17T21:30:40.091 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.055157994, 2024-09-17T21:30:40.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017780114, 2024-09-17T21:30:40.091 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.091 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.091 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.091 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.091 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.091 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.091 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80226 2024-09-17T21:30:40.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.091 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980031+0000", 2024-09-17T21:30:40.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980031+0000", 2024-09-17T21:30:40.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980032+0000", 2024-09-17T21:30:40.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.01e-07 2024-09-17T21:30:40.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980034+0000", 2024-09-17T21:30:40.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8309999999999999e-06 2024-09-17T21:30:40.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980040+0000", 2024-09-17T21:30:40.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:30:40.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980045+0000", 2024-09-17T21:30:40.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7559999999999999e-06 2024-09-17T21:30:40.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997689+0000", 2024-09-17T21:30:40.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017644395 2024-09-17T21:30:40.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997751+0000", 2024-09-17T21:30:40.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2238999999999998e-05 2024-09-17T21:30:40.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997811+0000", 2024-09-17T21:30:40.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0078000000000003e-05 2024-09-17T21:30:40.095 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.095 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.095 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.095 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80227 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.095 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980051+0000", 2024-09-17T21:30:40.095 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0551379040000004, 2024-09-17T21:30:40.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017904265999999999, 2024-09-17T21:30:40.095 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.095 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.095 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.095 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.095 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.096 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.096 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80227 2024-09-17T21:30:40.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.096 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980051+0000", 2024-09-17T21:30:40.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980051+0000", 2024-09-17T21:30:40.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980052+0000", 2024-09-17T21:30:40.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1199999999999998e-07 2024-09-17T21:30:40.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980054+0000", 2024-09-17T21:30:40.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7659999999999999e-06 2024-09-17T21:30:40.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980060+0000", 2024-09-17T21:30:40.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5379999999999999e-06 2024-09-17T21:30:40.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980065+0000", 2024-09-17T21:30:40.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6e-06 2024-09-17T21:30:40.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997835+0000", 2024-09-17T21:30:40.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017770013000000001 2024-09-17T21:30:40.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997896+0000", 2024-09-17T21:30:40.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0803e-05 2024-09-17T21:30:40.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997956+0000", 2024-09-17T21:30:40.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0133999999999997e-05 2024-09-17T21:30:40.099 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.099 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.099 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.099 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80228 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3457024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.099 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980071+0000", 2024-09-17T21:30:40.099 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0551178879999998, 2024-09-17T21:30:40.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018027676999999999, 2024-09-17T21:30:40.099 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.099 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.099 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.100 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.100 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.100 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.100 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80228 2024-09-17T21:30:40.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.100 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980071+0000", 2024-09-17T21:30:40.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980071+0000", 2024-09-17T21:30:40.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980072+0000", 2024-09-17T21:30:40.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:30:40.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980074+0000", 2024-09-17T21:30:40.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.728e-06 2024-09-17T21:30:40.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980080+0000", 2024-09-17T21:30:40.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3339999999999998e-06 2024-09-17T21:30:40.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980085+0000", 2024-09-17T21:30:40.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6700000000000002e-06 2024-09-17T21:30:40.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.997978+0000", 2024-09-17T21:30:40.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017893678 2024-09-17T21:30:40.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998042+0000", 2024-09-17T21:30:40.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3193000000000003e-05 2024-09-17T21:30:40.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998099+0000", 2024-09-17T21:30:40.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7651000000000001e-05 2024-09-17T21:30:40.103 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.103 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.103 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.104 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80229 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.104 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980091+0000", 2024-09-17T21:30:40.104 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0550982229999999, 2024-09-17T21:30:40.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018156578999999999, 2024-09-17T21:30:40.104 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.104 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.104 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.104 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.104 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.104 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.104 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80229 2024-09-17T21:30:40.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.105 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980091+0000", 2024-09-17T21:30:40.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980091+0000", 2024-09-17T21:30:40.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980092+0000", 2024-09-17T21:30:40.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:30:40.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980094+0000", 2024-09-17T21:30:40.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8339999999999999e-06 2024-09-17T21:30:40.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980100+0000", 2024-09-17T21:30:40.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4470000000000003e-06 2024-09-17T21:30:40.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980105+0000", 2024-09-17T21:30:40.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:30:40.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998122+0000", 2024-09-17T21:30:40.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018017610999999999 2024-09-17T21:30:40.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998189+0000", 2024-09-17T21:30:40.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6280999999999998e-05 2024-09-17T21:30:40.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998248+0000", 2024-09-17T21:30:40.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9040999999999999e-05 2024-09-17T21:30:40.108 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.108 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.108 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.108 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80230 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3686400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.108 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980111+0000", 2024-09-17T21:30:40.108 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.055078172, 2024-09-17T21:30:40.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018282363999999999, 2024-09-17T21:30:40.108 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.109 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.109 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.109 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.109 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.109 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.109 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80230 2024-09-17T21:30:40.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.109 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980111+0000", 2024-09-17T21:30:40.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980111+0000", 2024-09-17T21:30:40.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980112+0000", 2024-09-17T21:30:40.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5400000000000002e-07 2024-09-17T21:30:40.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980113+0000", 2024-09-17T21:30:40.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8199999999999999e-06 2024-09-17T21:30:40.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980120+0000", 2024-09-17T21:30:40.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3709999999999996e-06 2024-09-17T21:30:40.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980128+0000", 2024-09-17T21:30:40.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9079999999999995e-06 2024-09-17T21:30:40.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998272+0000", 2024-09-17T21:30:40.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018143920000000001 2024-09-17T21:30:40.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998334+0000", 2024-09-17T21:30:40.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1921000000000006e-05 2024-09-17T21:30:40.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998394+0000", 2024-09-17T21:30:40.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9969999999999997e-05 2024-09-17T21:30:40.115 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.115 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.115 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.115 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80231 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3801088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.115 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980135+0000", 2024-09-17T21:30:40.115 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0550548299999996, 2024-09-17T21:30:40.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018400430999999998, 2024-09-17T21:30:40.116 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.116 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.116 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.116 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.116 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.116 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.116 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80231 2024-09-17T21:30:40.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.116 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980135+0000", 2024-09-17T21:30:40.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980135+0000", 2024-09-17T21:30:40.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980135+0000", 2024-09-17T21:30:40.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3499999999999996e-07 2024-09-17T21:30:40.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980138+0000", 2024-09-17T21:30:40.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7970000000000001e-06 2024-09-17T21:30:40.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980144+0000", 2024-09-17T21:30:40.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.392e-06 2024-09-17T21:30:40.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980153+0000", 2024-09-17T21:30:40.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7010000000000002e-06 2024-09-17T21:30:40.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998417+0000", 2024-09-17T21:30:40.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018263643 2024-09-17T21:30:40.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998476+0000", 2024-09-17T21:30:40.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9417999999999999e-05 2024-09-17T21:30:40.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998535+0000", 2024-09-17T21:30:40.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8944999999999999e-05 2024-09-17T21:30:40.119 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.119 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.120 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.120 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80232 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.120 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980160+0000", 2024-09-17T21:30:40.120 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0550293960000001, 2024-09-17T21:30:40.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018519280999999999, 2024-09-17T21:30:40.120 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.121 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.121 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.121 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80232 2024-09-17T21:30:40.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.121 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980160+0000", 2024-09-17T21:30:40.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980160+0000", 2024-09-17T21:30:40.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980160+0000", 2024-09-17T21:30:40.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8699999999999995e-07 2024-09-17T21:30:40.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980176+0000", 2024-09-17T21:30:40.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5529e-05 2024-09-17T21:30:40.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980183+0000", 2024-09-17T21:30:40.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0829999999999998e-06 2024-09-17T21:30:40.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980188+0000", 2024-09-17T21:30:40.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1000000000000003e-06 2024-09-17T21:30:40.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998558+0000", 2024-09-17T21:30:40.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018369897999999999 2024-09-17T21:30:40.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998620+0000", 2024-09-17T21:30:40.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1550999999999994e-05 2024-09-17T21:30:40.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998679+0000", 2024-09-17T21:30:40.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9633000000000003e-05 2024-09-17T21:30:40.124 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.124 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.124 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.125 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80234 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.125 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980197+0000", 2024-09-17T21:30:40.125 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0549918839999997, 2024-09-17T21:30:40.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018624308999999999, 2024-09-17T21:30:40.125 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.125 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.125 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.125 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.125 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.125 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.125 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80234 2024-09-17T21:30:40.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.125 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980197+0000", 2024-09-17T21:30:40.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980197+0000", 2024-09-17T21:30:40.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980198+0000", 2024-09-17T21:30:40.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.34e-07 2024-09-17T21:30:40.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980200+0000", 2024-09-17T21:30:40.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9589999999999998e-06 2024-09-17T21:30:40.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980206+0000", 2024-09-17T21:30:40.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5250000000000002e-06 2024-09-17T21:30:40.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980211+0000", 2024-09-17T21:30:40.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:30:40.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998702+0000", 2024-09-17T21:30:40.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018490273000000002 2024-09-17T21:30:40.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998764+0000", 2024-09-17T21:30:40.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2305999999999996e-05 2024-09-17T21:30:40.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998822+0000", 2024-09-17T21:30:40.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7936000000000002e-05 2024-09-17T21:30:40.129 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.129 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.129 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.129 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:80236 3.4 3:2293b9c8:::rbd_data.11203c8fd766.0000000000000001:1c [read 4145152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T21:30:40.129 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:34.980218+0000", 2024-09-17T21:30:40.129 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0549712089999996, 2024-09-17T21:30:40.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018747997999999998, 2024-09-17T21:30:40.129 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.129 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.129 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.129 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.130 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.130 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.130 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 80236 2024-09-17T21:30:40.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.130 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980218+0000", 2024-09-17T21:30:40.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980218+0000", 2024-09-17T21:30:40.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980219+0000", 2024-09-17T21:30:40.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.39e-07 2024-09-17T21:30:40.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980221+0000", 2024-09-17T21:30:40.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:30:40.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980227+0000", 2024-09-17T21:30:40.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8240000000000002e-06 2024-09-17T21:30:40.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.980232+0000", 2024-09-17T21:30:40.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999998e-06 2024-09-17T21:30:40.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998844+0000", 2024-09-17T21:30:40.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018612257 2024-09-17T21:30:40.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998907+0000", 2024-09-17T21:30:40.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2860999999999993e-05 2024-09-17T21:30:40.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:34.998966+0000", 2024-09-17T21:30:40.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8897999999999998e-05 2024-09-17T21:30:40.133 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.133 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.133 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.133 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.133 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.133 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:40.134 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:30:40.138 INFO:teuthology.orchestra.run.smithi129.stdout:4477 write 0x6cd35ac thru 0x6cde405 (0xae5a bytes) 2024-09-17T21:30:40.138 INFO:teuthology.orchestra.run.smithi129.stdout:4479 punch from 0xc3ccdd to 0xc417f5, (0x4b18 bytes) 2024-09-17T21:30:40.138 INFO:teuthology.orchestra.run.smithi129.stdout:4480 read 0x5ac2173 thru 0x5ac9f83 (0x7e11 bytes) 2024-09-17T21:30:40.138 INFO:teuthology.orchestra.run.smithi129.stdout:4481 read 0x8c9dda thru 0x8d8904 (0xeb2b bytes) 2024-09-17T21:30:40.138 INFO:teuthology.orchestra.run.smithi129.stdout:4482 write 0x36e8b7e thru 0x36eb526 (0x29a9 bytes) 2024-09-17T21:30:40.138 INFO:teuthology.orchestra.run.smithi129.stdout:4483 read 0x40f14c1 thru 0x40f4b53 (0x3693 bytes) 2024-09-17T21:30:40.138 INFO:teuthology.orchestra.run.smithi129.stdout:4484 write 0x35afc19 thru 0x35be663 (0xea4b bytes) 2024-09-17T21:30:40.138 INFO:teuthology.orchestra.run.smithi129.stdout:4487 writesame 0x7e7e20b thru 0x7e826cf data_size 0x7(0x44c5 bytes) 2024-09-17T21:30:40.138 INFO:teuthology.orchestra.run.smithi129.stdout:4488 punch from 0x336eca8 to 0x337b5ed, (0xc945 bytes) 2024-09-17T21:30:40.139 INFO:teuthology.orchestra.run.smithi129.stdout:4489 trunc from 0x7e826d0 to 0x9048060 2024-09-17T21:30:40.139 INFO:teuthology.orchestra.run.smithi129.stdout:4490 read 0xcb327a thru 0xcc05d3 (0xd35a bytes) 2024-09-17T21:30:40.139 INFO:teuthology.orchestra.run.smithi129.stdout:4491 punch from 0x16b01b8 to 0x16b4f6d, (0x4db5 bytes) 2024-09-17T21:30:40.139 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:40.139 INFO:teuthology.orchestra.run.smithi129.stdout:4493 writesame 0xbd5e064 thru 0xbd68f83 data_size 0x30(0xaf20 bytes) 2024-09-17T21:30:40.139 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:40.139 INFO:teuthology.orchestra.run.smithi129.stdout:4495 punch from 0x82c3156 to 0x82c8b79, (0x5a23 bytes) 2024-09-17T21:30:40.139 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:40.139 INFO:teuthology.orchestra.run.smithi129.stdout:4499 write 0x79fa76d thru 0x79fab0f (0x3a3 bytes) 2024-09-17T21:30:40.139 INFO:teuthology.orchestra.run.smithi129.stdout:4500 write 0xee1090d thru 0xee1b177 (0xa86b bytes) 2024-09-17T21:30:40.139 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:40.139 INFO:teuthology.orchestra.run.smithi129.stdout:4504 trunc from 0xee1b178 to 0xc6d8890 2024-09-17T21:30:40.139 INFO:teuthology.orchestra.run.smithi129.stdout:4506 write 0x9adfdf5 thru 0x9ae70a0 (0x72ac bytes) 2024-09-17T21:30:40.139 INFO:teuthology.orchestra.run.smithi129.stdout:4507 write 0xbec6dbe thru 0xbed5046 (0xe289 bytes) 2024-09-17T21:30:40.140 INFO:teuthology.orchestra.run.smithi129.stdout:4508 punch from 0x9fb0cda to 0x9fb7340, (0x6666 bytes) 2024-09-17T21:30:40.140 INFO:teuthology.orchestra.run.smithi129.stdout:4509 writesame 0x7a6a4bc thru 0x7a7024e data_size 0x5(0x5d93 bytes) 2024-09-17T21:30:40.140 INFO:teuthology.orchestra.run.smithi129.stdout:4511 write 0xdc63e2d thru 0xdc69aa8 (0x5c7c bytes) 2024-09-17T21:30:40.140 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:40.140 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:40.140 INFO:teuthology.orchestra.run.smithi129.stdout:4514 write 0x2b0833a thru 0x2b0c80a (0x44d1 bytes) 2024-09-17T21:30:40.140 INFO:teuthology.orchestra.run.smithi129.stdout:4517 write 0x20ff157 thru 0x210a753 (0xb5fd bytes) 2024-09-17T21:30:40.140 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:40.140 INFO:teuthology.orchestra.run.smithi129.stdout:4519 read 0x3ad315f thru 0x3adcf38 (0x9dda bytes) 2024-09-17T21:30:40.140 INFO:teuthology.orchestra.run.smithi129.stdout:4521 read 0x3623492 thru 0x3628557 (0x50c6 bytes) 2024-09-17T21:30:40.140 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:40.140 INFO:teuthology.orchestra.run.smithi129.stdout:4523 write 0xd34a0fc thru 0xd352490 (0x8395 bytes) 2024-09-17T21:30:40.140 INFO:teuthology.orchestra.run.smithi129.stdout:4524 write 0xa23a69c thru 0xa2400f4 (0x5a59 bytes) 2024-09-17T21:30:40.141 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:40.135+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:40.142 INFO:teuthology.orchestra.run.smithi129.stdout:4525 trunc from 0xdc69aa9 to 0xcdc5428 2024-09-17T21:30:40.160 INFO:teuthology.orchestra.run.smithi129.stdout:4526 trunc from 0xcdc5428 to 0x2ecb2ca 2024-09-17T21:30:40.176 INFO:teuthology.orchestra.run.smithi129.stdout:4528 trunc from 0x2ecb2ca to 0x484b542 2024-09-17T21:30:40.189 INFO:teuthology.orchestra.run.smithi129.stdout:4529 writesame 0x66d3a8a thru 0x66e37ea data_size 0x1(0xfd61 bytes) 2024-09-17T21:30:40.190 INFO:teuthology.orchestra.run.smithi129.stdout:4530 read 0x5605ba3 thru 0x560765f (0x1abd bytes) 2024-09-17T21:30:40.191 INFO:teuthology.orchestra.run.smithi129.stdout:4531 punch from 0x6a34e2 to 0x6a6c06, (0x3724 bytes) 2024-09-17T21:30:40.191 INFO:teuthology.orchestra.run.smithi129.stdout:4532 punch from 0x25069a8 to 0x250c509, (0x5b61 bytes) 2024-09-17T21:30:40.191 INFO:teuthology.orchestra.run.smithi129.stdout:4533 read 0x2810091 thru 0x2812902 (0x2872 bytes) 2024-09-17T21:30:40.191 INFO:teuthology.orchestra.run.smithi129.stdout:4534 read 0x110cb11 thru 0x1114e6c (0x835c bytes) 2024-09-17T21:30:40.199 INFO:teuthology.orchestra.run.smithi129.stdout:4535 trunc from 0x66e37eb to 0x5453e7 2024-09-17T21:30:40.215 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:40.215 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:40.231 INFO:teuthology.orchestra.run.smithi129.stdout:4539 write 0x99df8b9 thru 0x99eea4c (0xf194 bytes) 2024-09-17T21:30:40.238 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:40.235+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:40.240 INFO:teuthology.orchestra.run.smithi129.stdout:4540 trunc from 0x99eea4d to 0x4341cc0 2024-09-17T21:30:40.251 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:40.251 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:40.251 INFO:teuthology.orchestra.run.smithi129.stdout:4543 read 0x3e33eed thru 0x3e3407d (0x191 bytes) 2024-09-17T21:30:40.251 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:40.252 INFO:teuthology.orchestra.run.smithi129.stdout:4545 write 0x3bf42b6 thru 0x3bf95b8 (0x5303 bytes) 2024-09-17T21:30:40.253 INFO:teuthology.orchestra.run.smithi129.stdout:4546 read 0x83b37d thru 0x845c14 (0xa898 bytes) 2024-09-17T21:30:40.253 INFO:teuthology.orchestra.run.smithi129.stdout:4547 read 0x2dda73a thru 0x2ddda8a (0x3351 bytes) 2024-09-17T21:30:40.253 INFO:teuthology.orchestra.run.smithi129.stdout:4548 flatten 2024-09-17T21:30:40.258 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:40.259 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:40.259 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:40.259 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:40.264 INFO:teuthology.orchestra.run.smithi129.stdout:4549 trunc from 0x4341cc0 to 0x93f6130 2024-09-17T21:30:40.271 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:30:40.280 INFO:teuthology.orchestra.run.smithi129.stdout:4552 write 0xc67ad9f thru 0xc6827ea (0x7a4c bytes) 2024-09-17T21:30:40.283 INFO:teuthology.orchestra.run.smithi129.stdout:4553 read 0xc4a4742 thru 0xc4ae9a3 (0xa262 bytes) 2024-09-17T21:30:40.283 INFO:teuthology.orchestra.run.smithi129.stdout:4554 read 0x95cddc7 thru 0x95dce94 (0xf0ce bytes) 2024-09-17T21:30:40.283 INFO:teuthology.orchestra.run.smithi129.stdout:4555 write 0x208aceb thru 0x2099a84 (0xed9a bytes) 2024-09-17T21:30:40.292 INFO:teuthology.orchestra.run.smithi129.stdout:4556 write 0xca1a7f4 thru 0xca27d9d (0xd5aa bytes) 2024-09-17T21:30:40.296 INFO:teuthology.orchestra.run.smithi129.stdout:4557 read 0xb06f62f thru 0xb07aa15 (0xb3e7 bytes) 2024-09-17T21:30:40.296 INFO:teuthology.orchestra.run.smithi129.stdout:4558 write 0x82fba06 thru 0x8300e84 (0x547f bytes) 2024-09-17T21:30:40.297 INFO:teuthology.orchestra.run.smithi129.stdout:4560 read 0x96ae6c6 thru 0x96b2063 (0x399e bytes) 2024-09-17T21:30:40.297 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:40.297 INFO:teuthology.orchestra.run.smithi129.stdout:4562 read 0x1298fd5 thru 0x129e3d1 (0x53fd bytes) 2024-09-17T21:30:40.298 INFO:teuthology.orchestra.run.smithi129.stdout:4563 punch from 0x5c8879 to 0x5cc399, (0x3b20 bytes) 2024-09-17T21:30:40.298 INFO:teuthology.orchestra.run.smithi129.stdout:4564 read 0x9627a20 thru 0x9636f32 (0xf513 bytes) 2024-09-17T21:30:40.298 INFO:teuthology.orchestra.run.smithi129.stdout:4565 read 0xbd6f129 thru 0xbd72642 (0x351a bytes) 2024-09-17T21:30:40.303 INFO:teuthology.orchestra.run.smithi129.stdout:4566 writesame 0xd116fb3 thru 0xd123343 data_size 0x5f(0xc391 bytes) 2024-09-17T21:30:40.306 INFO:teuthology.orchestra.run.smithi129.stdout:4567 trunc from 0xd123344 to 0xaa4535a 2024-09-17T21:30:40.319 INFO:teuthology.orchestra.run.smithi129.stdout:4568 writesame 0xbe2fffd thru 0xbe33d3d data_size 0x1(0x3d41 bytes) 2024-09-17T21:30:40.319 INFO:teuthology.orchestra.run.smithi129.stdout:4571 write 0x76005db thru 0x760a156 (0x9b7c bytes) 2024-09-17T21:30:40.321 INFO:teuthology.orchestra.run.smithi129.stdout:4572 write 0x9738d8e thru 0x973a05e (0x12d1 bytes) 2024-09-17T21:30:40.324 INFO:teuthology.orchestra.run.smithi129.stdout:4575 punch from 0x1ed6887 to 0x1eddf8c, (0x7705 bytes) 2024-09-17T21:30:40.324 INFO:teuthology.orchestra.run.smithi129.stdout:4576 punch from 0xb4c1319 to 0xb4c2264, (0xf4b bytes) 2024-09-17T21:30:40.325 INFO:teuthology.orchestra.run.smithi129.stdout:4577 writesame 0x5d61a24 thru 0x5d6f6a2 data_size 0x1(0xdc7f bytes) 2024-09-17T21:30:40.339 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:40.336+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:40.351 INFO:teuthology.orchestra.run.smithi129.stdout:4578 write 0x3d85833 thru 0x3d8f7f0 (0x9fbe bytes) 2024-09-17T21:30:40.354 INFO:teuthology.orchestra.run.smithi129.stdout:4579 read 0x76c1bb1 thru 0x76c91f2 (0x7642 bytes) 2024-09-17T21:30:40.355 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:40.355 INFO:teuthology.orchestra.run.smithi129.stdout:4582 punch from 0x3cc0c0a to 0x3cc37eb, (0x2be1 bytes) 2024-09-17T21:30:40.361 INFO:teuthology.orchestra.run.smithi129.stdout:4583 writesame 0xcf8be0c thru 0xcf8fbee data_size 0x1(0x3de3 bytes) 2024-09-17T21:30:40.361 INFO:teuthology.orchestra.run.smithi129.stdout:4584 write 0x645f584 thru 0x64656e4 (0x6161 bytes) 2024-09-17T21:30:40.363 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:40.371 INFO:teuthology.orchestra.run.smithi129.stdout:4587 write 0xec0faf7 thru 0xec163f6 (0x6900 bytes) 2024-09-17T21:30:40.374 INFO:teuthology.orchestra.run.smithi129.stdout:4588 punch from 0x921f696 to 0x9227c8c, (0x85f6 bytes) 2024-09-17T21:30:40.378 INFO:teuthology.orchestra.run.smithi129.stdout:4589 trunc from 0xec163f7 to 0xb1e4422 2024-09-17T21:30:40.385 INFO:teuthology.orchestra.run.smithi129.stdout:4590 read 0x82b1f96 thru 0x82bd45a (0xb4c5 bytes) 2024-09-17T21:30:40.390 INFO:teuthology.orchestra.run.smithi129.stdout:4592 read 0x6098cea thru 0x60a0f53 (0x826a bytes) 2024-09-17T21:30:40.390 INFO:teuthology.orchestra.run.smithi129.stdout:4593 punch from 0x9a651c8 to 0x9a6db9a, (0x89d2 bytes) 2024-09-17T21:30:40.392 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:40.392 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:40.393 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:40.393 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:40.393 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:40.399 INFO:teuthology.orchestra.run.smithi129.stdout:4594 write 0xe98c28e thru 0xe9966e9 (0xa45c bytes) 2024-09-17T21:30:40.403 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:30:40.404 INFO:teuthology.orchestra.run.smithi129.stdout:4596 writesame 0xe33a840 thru 0xe342e2b data_size 0x6(0x85ec bytes) 2024-09-17T21:30:40.408 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:40.408 INFO:teuthology.orchestra.run.smithi129.stdout:4599 write 0x11e71aa thru 0x11eeead (0x7d04 bytes) 2024-09-17T21:30:40.410 INFO:teuthology.orchestra.run.smithi129.stdout:4601 writesame 0xcaacb9d thru 0xcab6b0e data_size 0x3(0x9f72 bytes) 2024-09-17T21:30:40.418 INFO:teuthology.orchestra.run.smithi129.stdout:4602 writesame 0xacfb919 thru 0xad087bf data_size 0x1(0xcea7 bytes) 2024-09-17T21:30:40.440 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:40.438+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:40.443 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:40.443 INFO:teuthology.orchestra.run.smithi129.stdout:4606 write 0xdac4ba2 thru 0xdad3a6e (0xeecd bytes) 2024-09-17T21:30:40.444 INFO:teuthology.orchestra.run.smithi129.stdout:4607 read 0xe22f932 thru 0xe2303a6 (0xa75 bytes) 2024-09-17T21:30:40.461 INFO:teuthology.orchestra.run.smithi129.stdout:4608 trunc from 0xe9966ea to 0x144c18c 2024-09-17T21:30:40.474 INFO:teuthology.orchestra.run.smithi129.stdout:4609 punch from 0xe76353 to 0xe7a71d, (0x43ca bytes) 2024-09-17T21:30:40.495 INFO:teuthology.orchestra.run.smithi129.stdout:4610 write 0xd742944 thru 0xd746036 (0x36f3 bytes) 2024-09-17T21:30:40.497 INFO:teuthology.orchestra.run.smithi129.stdout:4611 write 0x6166e6d thru 0x6173d6a (0xcefe bytes) 2024-09-17T21:30:40.499 INFO:teuthology.orchestra.run.smithi129.stdout:4612 punch from 0xaaeb65f to 0xaaf2bbb, (0x755c bytes) 2024-09-17T21:30:40.499 INFO:teuthology.orchestra.run.smithi129.stdout:4613 punch from 0x3950cc9 to 0x395a776, (0x9aad bytes) 2024-09-17T21:30:40.499 INFO:teuthology.orchestra.run.smithi129.stdout:4615 write 0x9602004 thru 0x9602c1c (0xc19 bytes) 2024-09-17T21:30:40.502 INFO:teuthology.orchestra.run.smithi129.stdout:4616 clone 32 order 22 su 65536 sc 14 2024-09-17T21:30:40.523 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:40.524 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:40.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:40.524 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:40.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.524 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:37817 3.17 3:eab85a8d:::rbd_header.1120e62eea89:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:30:40.524 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:40.524 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.824948354, 2024-09-17T21:30:40.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86635970399999995, 2024-09-17T21:30:40.524 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.524 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.524 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.524 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.524 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.525 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.525 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37817 2024-09-17T21:30:40.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.525 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:40.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:40.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695457+0000", 2024-09-17T21:30:40.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5799999999999998e-07 2024-09-17T21:30:40.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695461+0000", 2024-09-17T21:30:40.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1110000000000001e-06 2024-09-17T21:30:40.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695471+0000", 2024-09-17T21:30:40.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0441e-05 2024-09-17T21:30:40.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695477+0000", 2024-09-17T21:30:40.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7599999999999999e-06 2024-09-17T21:30:40.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695533+0000", 2024-09-17T21:30:40.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6570999999999998e-05 2024-09-17T21:30:40.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:30:40.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695543+0000", 2024-09-17T21:30:40.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9979999999999998e-06 2024-09-17T21:30:40.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561472+0000", 2024-09-17T21:30:40.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86592841899999995 2024-09-17T21:30:40.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561610+0000", 2024-09-17T21:30:40.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013854699999999999 2024-09-17T21:30:40.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561816+0000", 2024-09-17T21:30:40.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020509900000000001 2024-09-17T21:30:40.529 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.529 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.529 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.529 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50202 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.530 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:40.530 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.412777563999999, 2024-09-17T21:30:40.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.076620261999999995, 2024-09-17T21:30:40.530 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.530 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.530 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:40.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.530 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50202 2024-09-17T21:30:40.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.530 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:40.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:40.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107629+0000", 2024-09-17T21:30:40.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3709999999999998e-06 2024-09-17T21:30:40.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119765+0000", 2024-09-17T21:30:40.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012135683 2024-09-17T21:30:40.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119787+0000", 2024-09-17T21:30:40.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2748999999999999e-05 2024-09-17T21:30:40.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119797+0000", 2024-09-17T21:30:40.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8169999999999994e-06 2024-09-17T21:30:40.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119872+0000", 2024-09-17T21:30:40.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270999999999994e-05 2024-09-17T21:30:40.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119987+0000", 2024-09-17T21:30:40.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011557699999999999 2024-09-17T21:30:40.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:40.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.121766+0000", 2024-09-17T21:30:40.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00177853 2024-09-17T21:30:40.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:40.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178307+0000", 2024-09-17T21:30:40.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056541559999999998 2024-09-17T21:30:40.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:40.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184097+0000", 2024-09-17T21:30:40.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057899029999999999 2024-09-17T21:30:40.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:40.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184172+0000", 2024-09-17T21:30:40.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5226000000000001e-05 2024-09-17T21:30:40.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184247+0000", 2024-09-17T21:30:40.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4574999999999997e-05 2024-09-17T21:30:40.536 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.536 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.536 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.536 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50203 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [truncate 20971520] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.536 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:40.536 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.400577659, 2024-09-17T21:30:40.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065009779000000004, 2024-09-17T21:30:40.536 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.536 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.537 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:40.537 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.537 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.537 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.537 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50203 2024-09-17T21:30:40.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.537 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:40.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:40.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119828+0000", 2024-09-17T21:30:40.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6390000000000001e-06 2024-09-17T21:30:40.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119832+0000", 2024-09-17T21:30:40.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7699999999999999e-06 2024-09-17T21:30:40.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119845+0000", 2024-09-17T21:30:40.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3093e-05 2024-09-17T21:30:40.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119852+0000", 2024-09-17T21:30:40.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6950000000000003e-06 2024-09-17T21:30:40.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.160945+0000", 2024-09-17T21:30:40.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041093076999999999 2024-09-17T21:30:40.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161071+0000", 2024-09-17T21:30:40.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012569799999999999 2024-09-17T21:30:40.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:40.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161283+0000", 2024-09-17T21:30:40.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021201999999999999 2024-09-17T21:30:40.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:40.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.179360+0000", 2024-09-17T21:30:40.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018077784999999999 2024-09-17T21:30:40.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:40.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184726+0000", 2024-09-17T21:30:40.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053652960000000003 2024-09-17T21:30:40.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:40.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184787+0000", 2024-09-17T21:30:40.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1402000000000006e-05 2024-09-17T21:30:40.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184836+0000", 2024-09-17T21:30:40.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9304000000000001e-05 2024-09-17T21:30:40.542 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.542 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.542 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.542 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50206 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.542 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:40.542 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.363496119000001, 2024-09-17T21:30:40.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.090361968000000001, 2024-09-17T21:30:40.542 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.542 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.542 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:40.543 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.543 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.543 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.543 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50206 2024-09-17T21:30:40.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.543 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:40.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:40.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156911+0000", 2024-09-17T21:30:40.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-17T21:30:40.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173670+0000", 2024-09-17T21:30:40.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016758907 2024-09-17T21:30:40.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173696+0000", 2024-09-17T21:30:40.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6329000000000002e-05 2024-09-17T21:30:40.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173708+0000", 2024-09-17T21:30:40.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2512e-05 2024-09-17T21:30:40.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173788+0000", 2024-09-17T21:30:40.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9332999999999994e-05 2024-09-17T21:30:40.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173918+0000", 2024-09-17T21:30:40.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013063300000000001 2024-09-17T21:30:40.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:40.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.176228+0000", 2024-09-17T21:30:40.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002309757 2024-09-17T21:30:40.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:40.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.245164+0000", 2024-09-17T21:30:40.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068936301000000005 2024-09-17T21:30:40.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:40.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247176+0000", 2024-09-17T21:30:40.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020117590000000001 2024-09-17T21:30:40.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:40.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247215+0000", 2024-09-17T21:30:40.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9094000000000002e-05 2024-09-17T21:30:40.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247270+0000", 2024-09-17T21:30:40.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4793999999999997e-05 2024-09-17T21:30:40.548 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.548 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.548 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.548 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50207 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [truncate 21556327] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.548 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:40.548 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.346647139000002, 2024-09-17T21:30:40.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.074055363999999999, 2024-09-17T21:30:40.549 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.549 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.549 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:40.549 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.549 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.549 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.549 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50207 2024-09-17T21:30:40.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.549 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:40.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:40.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173759+0000", 2024-09-17T21:30:40.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8160000000000001e-06 2024-09-17T21:30:40.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173763+0000", 2024-09-17T21:30:40.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5360000000000003e-06 2024-09-17T21:30:40.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173771+0000", 2024-09-17T21:30:40.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8849999999999999e-06 2024-09-17T21:30:40.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173777+0000", 2024-09-17T21:30:40.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7799999999999997e-06 2024-09-17T21:30:40.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221363+0000", 2024-09-17T21:30:40.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047585493 2024-09-17T21:30:40.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221430+0000", 2024-09-17T21:30:40.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7129000000000001e-05 2024-09-17T21:30:40.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:40.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221550+0000", 2024-09-17T21:30:40.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120658 2024-09-17T21:30:40.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:40.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246113+0000", 2024-09-17T21:30:40.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02456262 2024-09-17T21:30:40.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:40.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247744+0000", 2024-09-17T21:30:40.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001630692 2024-09-17T21:30:40.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:40.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247777+0000", 2024-09-17T21:30:40.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3015000000000002e-05 2024-09-17T21:30:40.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247812+0000", 2024-09-17T21:30:40.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.574e-05 2024-09-17T21:30:40.554 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.554 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.554 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.554 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50828 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.554 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:40.554 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.038535206999999, 2024-09-17T21:30:40.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047324185999999997, 2024-09-17T21:30:40.555 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.555 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.555 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.555 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.555 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.555 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.555 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50828 2024-09-17T21:30:40.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.555 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:40.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:40.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:40.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.46e-07 2024-09-17T21:30:40.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481871+0000", 2024-09-17T21:30:40.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T21:30:40.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481878+0000", 2024-09-17T21:30:40.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:30:40.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481882+0000", 2024-09-17T21:30:40.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:30:40.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529038+0000", 2024-09-17T21:30:40.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047156034999999999 2024-09-17T21:30:40.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529099+0000", 2024-09-17T21:30:40.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0316000000000003e-05 2024-09-17T21:30:40.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529193+0000", 2024-09-17T21:30:40.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4369000000000004e-05 2024-09-17T21:30:40.559 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.559 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.559 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.559 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50829 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 12845056~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.559 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:40.560 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.038516413, 2024-09-17T21:30:40.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047887596999999997, 2024-09-17T21:30:40.560 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.560 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.560 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.560 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.560 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.560 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.560 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50829 2024-09-17T21:30:40.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.560 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:40.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:40.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:40.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:30:40.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481890+0000", 2024-09-17T21:30:40.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.877e-06 2024-09-17T21:30:40.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481895+0000", 2024-09-17T21:30:40.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:30:40.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481900+0000", 2024-09-17T21:30:40.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4440000000000001e-06 2024-09-17T21:30:40.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529225+0000", 2024-09-17T21:30:40.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047325117999999999 2024-09-17T21:30:40.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529292+0000", 2024-09-17T21:30:40.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7100000000000005e-05 2024-09-17T21:30:40.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529775+0000", 2024-09-17T21:30:40.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048322799999999999 2024-09-17T21:30:40.563 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.564 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.564 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.564 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50830 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.564 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:40.564 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.038498011000001, 2024-09-17T21:30:40.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048062123999999998, 2024-09-17T21:30:40.564 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.564 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.564 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.564 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.565 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.565 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.565 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50830 2024-09-17T21:30:40.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.565 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:40.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:40.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481907+0000", 2024-09-17T21:30:40.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:30:40.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481909+0000", 2024-09-17T21:30:40.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T21:30:40.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481915+0000", 2024-09-17T21:30:40.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.02e-06 2024-09-17T21:30:40.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481920+0000", 2024-09-17T21:30:40.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6799999999999998e-06 2024-09-17T21:30:40.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529793+0000", 2024-09-17T21:30:40.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047873042999999997 2024-09-17T21:30:40.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529870+0000", 2024-09-17T21:30:40.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6967999999999999e-05 2024-09-17T21:30:40.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529968+0000", 2024-09-17T21:30:40.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8085999999999996e-05 2024-09-17T21:30:40.568 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.568 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.568 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.568 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50831 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 18087936~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.038477664999998, 2024-09-17T21:30:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048663304999999997, 2024-09-17T21:30:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50831 2024-09-17T21:30:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:40.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:40.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:40.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:30:40.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481929+0000", 2024-09-17T21:30:40.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2189999999999998e-06 2024-09-17T21:30:40.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481935+0000", 2024-09-17T21:30:40.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.383e-06 2024-09-17T21:30:40.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481939+0000", 2024-09-17T21:30:40.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2479999999999998e-06 2024-09-17T21:30:40.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530095+0000", 2024-09-17T21:30:40.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048156617999999998 2024-09-17T21:30:40.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530159+0000", 2024-09-17T21:30:40.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3394999999999999e-05 2024-09-17T21:30:40.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530590+0000", 2024-09-17T21:30:40.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043101199999999999 2024-09-17T21:30:40.573 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.573 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.573 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.573 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50832 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 19660800~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.573 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:40.573 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.038459018000001, 2024-09-17T21:30:40.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049339466999999998, 2024-09-17T21:30:40.573 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.573 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.573 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.574 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.574 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.574 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.574 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50832 2024-09-17T21:30:40.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.574 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:40.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:40.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481946+0000", 2024-09-17T21:30:40.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:40.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481951+0000", 2024-09-17T21:30:40.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1479999999999997e-06 2024-09-17T21:30:40.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481956+0000", 2024-09-17T21:30:40.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7339999999999996e-06 2024-09-17T21:30:40.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481961+0000", 2024-09-17T21:30:40.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4889999999999997e-06 2024-09-17T21:30:40.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530630+0000", 2024-09-17T21:30:40.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048668596000000001 2024-09-17T21:30:40.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530720+0000", 2024-09-17T21:30:40.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0163999999999999e-05 2024-09-17T21:30:40.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531285+0000", 2024-09-17T21:30:40.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056493199999999998 2024-09-17T21:30:40.577 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.577 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.577 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.578 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50833 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.578 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:40.578 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.038436873999999, 2024-09-17T21:30:40.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049908069999999999, 2024-09-17T21:30:40.578 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.578 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.578 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.578 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.578 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.578 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.578 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50833 2024-09-17T21:30:40.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.579 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:40.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:40.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481968+0000", 2024-09-17T21:30:40.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5600000000000001e-07 2024-09-17T21:30:40.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481970+0000", 2024-09-17T21:30:40.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:30:40.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481975+0000", 2024-09-17T21:30:40.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8499999999999999e-06 2024-09-17T21:30:40.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481980+0000", 2024-09-17T21:30:40.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-17T21:30:40.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531337+0000", 2024-09-17T21:30:40.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049356874000000002 2024-09-17T21:30:40.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531392+0000", 2024-09-17T21:30:40.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4826e-05 2024-09-17T21:30:40.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531875+0000", 2024-09-17T21:30:40.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048345099999999998 2024-09-17T21:30:40.582 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.582 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.582 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.582 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50834 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 24903680~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.582 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:40.582 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.038412105999999, 2024-09-17T21:30:40.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050534015000000002, 2024-09-17T21:30:40.583 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.583 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.583 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.583 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50834 2024-09-17T21:30:40.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.583 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:40.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:40.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481993+0000", 2024-09-17T21:30:40.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:30:40.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481995+0000", 2024-09-17T21:30:40.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.199e-06 2024-09-17T21:30:40.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482001+0000", 2024-09-17T21:30:40.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.772e-06 2024-09-17T21:30:40.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482006+0000", 2024-09-17T21:30:40.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0660000000000003e-06 2024-09-17T21:30:40.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531923+0000", 2024-09-17T21:30:40.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049917200000000002 2024-09-17T21:30:40.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532004+0000", 2024-09-17T21:30:40.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0758000000000002e-05 2024-09-17T21:30:40.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532526+0000", 2024-09-17T21:30:40.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00052258899999999995 2024-09-17T21:30:40.586 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.586 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.586 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.587 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50835 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 28573696~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.587 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:40.587 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.038392123000001, 2024-09-17T21:30:40.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051204701999999998, 2024-09-17T21:30:40.587 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.587 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.587 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.587 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.587 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.587 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.587 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50835 2024-09-17T21:30:40.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.588 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:40.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:40.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482013+0000", 2024-09-17T21:30:40.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:30:40.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482015+0000", 2024-09-17T21:30:40.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.272e-06 2024-09-17T21:30:40.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482020+0000", 2024-09-17T21:30:40.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3040000000000004e-06 2024-09-17T21:30:40.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482025+0000", 2024-09-17T21:30:40.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5650000000000003e-06 2024-09-17T21:30:40.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532569+0000", 2024-09-17T21:30:40.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050544661999999997 2024-09-17T21:30:40.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532649+0000", 2024-09-17T21:30:40.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9640000000000003e-05 2024-09-17T21:30:40.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533217+0000", 2024-09-17T21:30:40.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056784100000000003 2024-09-17T21:30:40.591 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.591 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.591 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.591 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50836 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 30146560~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.591 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:40.591 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.038372506000002, 2024-09-17T21:30:40.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051875589, 2024-09-17T21:30:40.591 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.591 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.591 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.592 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.592 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.592 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.592 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50836 2024-09-17T21:30:40.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.592 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:40.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:40.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:40.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3700000000000001e-07 2024-09-17T21:30:40.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482034+0000", 2024-09-17T21:30:40.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0059999999999999e-06 2024-09-17T21:30:40.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482040+0000", 2024-09-17T21:30:40.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:30:40.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482044+0000", 2024-09-17T21:30:40.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:30:40.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533260+0000", 2024-09-17T21:30:40.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051215416 2024-09-17T21:30:40.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533348+0000", 2024-09-17T21:30:40.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8695000000000004e-05 2024-09-17T21:30:40.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533907+0000", 2024-09-17T21:30:40.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055902199999999995 2024-09-17T21:30:40.595 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.595 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.595 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.595 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50844 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.595 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:40.595 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.038209080000001, 2024-09-17T21:30:40.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0475644, 2024-09-17T21:30:40.595 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.596 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.596 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.596 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.596 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.596 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.596 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50844 2024-09-17T21:30:40.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.596 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:40.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:40.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482196+0000", 2024-09-17T21:30:40.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.39e-07 2024-09-17T21:30:40.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482198+0000", 2024-09-17T21:30:40.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0099999999999998e-06 2024-09-17T21:30:40.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482204+0000", 2024-09-17T21:30:40.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.93e-06 2024-09-17T21:30:40.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482208+0000", 2024-09-17T21:30:40.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7439999999999998e-06 2024-09-17T21:30:40.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529318+0000", 2024-09-17T21:30:40.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047110051999999999 2024-09-17T21:30:40.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529396+0000", 2024-09-17T21:30:40.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7712000000000004e-05 2024-09-17T21:30:40.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529760+0000", 2024-09-17T21:30:40.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00036351299999999999 2024-09-17T21:30:40.599 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.599 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.600 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.600 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50845 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 21757952~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.600 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:40.600 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.038189976999998, 2024-09-17T21:30:40.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048084625999999998, 2024-09-17T21:30:40.600 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.600 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.600 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.600 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.600 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.600 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.601 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50845 2024-09-17T21:30:40.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.601 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:40.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:40.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482215+0000", 2024-09-17T21:30:40.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:30:40.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482216+0000", 2024-09-17T21:30:40.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.823e-06 2024-09-17T21:30:40.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482222+0000", 2024-09-17T21:30:40.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.507e-06 2024-09-17T21:30:40.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482227+0000", 2024-09-17T21:30:40.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6609999999999999e-06 2024-09-17T21:30:40.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529792+0000", 2024-09-17T21:30:40.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047565075999999998 2024-09-17T21:30:40.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529871+0000", 2024-09-17T21:30:40.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8855000000000003e-05 2024-09-17T21:30:40.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530299+0000", 2024-09-17T21:30:40.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000428307 2024-09-17T21:30:40.604 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.604 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.604 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.605 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50846 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 25427968~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.605 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:40.605 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.03816759, 2024-09-17T21:30:40.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048735142000000002, 2024-09-17T21:30:40.605 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.605 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.605 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.605 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.605 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.605 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.605 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50846 2024-09-17T21:30:40.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.605 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:40.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:40.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:40.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:30:40.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482239+0000", 2024-09-17T21:30:40.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.871e-06 2024-09-17T21:30:40.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482244+0000", 2024-09-17T21:30:40.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.49e-06 2024-09-17T21:30:40.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482249+0000", 2024-09-17T21:30:40.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6269999999999999e-06 2024-09-17T21:30:40.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530347+0000", 2024-09-17T21:30:40.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048097721000000003 2024-09-17T21:30:40.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530437+0000", 2024-09-17T21:30:40.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0604000000000002e-05 2024-09-17T21:30:40.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530972+0000", 2024-09-17T21:30:40.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053445000000000001 2024-09-17T21:30:40.609 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.609 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.609 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.609 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50847 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 27000832~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.609 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:40.609 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.038149694000001, 2024-09-17T21:30:40.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049308668, 2024-09-17T21:30:40.609 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.609 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.609 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.609 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.610 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.610 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.610 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50847 2024-09-17T21:30:40.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.610 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:40.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:40.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:40.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:30:40.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482257+0000", 2024-09-17T21:30:40.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.846e-06 2024-09-17T21:30:40.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482264+0000", 2024-09-17T21:30:40.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8170000000000001e-06 2024-09-17T21:30:40.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482283+0000", 2024-09-17T21:30:40.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9718e-05 2024-09-17T21:30:40.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531018+0000", 2024-09-17T21:30:40.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048734497000000002 2024-09-17T21:30:40.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531080+0000", 2024-09-17T21:30:40.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1795e-05 2024-09-17T21:30:40.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531563+0000", 2024-09-17T21:30:40.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048366499999999998 2024-09-17T21:30:40.613 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.613 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.614 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.614 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50848 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 30670848~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.614 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:40.614 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.038113152000001, 2024-09-17T21:30:40.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049893337000000003, 2024-09-17T21:30:40.614 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.614 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.614 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.614 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.614 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.614 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.614 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50848 2024-09-17T21:30:40.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.615 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:40.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:40.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482292+0000", 2024-09-17T21:30:40.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T21:30:40.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482294+0000", 2024-09-17T21:30:40.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3209999999999999e-06 2024-09-17T21:30:40.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482300+0000", 2024-09-17T21:30:40.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7189999999999998e-06 2024-09-17T21:30:40.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482305+0000", 2024-09-17T21:30:40.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T21:30:40.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531606+0000", 2024-09-17T21:30:40.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049301273999999999 2024-09-17T21:30:40.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531696+0000", 2024-09-17T21:30:40.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0161999999999995e-05 2024-09-17T21:30:40.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532184+0000", 2024-09-17T21:30:40.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048806400000000002 2024-09-17T21:30:40.618 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.618 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.618 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.618 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50849 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 32243712~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:40.618 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:40.618 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.038093710999998, 2024-09-17T21:30:40.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050444607000000002, 2024-09-17T21:30:40.618 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:40.618 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:40.618 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:40.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:40.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:40.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:40.619 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50849 2024-09-17T21:30:40.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.619 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:40.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:40.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:40.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:40.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:40.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:40.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:40.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482311+0000", 2024-09-17T21:30:40.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:30:40.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:40.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482316+0000", 2024-09-17T21:30:40.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1209999999999998e-06 2024-09-17T21:30:40.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:40.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482321+0000", 2024-09-17T21:30:40.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4779999999999999e-06 2024-09-17T21:30:40.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:40.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482326+0000", 2024-09-17T21:30:40.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7099999999999998e-06 2024-09-17T21:30:40.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:40.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532221+0000", 2024-09-17T21:30:40.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049894936000000001 2024-09-17T21:30:40.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:40.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532287+0000", 2024-09-17T21:30:40.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5914000000000006e-05 2024-09-17T21:30:40.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:40.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:40.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:40.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532755+0000", 2024-09-17T21:30:40.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000468107 2024-09-17T21:30:40.622 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.622 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.622 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.622 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:40.622 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:40.622 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:40.624 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:30:40.624 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:40.621+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:40.724 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:40.722+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:40.768 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:40.768 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:40.768 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:40.768 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:40.777 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:30:40.809 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is set 2024-09-17T21:30:40.824 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:40.822+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:40.889 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:40.889 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:40.889 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:40.889 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:40.889 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:40.899 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:30:40.926 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:40.923+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:41.009 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-17T21:30:41.012 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "pg_update_log_missing(3.5 epoch 45/37 rep_tid 844 entries 45'73 (0'0) error 3:a65c40ae:::rbd_data.1120b21c3309.0000000000000126:head by client.4384.0:48900 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 45'70)", 2024-09-17T21:30:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.962060336999997, 2024-09-17T21:30:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019554365000000001, 2024-09-17T21:30:41.012 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T21:30:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047112+0000", 2024-09-17T21:30:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:30:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047121+0000", 2024-09-17T21:30:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4379999999999995e-06 2024-09-17T21:30:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047139+0000", 2024-09-17T21:30:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8235000000000001e-05 2024-09-17T21:30:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047303+0000", 2024-09-17T21:30:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016389400000000001 2024-09-17T21:30:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047396+0000", 2024-09-17T21:30:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3355000000000003e-05 2024-09-17T21:30:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066666+0000", 2024-09-17T21:30:41.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019269649999999999 2024-09-17T21:30:41.016 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.016 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.016 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.016 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48906 3.4 e45/25 3:2503fe27:::rbd_data.1120b21c3309.0000000000000127:head v 45'148, mlcod=45'146)", 2024-09-17T21:30:41.016 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:41.016 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.959934158000003, 2024-09-17T21:30:41.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018169739000000001, 2024-09-17T21:30:41.016 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.017 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.017 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:41.017 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:41.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:41.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:41.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:30:41.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049244+0000", 2024-09-17T21:30:41.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8390000000000003e-06 2024-09-17T21:30:41.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066411+0000", 2024-09-17T21:30:41.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017166818 2024-09-17T21:30:41.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066426+0000", 2024-09-17T21:30:41.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4888999999999999e-05 2024-09-17T21:30:41.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066611+0000", 2024-09-17T21:30:41.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018464499999999999 2024-09-17T21:30:41.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066630+0000", 2024-09-17T21:30:41.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9201000000000001e-05 2024-09-17T21:30:41.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:41.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067373+0000", 2024-09-17T21:30:41.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00074322600000000004 2024-09-17T21:30:41.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067407+0000", 2024-09-17T21:30:41.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4357999999999999e-05 2024-09-17T21:30:41.020 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.021 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.021 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.021 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50206 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'301, mlcod=45'299)", 2024-09-17T21:30:41.021 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:41.021 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.830457314, 2024-09-17T21:30:41.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068285810000000002, 2024-09-17T21:30:41.021 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.021 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.021 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:41.021 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:41.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:41.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178716+0000", 2024-09-17T21:30:41.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.438e-06 2024-09-17T21:30:41.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192044+0000", 2024-09-17T21:30:41.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013328370000000001 2024-09-17T21:30:41.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192069+0000", 2024-09-17T21:30:41.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4496000000000001e-05 2024-09-17T21:30:41.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192085+0000", 2024-09-17T21:30:41.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5758999999999999e-05 2024-09-17T21:30:41.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192167+0000", 2024-09-17T21:30:41.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2842000000000004e-05 2024-09-17T21:30:41.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192201+0000", 2024-09-17T21:30:41.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3127999999999999e-05 2024-09-17T21:30:41.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:41.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246938+0000", 2024-09-17T21:30:41.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054736982000000003 2024-09-17T21:30:41.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247000+0000", 2024-09-17T21:30:41.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2794999999999998e-05 2024-09-17T21:30:41.025 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.025 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.025 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.026 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50207 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'303, mlcod=45'299)", 2024-09-17T21:30:41.026 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:41.026 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.787510265000002, 2024-09-17T21:30:41.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026003433999999999, 2024-09-17T21:30:41.026 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.026 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.026 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:41.026 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:41.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:41.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221663+0000", 2024-09-17T21:30:41.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.271e-06 2024-09-17T21:30:41.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221672+0000", 2024-09-17T21:30:41.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6810000000000004e-06 2024-09-17T21:30:41.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221684+0000", 2024-09-17T21:30:41.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:30:41.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221691+0000", 2024-09-17T21:30:41.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:30:41.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221743+0000", 2024-09-17T21:30:41.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2868999999999998e-05 2024-09-17T21:30:41.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221765+0000", 2024-09-17T21:30:41.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1316999999999999e-05 2024-09-17T21:30:41.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:41.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247616+0000", 2024-09-17T21:30:41.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025851504000000001 2024-09-17T21:30:41.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247665+0000", 2024-09-17T21:30:41.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8739000000000002e-05 2024-09-17T21:30:41.030 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.030 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.030 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.030 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57808 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 196608~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:41.031 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:41.031 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.231211224999999, 2024-09-17T21:30:41.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017751475999999999, 2024-09-17T21:30:41.031 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.031 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.031 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:41.031 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:41.031 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:41.031 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:41.031 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57808 2024-09-17T21:30:41.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.031 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:41.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:41.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:41.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:30:41.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777968+0000", 2024-09-17T21:30:41.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8530000000000003e-06 2024-09-17T21:30:41.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777975+0000", 2024-09-17T21:30:41.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4410000000000004e-06 2024-09-17T21:30:41.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777980+0000", 2024-09-17T21:30:41.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7210000000000002e-06 2024-09-17T21:30:41.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795323+0000", 2024-09-17T21:30:41.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017343319999999999 2024-09-17T21:30:41.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795410+0000", 2024-09-17T21:30:41.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6567e-05 2024-09-17T21:30:41.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795712+0000", 2024-09-17T21:30:41.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030219200000000001 2024-09-17T21:30:41.035 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.035 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.035 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.035 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57809 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 458752~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:41.035 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:30:41.035 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.231185372999999, 2024-09-17T21:30:41.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018092869000000001, 2024-09-17T21:30:41.036 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.036 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.036 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:41.036 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:41.036 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:41.036 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:41.036 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57809 2024-09-17T21:30:41.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.037 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:30:41.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:30:41.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777987+0000", 2024-09-17T21:30:41.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T21:30:41.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777989+0000", 2024-09-17T21:30:41.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8649999999999999e-06 2024-09-17T21:30:41.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777994+0000", 2024-09-17T21:30:41.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4199999999999998e-06 2024-09-17T21:30:41.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778004+0000", 2024-09-17T21:30:41.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6900000000000004e-06 2024-09-17T21:30:41.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795745+0000", 2024-09-17T21:30:41.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017740859000000001 2024-09-17T21:30:41.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795815+0000", 2024-09-17T21:30:41.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0219e-05 2024-09-17T21:30:41.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796079+0000", 2024-09-17T21:30:41.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000264446 2024-09-17T21:30:41.040 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.040 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.040 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.041 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57810 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 720896~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:41.041 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:30:41.041 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.231161646, 2024-09-17T21:30:41.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018361253000000001, 2024-09-17T21:30:41.041 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.041 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.041 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:41.041 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:41.041 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:41.041 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:41.041 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57810 2024-09-17T21:30:41.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.041 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:30:41.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:30:41.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778011+0000", 2024-09-17T21:30:41.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:41.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778012+0000", 2024-09-17T21:30:41.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.872e-06 2024-09-17T21:30:41.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778018+0000", 2024-09-17T21:30:41.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8150000000000002e-06 2024-09-17T21:30:41.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778023+0000", 2024-09-17T21:30:41.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3939999999999998e-06 2024-09-17T21:30:41.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796106+0000", 2024-09-17T21:30:41.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018083446999999999 2024-09-17T21:30:41.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796159+0000", 2024-09-17T21:30:41.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3029999999999999e-05 2024-09-17T21:30:41.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796371+0000", 2024-09-17T21:30:41.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021231700000000001 2024-09-17T21:30:41.045 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.045 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.045 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.045 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57811 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:41.045 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:41.045 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.231143226, 2024-09-17T21:30:41.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018666035000000001, 2024-09-17T21:30:41.045 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.045 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.045 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:41.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:41.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:41.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:41.046 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57811 2024-09-17T21:30:41.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.046 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:41.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:41.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:41.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:30:41.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778031+0000", 2024-09-17T21:30:41.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.796e-06 2024-09-17T21:30:41.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778036+0000", 2024-09-17T21:30:41.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3290000000000001e-06 2024-09-17T21:30:41.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778041+0000", 2024-09-17T21:30:41.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5839999999999996e-06 2024-09-17T21:30:41.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796409+0000", 2024-09-17T21:30:41.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018368703 2024-09-17T21:30:41.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796483+0000", 2024-09-17T21:30:41.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3751999999999995e-05 2024-09-17T21:30:41.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796695+0000", 2024-09-17T21:30:41.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021153 2024-09-17T21:30:41.049 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.049 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.049 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.050 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57812 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1245184~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:41.050 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:41.050 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.231124329, 2024-09-17T21:30:41.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019000064000000001, 2024-09-17T21:30:41.050 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.050 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.050 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:41.050 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:41.050 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:41.050 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:41.050 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57812 2024-09-17T21:30:41.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.050 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:41.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:41.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:41.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:30:41.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778050+0000", 2024-09-17T21:30:41.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7e-06 2024-09-17T21:30:41.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778055+0000", 2024-09-17T21:30:41.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4049999999999999e-06 2024-09-17T21:30:41.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778059+0000", 2024-09-17T21:30:41.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1479999999999999e-06 2024-09-17T21:30:41.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796734+0000", 2024-09-17T21:30:41.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018674399000000001 2024-09-17T21:30:41.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796786+0000", 2024-09-17T21:30:41.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2879e-05 2024-09-17T21:30:41.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797048+0000", 2024-09-17T21:30:41.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026118400000000003 2024-09-17T21:30:41.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.054 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.054 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57813 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1507328~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:41.054 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:30:41.054 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.231106437000001, 2024-09-17T21:30:41.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019279392999999999, 2024-09-17T21:30:41.054 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.054 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.054 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:41.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:41.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:41.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:41.055 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57813 2024-09-17T21:30:41.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.055 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:30:41.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:30:41.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778066+0000", 2024-09-17T21:30:41.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6199999999999999e-07 2024-09-17T21:30:41.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778068+0000", 2024-09-17T21:30:41.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7990000000000001e-06 2024-09-17T21:30:41.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778074+0000", 2024-09-17T21:30:41.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:30:41.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778078+0000", 2024-09-17T21:30:41.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2579999999999997e-06 2024-09-17T21:30:41.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797077+0000", 2024-09-17T21:30:41.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018999180000000001 2024-09-17T21:30:41.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797123+0000", 2024-09-17T21:30:41.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5834999999999997e-05 2024-09-17T21:30:41.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797345+0000", 2024-09-17T21:30:41.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022191100000000001 2024-09-17T21:30:41.058 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.058 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.058 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.059 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57814 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:41.059 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:30:41.059 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.231087564999999, 2024-09-17T21:30:41.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019604947000000001, 2024-09-17T21:30:41.059 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.059 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.059 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:41.059 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:41.059 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:41.059 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:41.059 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57814 2024-09-17T21:30:41.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.059 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:30:41.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:30:41.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778085+0000", 2024-09-17T21:30:41.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:41.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778086+0000", 2024-09-17T21:30:41.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7850000000000001e-06 2024-09-17T21:30:41.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778093+0000", 2024-09-17T21:30:41.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6449999999999999e-06 2024-09-17T21:30:41.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778098+0000", 2024-09-17T21:30:41.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:30:41.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797383+0000", 2024-09-17T21:30:41.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019284879000000001 2024-09-17T21:30:41.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797431+0000", 2024-09-17T21:30:41.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8856000000000001e-05 2024-09-17T21:30:41.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797689+0000", 2024-09-17T21:30:41.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025785800000000002 2024-09-17T21:30:41.063 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.063 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.063 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.063 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57815 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2031616~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:41.063 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:41.063 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.231066897000002, 2024-09-17T21:30:41.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019910231, 2024-09-17T21:30:41.063 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.063 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.063 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:41.063 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:41.063 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:41.064 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:41.064 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57815 2024-09-17T21:30:41.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.064 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:41.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:41.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:41.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:30:41.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778118+0000", 2024-09-17T21:30:41.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2967e-05 2024-09-17T21:30:41.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778125+0000", 2024-09-17T21:30:41.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.601e-06 2024-09-17T21:30:41.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778130+0000", 2024-09-17T21:30:41.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1839999999999998e-06 2024-09-17T21:30:41.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797720+0000", 2024-09-17T21:30:41.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019589766000000002 2024-09-17T21:30:41.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797765+0000", 2024-09-17T21:30:41.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5281999999999998e-05 2024-09-17T21:30:41.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798015+0000", 2024-09-17T21:30:41.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025006200000000001 2024-09-17T21:30:41.067 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.067 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.067 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.067 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57816 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2293760~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:41.067 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:41.067 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.231034303000001, 2024-09-17T21:30:41.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020175322999999998, 2024-09-17T21:30:41.067 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.067 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.068 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:41.068 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:41.068 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:41.068 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:41.068 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57816 2024-09-17T21:30:41.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.068 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:41.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:41.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:41.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:30:41.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778140+0000", 2024-09-17T21:30:41.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T21:30:41.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778145+0000", 2024-09-17T21:30:41.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7320000000000003e-06 2024-09-17T21:30:41.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778154+0000", 2024-09-17T21:30:41.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6969999999999999e-06 2024-09-17T21:30:41.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798049+0000", 2024-09-17T21:30:41.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019894603 2024-09-17T21:30:41.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798106+0000", 2024-09-17T21:30:41.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7207000000000003e-05 2024-09-17T21:30:41.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798313+0000", 2024-09-17T21:30:41.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206861 2024-09-17T21:30:41.071 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.071 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.071 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.072 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57817 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2555904~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:41.072 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:41.072 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.231011057, 2024-09-17T21:30:41.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020466314999999999, 2024-09-17T21:30:41.072 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.072 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.072 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:41.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:41.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:41.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:41.073 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57817 2024-09-17T21:30:41.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.073 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:41.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:41.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:41.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2399999999999999e-07 2024-09-17T21:30:41.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778163+0000", 2024-09-17T21:30:41.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.204e-06 2024-09-17T21:30:41.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778169+0000", 2024-09-17T21:30:41.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:30:41.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778173+0000", 2024-09-17T21:30:41.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:30:41.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798342+0000", 2024-09-17T21:30:41.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020168416000000002 2024-09-17T21:30:41.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798388+0000", 2024-09-17T21:30:41.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6171000000000002e-05 2024-09-17T21:30:41.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798627+0000", 2024-09-17T21:30:41.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000239058 2024-09-17T21:30:41.076 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.076 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.076 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.076 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57818 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2818048~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:41.076 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:30:41.077 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.230992359999998, 2024-09-17T21:30:41.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020767240999999999, 2024-09-17T21:30:41.077 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.077 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.077 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:41.077 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:41.077 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:41.077 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:41.077 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57818 2024-09-17T21:30:41.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.077 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:30:41.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:30:41.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778180+0000", 2024-09-17T21:30:41.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:30:41.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778182+0000", 2024-09-17T21:30:41.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7269999999999999e-06 2024-09-17T21:30:41.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778188+0000", 2024-09-17T21:30:41.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6080000000000001e-06 2024-09-17T21:30:41.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778202+0000", 2024-09-17T21:30:41.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3369e-05 2024-09-17T21:30:41.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798660+0000", 2024-09-17T21:30:41.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020458171000000001 2024-09-17T21:30:41.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798704+0000", 2024-09-17T21:30:41.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4273000000000001e-05 2024-09-17T21:30:41.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798947+0000", 2024-09-17T21:30:41.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242651 2024-09-17T21:30:41.081 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.081 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.081 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.081 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57819 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:41.081 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:30:41.081 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.230961490999999, 2024-09-17T21:30:41.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020892340999999998, 2024-09-17T21:30:41.082 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.082 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.082 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:41.082 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:41.082 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:41.082 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:41.082 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57819 2024-09-17T21:30:41.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.082 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:30:41.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:30:41.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778211+0000", 2024-09-17T21:30:41.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:30:41.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778217+0000", 2024-09-17T21:30:41.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T21:30:41.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778224+0000", 2024-09-17T21:30:41.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3520000000000003e-06 2024-09-17T21:30:41.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778228+0000", 2024-09-17T21:30:41.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7480000000000001e-06 2024-09-17T21:30:41.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798977+0000", 2024-09-17T21:30:41.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020748571 2024-09-17T21:30:41.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799035+0000", 2024-09-17T21:30:41.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7908999999999997e-05 2024-09-17T21:30:41.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799103+0000", 2024-09-17T21:30:41.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7794999999999997e-05 2024-09-17T21:30:41.085 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.086 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.086 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.086 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57820 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:41.086 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:41.086 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.230936299, 2024-09-17T21:30:41.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021016658000000001, 2024-09-17T21:30:41.086 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.086 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.086 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:41.086 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:41.086 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:41.086 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:41.087 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57820 2024-09-17T21:30:41.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.087 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:41.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:41.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:41.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:30:41.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778238+0000", 2024-09-17T21:30:41.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.945e-06 2024-09-17T21:30:41.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778244+0000", 2024-09-17T21:30:41.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6520000000000003e-06 2024-09-17T21:30:41.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778248+0000", 2024-09-17T21:30:41.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5920000000000002e-06 2024-09-17T21:30:41.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799131+0000", 2024-09-17T21:30:41.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020883288999999999 2024-09-17T21:30:41.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799185+0000", 2024-09-17T21:30:41.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3223000000000002e-05 2024-09-17T21:30:41.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799252+0000", 2024-09-17T21:30:41.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7576999999999994e-05 2024-09-17T21:30:41.090 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.090 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.090 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.090 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57821 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:41.090 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:41.090 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.230917953999999, 2024-09-17T21:30:41.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021153528000000001, 2024-09-17T21:30:41.090 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.091 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.091 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:41.091 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:41.091 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:41.091 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:41.091 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57821 2024-09-17T21:30:41.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.091 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:41.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:41.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:41.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-07 2024-09-17T21:30:41.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778256+0000", 2024-09-17T21:30:41.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.604e-06 2024-09-17T21:30:41.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778262+0000", 2024-09-17T21:30:41.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7629999999999997e-06 2024-09-17T21:30:41.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778266+0000", 2024-09-17T21:30:41.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:30:41.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799284+0000", 2024-09-17T21:30:41.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021017312 2024-09-17T21:30:41.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799326+0000", 2024-09-17T21:30:41.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2639000000000003e-05 2024-09-17T21:30:41.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799407+0000", 2024-09-17T21:30:41.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1077999999999999e-05 2024-09-17T21:30:41.094 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.094 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.094 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.095 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57822 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:41.095 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:30:41.095 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.230899644000001, 2024-09-17T21:30:41.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021268966, 2024-09-17T21:30:41.095 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.095 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.095 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:41.095 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:41.095 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:41.095 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:41.095 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57822 2024-09-17T21:30:41.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.095 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:30:41.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:30:41.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778273+0000", 2024-09-17T21:30:41.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:30:41.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778274+0000", 2024-09-17T21:30:41.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.598e-06 2024-09-17T21:30:41.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778283+0000", 2024-09-17T21:30:41.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8279999999999992e-06 2024-09-17T21:30:41.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778289+0000", 2024-09-17T21:30:41.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5609999999999998e-06 2024-09-17T21:30:41.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799432+0000", 2024-09-17T21:30:41.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021143862999999999 2024-09-17T21:30:41.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799473+0000", 2024-09-17T21:30:41.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1075999999999998e-05 2024-09-17T21:30:41.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799541+0000", 2024-09-17T21:30:41.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7657999999999995e-05 2024-09-17T21:30:41.098 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.099 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.099 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.099 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57823 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:41.099 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:41.099 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.230876683000002, 2024-09-17T21:30:41.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021379770999999999, 2024-09-17T21:30:41.099 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:41.099 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:41.099 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:41.099 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:41.099 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:41.099 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:41.100 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57823 2024-09-17T21:30:41.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.100 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:41.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:41.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:41.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:41.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:41.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:41.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:41.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:41.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:30:41.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:41.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778297+0000", 2024-09-17T21:30:41.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:30:41.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:41.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778303+0000", 2024-09-17T21:30:41.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4559999999999999e-06 2024-09-17T21:30:41.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:41.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778307+0000", 2024-09-17T21:30:41.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6229999999999996e-06 2024-09-17T21:30:41.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:41.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799560+0000", 2024-09-17T21:30:41.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021252931999999999 2024-09-17T21:30:41.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:41.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799609+0000", 2024-09-17T21:30:41.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8301999999999999e-05 2024-09-17T21:30:41.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:41.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:41.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:41.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799675+0000", 2024-09-17T21:30:41.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6205999999999996e-05 2024-09-17T21:30:41.103 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.103 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.103 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.103 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:41.103 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:41.103 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:41.104 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:30:41.105 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:30:41.106 INFO:teuthology.orchestra.run.smithi129.stdout:4617 read 0xd3b1142 thru 0xd3b8c2e (0x7aed bytes) 2024-09-17T21:30:41.106 INFO:teuthology.orchestra.run.smithi129.stdout:4619 write 0x7be4a78 thru 0x7bea2a3 (0x582c bytes) 2024-09-17T21:30:41.107 INFO:teuthology.orchestra.run.smithi129.stdout:4620 read 0xc7f8a4c thru 0xc8017d6 (0x8d8b bytes) 2024-09-17T21:30:41.107 INFO:teuthology.orchestra.run.smithi129.stdout:4621 punch from 0x35f66f1 to 0x35f99fb, (0x330a bytes) 2024-09-17T21:30:41.107 INFO:teuthology.orchestra.run.smithi129.stdout:4622 trunc from 0xd746037 to 0x8c07df7 2024-09-17T21:30:41.107 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:41.107 INFO:teuthology.orchestra.run.smithi129.stdout:4625 write 0x1011e73 thru 0x101c82f (0xa9bd bytes) 2024-09-17T21:30:41.107 INFO:teuthology.orchestra.run.smithi129.stdout:4626 write 0xc628ed thru 0xc6bf91 (0x96a5 bytes) 2024-09-17T21:30:41.107 INFO:teuthology.orchestra.run.smithi129.stdout:4627 trunc from 0x8c07df7 to 0x3b20e09 2024-09-17T21:30:41.107 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:41.102+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:41.152 INFO:teuthology.orchestra.run.smithi129.stdout:4628 writesame 0xba5eff7 thru 0xba6532b data_size 0x1(0x6335 bytes) 2024-09-17T21:30:41.152 INFO:teuthology.orchestra.run.smithi129.stdout:4629 writesame 0x553cf8a thru 0x553e5a4 data_size 0x1(0x161b bytes) 2024-09-17T21:30:41.155 INFO:teuthology.orchestra.run.smithi129.stdout:4630 read 0xb688043 thru 0xb694af6 (0xcab4 bytes) 2024-09-17T21:30:41.155 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:41.162 INFO:teuthology.orchestra.run.smithi129.stdout:4633 write 0xd4d02de thru 0xd4d2b8e (0x28b1 bytes) 2024-09-17T21:30:41.165 INFO:teuthology.orchestra.run.smithi129.stdout:4635 read 0x36a7f41 thru 0x36b4297 (0xc357 bytes) 2024-09-17T21:30:41.166 INFO:teuthology.orchestra.run.smithi129.stdout:4636 write 0xa8de9d2 thru 0xa8ee947 (0xff76 bytes) 2024-09-17T21:30:41.169 INFO:teuthology.orchestra.run.smithi129.stdout:4638 write 0x902004b thru 0x9026381 (0x6337 bytes) 2024-09-17T21:30:41.173 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:41.173 INFO:teuthology.orchestra.run.smithi129.stdout:4640 read 0x1e41d67 thru 0x1e4810e (0x63a8 bytes) 2024-09-17T21:30:41.173 INFO:teuthology.orchestra.run.smithi129.stdout:4641 read 0x5665a0e thru 0x56671be (0x17b1 bytes) 2024-09-17T21:30:41.173 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:41.173 INFO:teuthology.orchestra.run.smithi129.stdout:4644 write 0xa2afce4 thru 0xa2b38bb (0x3bd8 bytes) 2024-09-17T21:30:41.178 INFO:teuthology.orchestra.run.smithi129.stdout:4646 punch from 0x3b856 to 0x465a4, (0xad4e bytes) 2024-09-17T21:30:41.178 INFO:teuthology.orchestra.run.smithi129.stdout:4648 writesame 0xc349ea4 thru 0xc34f58a data_size 0x1(0x56e7 bytes) 2024-09-17T21:30:41.190 INFO:teuthology.orchestra.run.smithi129.stdout:4649 read 0x139d11 thru 0x143cb3 (0x9fa3 bytes) 2024-09-17T21:30:41.193 INFO:teuthology.orchestra.run.smithi129.stdout:4650 trunc from 0xd4d2b8f to 0xb1c7222 2024-09-17T21:30:41.200 INFO:teuthology.orchestra.run.smithi129.stdout:4651 writesame 0x5c30978 thru 0x5c324ec data_size 0x21(0x1b75 bytes) 2024-09-17T21:30:41.202 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:41.202 INFO:teuthology.orchestra.run.smithi129.stdout:4654 write 0x5ebfb4 thru 0x5f590f (0x995c bytes) 2024-09-17T21:30:41.206 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:41.203+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:41.211 INFO:teuthology.orchestra.run.smithi129.stdout:4655 read 0x7f49e41 thru 0x7f4ab12 (0xcd2 bytes) 2024-09-17T21:30:41.211 INFO:teuthology.orchestra.run.smithi129.stdout:4657 writesame 0x489827e thru 0x48a2a08 data_size 0x33(0xa78b bytes) 2024-09-17T21:30:41.212 INFO:teuthology.orchestra.run.smithi129.stdout:4659 read 0x30de1bf thru 0x30ee064 (0xfea6 bytes) 2024-09-17T21:30:41.213 INFO:teuthology.orchestra.run.smithi129.stdout:4661 punch from 0xdf68e4 to 0xe0071a, (0x9e36 bytes) 2024-09-17T21:30:41.213 INFO:teuthology.orchestra.run.smithi129.stdout:4662 read 0x24069f thru 0x247b83 (0x74e5 bytes) 2024-09-17T21:30:41.213 INFO:teuthology.orchestra.run.smithi129.stdout:4663 writesame 0x44c303f thru 0x44d121f data_size 0x19(0xe1e1 bytes) 2024-09-17T21:30:41.216 INFO:teuthology.orchestra.run.smithi129.stdout:4664 write 0x7b2f70f thru 0x7b326d3 (0x2fc5 bytes) 2024-09-17T21:30:41.227 INFO:teuthology.orchestra.run.smithi129.stdout:4666 trunc from 0xb1c7222 to 0x55897c0 2024-09-17T21:30:41.244 INFO:teuthology.orchestra.run.smithi129.stdout:4667 trunc from 0x55897c0 to 0x43e03c2 2024-09-17T21:30:41.254 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:41.254 INFO:teuthology.orchestra.run.smithi129.stdout:4670 punch from 0x2396f6b to 0x23a29ce, (0xba63 bytes) 2024-09-17T21:30:41.254 INFO:teuthology.orchestra.run.smithi129.stdout:4671 clone 33 order 20 su 16384 sc 8 2024-09-17T21:30:41.266 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:41.267 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:41.267 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:41.267 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:41.276 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:30:41.284 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:41.284 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:41.301 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:30:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:30:41.306 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:41.304+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:41.318 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:41.318 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:41.334 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:41.334 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:41.351 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:41.351 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:41.368 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:41.369 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:41.385 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:41.386 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:41.389 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:41.389 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:41.398 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:30:41.403 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:41.403 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:41.407 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:41.405+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:41.506 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:41.506 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:41.507 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:41.505+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:41.515 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:30:41.609 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:41.606+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:41.620 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:41.620 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:41.629 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:30:41.710 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:41.708+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:41.732 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:41.732 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:41.740 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:30:41.811 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:41.809+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:41.853 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:41.853 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:41.862 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:30:41.870 INFO:teuthology.orchestra.run.smithi129.stdout:flattening image image_client.0-clone32 2024-09-17T21:30:41.912 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:41.909+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:41.977 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-17T21:30:41.980 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:41.981 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:41.990 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:30:42.013 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:42.010+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:42.094 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:42.094 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:42.102 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:30:42.146 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:42.144+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:42.206 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:42.207 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:42.215 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:30:42.216 INFO:teuthology.orchestra.run.smithi129.stdout:4672 punch from 0x3bd98cd to 0x3be4ab9, (0xb1ec bytes) 2024-09-17T21:30:42.216 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:42.212+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:42.219 INFO:teuthology.orchestra.run.smithi129.stdout:4674 read 0xad9100 thru 0xae5ac1 (0xc9c2 bytes) 2024-09-17T21:30:42.219 INFO:teuthology.orchestra.run.smithi129.stdout:4675 read 0x3a250c2 thru 0x3a26657 (0x1596 bytes) 2024-09-17T21:30:42.220 INFO:teuthology.orchestra.run.smithi129.stdout:4676 read 0x35a501e thru 0x35b4f0c (0xfeef bytes) 2024-09-17T21:30:42.227 INFO:teuthology.orchestra.run.smithi129.stdout:4677 write 0x4f24d39 thru 0x4f2ec60 (0x9f28 bytes) 2024-09-17T21:30:42.235 INFO:teuthology.orchestra.run.smithi129.stdout:4678 write 0x19d8d60 thru 0x19e8675 (0xf916 bytes) 2024-09-17T21:30:42.252 INFO:teuthology.orchestra.run.smithi129.stdout:4679 read 0x36086c6 thru 0x360f76c (0x70a7 bytes) 2024-09-17T21:30:42.252 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:42.252 INFO:teuthology.orchestra.run.smithi129.stdout:4681 clone 34 order 18 su 8192 sc 12 2024-09-17T21:30:42.316 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:42.314+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:42.358 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:42.359 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:42.367 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:30:42.451 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:42.449+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:42.511 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:42.511 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:42.517 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:42.515+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:42.520 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:30:42.618 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:42.616+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:42.664 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:42.664 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:42.672 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:30:42.719 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:42.717+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:42.805 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-17T21:30:42.805 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-17T21:30:42.820 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:42.820 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:42.825 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:30:42.830 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:30:42.849 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:42.847+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:42.877 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone33 from 0x4f2ec61 (overlap 0x43e03c2) to 0x0 2024-09-17T21:30:42.921 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:42.919+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:42.959 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:42.960 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:42.973 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:30:43.022 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:43.020+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:43.120 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-17T21:30:43.123 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:43.120+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:43.139 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:43.139 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:43.148 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:30:43.202 INFO:teuthology.orchestra.run.smithi129.stdout:4682 punch from 0x146e2f5 to 0x1470959, (0x2664 bytes) 2024-09-17T21:30:43.206 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:43.215 INFO:teuthology.orchestra.run.smithi129.stdout:4684 write 0x75ba5a8 thru 0x75be252 (0x3cab bytes) 2024-09-17T21:30:43.224 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:43.222+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:43.225 INFO:teuthology.orchestra.run.smithi129.stdout:4686 writesame 0x2c99d1f thru 0x2ca0382 data_size 0x2(0x6664 bytes) 2024-09-17T21:30:43.246 INFO:teuthology.orchestra.run.smithi129.stdout:4688 read 0x55bfbe0 thru 0x55c28e4 (0x2d05 bytes) 2024-09-17T21:30:43.259 INFO:teuthology.orchestra.run.smithi129.stdout:4690 write 0xd5929b6 thru 0xd59f2f7 (0xc942 bytes) 2024-09-17T21:30:43.269 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:43.269 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:43.271 INFO:teuthology.orchestra.run.smithi129.stdout:4691 write 0x90c7d72 thru 0x90d43a8 (0xc637 bytes) 2024-09-17T21:30:43.281 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:30:43.284 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:43.290 INFO:teuthology.orchestra.run.smithi129.stdout:4694 trunc from 0xd59f2f8 to 0x7e9350d 2024-09-17T21:30:43.302 INFO:teuthology.orchestra.run.smithi129.stdout:4695 punch from 0x5dd026b to 0x5ddcd03, (0xca98 bytes) 2024-09-17T21:30:43.302 INFO:teuthology.orchestra.run.smithi129.stdout:4696 trunc from 0x7e9350d to 0x80d4b5b 2024-09-17T21:30:43.306 INFO:teuthology.orchestra.run.smithi129.stdout:4697 writesame 0x3103f37 thru 0x31067bf data_size 0x3(0x2889 bytes) 2024-09-17T21:30:43.316 INFO:teuthology.orchestra.run.smithi129.stdout:4698 write 0x649a6c thru 0x65892b (0xeec0 bytes) 2024-09-17T21:30:43.325 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:43.322+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:43.349 INFO:teuthology.orchestra.run.smithi129.stdout:4699 trunc from 0x80d4b5b to 0x7ad815 2024-09-17T21:30:43.387 INFO:teuthology.orchestra.run.smithi129.stdout:4701 write 0x9675ce9 thru 0x96828b8 (0xcbd0 bytes) 2024-09-17T21:30:43.398 INFO:teuthology.orchestra.run.smithi129.stdout:4703 write 0x5b3d95 thru 0x5bd8bf (0x9b2b bytes) 2024-09-17T21:30:43.414 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:30:43.425 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:43.425 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:43.425 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:43.423+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:43.429 INFO:teuthology.orchestra.run.smithi129.stdout:4704 read 0xb43f06 thru 0xb49f06 (0x6001 bytes) 2024-09-17T21:30:43.430 INFO:teuthology.orchestra.run.smithi129.stdout:4705 read 0x72a50cd thru 0x72b490a (0xf83e bytes) 2024-09-17T21:30:43.435 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:30:43.437 INFO:teuthology.orchestra.run.smithi129.stdout:4706 write 0xb25e6ee thru 0xb266204 (0x7b17 bytes) 2024-09-17T21:30:43.450 INFO:teuthology.orchestra.run.smithi129.stdout:4707 read 0x80aed54 thru 0x80b63be (0x766b bytes) 2024-09-17T21:30:43.450 INFO:teuthology.orchestra.run.smithi129.stdout:4708 write 0x3e5ad28 thru 0x3e5f082 (0x435b bytes) 2024-09-17T21:30:43.468 INFO:teuthology.orchestra.run.smithi129.stdout:4709 trunc from 0xb266205 to 0x327bbe2 2024-09-17T21:30:43.497 INFO:teuthology.orchestra.run.smithi129.stdout:4711 write 0xbb13e4d thru 0xbb22b09 (0xecbd bytes) 2024-09-17T21:30:43.509 INFO:teuthology.orchestra.run.smithi129.stdout:4712 write 0x906b9d9 thru 0x907083b (0x4e63 bytes) 2024-09-17T21:30:43.514 INFO:teuthology.orchestra.run.smithi129.stdout:4714 read 0x7f2c3ec thru 0x7f2f522 (0x3137 bytes) 2024-09-17T21:30:43.522 INFO:teuthology.orchestra.run.smithi129.stdout:4715 write 0xdcdc93e thru 0xdce43df (0x7aa2 bytes) 2024-09-17T21:30:43.526 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:43.524+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:43.530 INFO:teuthology.orchestra.run.smithi129.stdout:4716 read 0x55e85d9 thru 0x55f4547 (0xbf6f bytes) 2024-09-17T21:30:43.530 INFO:teuthology.orchestra.run.smithi129.stdout:4717 writesame 0xb5b97eb thru 0xb5c2825 data_size 0x1(0x903b bytes) 2024-09-17T21:30:43.541 INFO:teuthology.orchestra.run.smithi129.stdout:4718 punch from 0x2ceba74 to 0x2cf9367, (0xd8f3 bytes) 2024-09-17T21:30:43.541 INFO:teuthology.orchestra.run.smithi129.stdout:4719 read 0xc241f2 thru 0xc25265 (0x1074 bytes) 2024-09-17T21:30:43.541 INFO:teuthology.orchestra.run.smithi129.stdout:4720 writesame 0xb94ac37 thru 0xb95a749 data_size 0x19(0xfb13 bytes) 2024-09-17T21:30:43.554 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:43.554 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:43.561 INFO:teuthology.orchestra.run.smithi129.stdout:4723 write 0xe69a5c8 thru 0xe69b8a6 (0x12df bytes) 2024-09-17T21:30:43.564 INFO:teuthology.orchestra.run.smithi129.stdout:4724 write 0xb78a9c0 thru 0xb7992a4 (0xe8e5 bytes) 2024-09-17T21:30:43.572 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:43.572 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:43.572 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:43.573 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:43.580 INFO:teuthology.orchestra.run.smithi129.stdout:4725 trunc from 0xe69b8a7 to 0x7c9d923 2024-09-17T21:30:43.588 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:30:43.596 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:30:43.596 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:30:43.610 INFO:teuthology.orchestra.run.smithi129.stdout:4726 write 0xe035cb2 thru 0xe0431fe (0xd54d bytes) 2024-09-17T21:30:43.615 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:43.615 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:43.627 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:43.625+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:43.635 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:43.635 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:43.638 INFO:teuthology.orchestra.run.smithi129.stdout:4727 trunc from 0xe0431ff to 0x14e8750 2024-09-17T21:30:43.655 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:43.656 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:43.665 INFO:teuthology.orchestra.run.smithi129.stdout:4729 write 0x8ee8ad7 thru 0x8ef4f4d (0xc477 bytes) 2024-09-17T21:30:43.677 INFO:teuthology.orchestra.run.smithi129.stdout:4730 read 0x473f933 thru 0x474ef0c (0xf5da bytes) 2024-09-17T21:30:43.677 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:43.677 INFO:teuthology.orchestra.run.smithi129.stdout:4733 read 0x41e0e93 thru 0x41eb9e2 (0xab50 bytes) 2024-09-17T21:30:43.677 INFO:teuthology.orchestra.run.smithi129.stdout:4734 read 0x50290ff thru 0x50335f5 (0xa4f7 bytes) 2024-09-17T21:30:43.677 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:43.677 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:43.678 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:43.684 INFO:teuthology.orchestra.run.smithi129.stdout:4737 trunc from 0x8ef4f4e to 0x3594cc0 2024-09-17T21:30:43.695 INFO:teuthology.orchestra.run.smithi129.stdout:4738 write 0x354fc56 thru 0x3554005 (0x43b0 bytes) 2024-09-17T21:30:43.697 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:43.697 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:43.702 INFO:teuthology.orchestra.run.smithi129.stdout:4739 read 0x19832b5 thru 0x198f97e (0xc6ca bytes) 2024-09-17T21:30:43.702 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:43.713 INFO:teuthology.orchestra.run.smithi129.stdout:4742 write 0x6d2bed4 thru 0x6d2e25e (0x238b bytes) 2024-09-17T21:30:43.717 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:43.717 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:43.721 INFO:teuthology.orchestra.run.smithi129.stdout:4743 punch from 0x17cb0b8 to 0x17da682, (0xf5ca bytes) 2024-09-17T21:30:43.722 INFO:teuthology.orchestra.run.smithi129.stdout:4744 read 0x63eb0ce thru 0x63f6d82 (0xbcb5 bytes) 2024-09-17T21:30:43.722 INFO:teuthology.orchestra.run.smithi129.stdout:4746 writesame 0x2712cf8 thru 0x2720686 data_size 0x4f(0xd98f bytes) 2024-09-17T21:30:43.728 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:43.726+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:43.741 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:43.741 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:43.741 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:43.741 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:43.742 INFO:teuthology.orchestra.run.smithi129.stdout:4747 trunc from 0x6d2e25f to 0xe6d1eae 2024-09-17T21:30:43.751 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:30:43.755 INFO:teuthology.orchestra.run.smithi129.stdout:4749 trunc from 0xe6d1eae to 0x8740406 2024-09-17T21:30:43.772 INFO:teuthology.orchestra.run.smithi129.stdout:4752 write 0xe1e1e91 thru 0xe1f1bf4 (0xfd64 bytes) 2024-09-17T21:30:43.789 INFO:teuthology.orchestra.run.smithi129.stdout:4755 writesame 0x801348c thru 0x801dd6b data_size 0x38(0xa8e0 bytes) 2024-09-17T21:30:43.797 INFO:teuthology.orchestra.run.smithi129.stdout:4758 punch from 0xb882878 to 0xb882a3d, (0x1c5 bytes) 2024-09-17T21:30:43.797 INFO:teuthology.orchestra.run.smithi129.stdout:4759 write 0xab3bac8 thru 0xab469a0 (0xaed9 bytes) 2024-09-17T21:30:43.807 INFO:teuthology.orchestra.run.smithi129.stdout:4760 read 0xa91afc1 thru 0xa92869b (0xd6db bytes) 2024-09-17T21:30:43.807 INFO:teuthology.orchestra.run.smithi129.stdout:4761 write 0xac42b61 thru 0xac4e106 (0xb5a6 bytes) 2024-09-17T21:30:43.809 INFO:teuthology.orchestra.run.smithi129.stdout:4763 write 0x695c097 thru 0x6962120 (0x608a bytes) 2024-09-17T21:30:43.816 INFO:teuthology.orchestra.run.smithi129.stdout:4764 write 0xc7d9b24 thru 0xc7e3c6a (0xa147 bytes) 2024-09-17T21:30:43.821 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T21:30:43.821 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is set 2024-09-17T21:30:43.829 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:43.827+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:43.832 INFO:teuthology.orchestra.run.smithi129.stdout:4765 trunc from 0xe1f1bf5 to 0xd909816 2024-09-17T21:30:43.842 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T21:30:43.844 INFO:teuthology.orchestra.run.smithi129.stdout:4768 writesame 0x773a9c8 thru 0x7741d3b data_size 0x12(0x7374 bytes) 2024-09-17T21:30:43.854 INFO:teuthology.orchestra.run.smithi129.stdout:4769 read 0xc3a97c4 thru 0xc3adf26 (0x4763 bytes) 2024-09-17T21:30:43.854 INFO:teuthology.orchestra.run.smithi129.stdout:4770 read 0x4e93d5 thru 0x4f47d8 (0xb404 bytes) 2024-09-17T21:30:43.862 INFO:teuthology.orchestra.run.smithi129.stdout:4771 writesame 0xe3d5efd thru 0xe3d9a78 data_size 0x5e(0x3b7c bytes) 2024-09-17T21:30:43.862 INFO:teuthology.orchestra.run.smithi129.stdout:4772 read 0xb9a579f thru 0xb9b275e (0xcfc0 bytes) 2024-09-17T21:30:43.862 INFO:teuthology.orchestra.run.smithi129.stdout:4773 read 0x2a51bac thru 0x2a536cc (0x1b21 bytes) 2024-09-17T21:30:43.862 INFO:teuthology.orchestra.run.smithi129.stdout:4774 write 0x939096f thru 0x9399c72 (0x9304 bytes) 2024-09-17T21:30:43.877 INFO:teuthology.orchestra.run.smithi129.stdout:4776 write 0x20e362b thru 0x20e37ad (0x183 bytes) 2024-09-17T21:30:43.880 INFO:teuthology.orchestra.run.smithi129.stdout:4777 writesame 0x3165208 thru 0x316a74c data_size 0x1(0x5545 bytes) 2024-09-17T21:30:43.891 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:43.892 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:43.893 INFO:teuthology.orchestra.run.smithi129.stdout:4780 read 0x66623d4 thru 0x6668b21 (0x674e bytes) 2024-09-17T21:30:43.901 INFO:teuthology.orchestra.run.smithi129.stdout:4781 trunc from 0xe3d9a79 to 0x7d872ba 2024-09-17T21:30:43.907 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:30:43.916 INFO:teuthology.orchestra.run.smithi129.stdout:4782 read 0x3619c2d thru 0x361e0bd (0x4491 bytes) 2024-09-17T21:30:43.916 INFO:teuthology.orchestra.run.smithi129.stdout:4783 punch from 0x220df65 to 0x2211f74, (0x400f bytes) 2024-09-17T21:30:43.916 INFO:teuthology.orchestra.run.smithi129.stdout:4784 punch from 0x48066b3 to 0x480e359, (0x7ca6 bytes) 2024-09-17T21:30:43.916 INFO:teuthology.orchestra.run.smithi129.stdout:4785 read 0x84fe4d thru 0x8575b4 (0x7768 bytes) 2024-09-17T21:30:43.917 INFO:teuthology.orchestra.run.smithi129.stdout:4788 write 0xf7eaaa thru 0xf82f8c (0x44e3 bytes) 2024-09-17T21:30:43.930 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:43.928+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:43.934 INFO:teuthology.orchestra.run.smithi129.stdout:4789 write 0xe34f640 thru 0xe35650f (0x6ed0 bytes) 2024-09-17T21:30:43.941 INFO:teuthology.orchestra.run.smithi129.stdout:4790 write 0x30808cc thru 0x308ccc6 (0xc3fb bytes) 2024-09-17T21:30:43.950 INFO:teuthology.orchestra.run.smithi129.stdout:4791 write 0xa11e63e thru 0xa11f526 (0xee9 bytes) 2024-09-17T21:30:43.953 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:43.953 INFO:teuthology.orchestra.run.smithi129.stdout:4794 clone 35 order 21 su 262144 sc 7 2024-09-17T21:30:44.031 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:44.029+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:44.034 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:37817 3.17 3:eab85a8d:::rbd_header.1120e62eea89:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:30:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.335710014999997, 2024-09-17T21:30:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86635970399999995, 2024-09-17T21:30:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37817 2024-09-17T21:30:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695457+0000", 2024-09-17T21:30:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5799999999999998e-07 2024-09-17T21:30:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695461+0000", 2024-09-17T21:30:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1110000000000001e-06 2024-09-17T21:30:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695471+0000", 2024-09-17T21:30:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0441e-05 2024-09-17T21:30:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695477+0000", 2024-09-17T21:30:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7599999999999999e-06 2024-09-17T21:30:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695533+0000", 2024-09-17T21:30:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6570999999999998e-05 2024-09-17T21:30:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:30:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695543+0000", 2024-09-17T21:30:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9979999999999998e-06 2024-09-17T21:30:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561472+0000", 2024-09-17T21:30:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86592841899999995 2024-09-17T21:30:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561610+0000", 2024-09-17T21:30:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013854699999999999 2024-09-17T21:30:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561816+0000", 2024-09-17T21:30:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020509900000000001 2024-09-17T21:30:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50202 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.923539224999999, 2024-09-17T21:30:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.076620261999999995, 2024-09-17T21:30:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50202 2024-09-17T21:30:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107629+0000", 2024-09-17T21:30:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3709999999999998e-06 2024-09-17T21:30:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119765+0000", 2024-09-17T21:30:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012135683 2024-09-17T21:30:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119787+0000", 2024-09-17T21:30:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2748999999999999e-05 2024-09-17T21:30:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119797+0000", 2024-09-17T21:30:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8169999999999994e-06 2024-09-17T21:30:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119872+0000", 2024-09-17T21:30:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270999999999994e-05 2024-09-17T21:30:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119987+0000", 2024-09-17T21:30:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011557699999999999 2024-09-17T21:30:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.121766+0000", 2024-09-17T21:30:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00177853 2024-09-17T21:30:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178307+0000", 2024-09-17T21:30:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056541559999999998 2024-09-17T21:30:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184097+0000", 2024-09-17T21:30:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057899029999999999 2024-09-17T21:30:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184172+0000", 2024-09-17T21:30:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5226000000000001e-05 2024-09-17T21:30:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184247+0000", 2024-09-17T21:30:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4574999999999997e-05 2024-09-17T21:30:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50203 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [truncate 20971520] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.911339320000003, 2024-09-17T21:30:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065009779000000004, 2024-09-17T21:30:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50203 2024-09-17T21:30:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119828+0000", 2024-09-17T21:30:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6390000000000001e-06 2024-09-17T21:30:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119832+0000", 2024-09-17T21:30:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7699999999999999e-06 2024-09-17T21:30:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119845+0000", 2024-09-17T21:30:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3093e-05 2024-09-17T21:30:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119852+0000", 2024-09-17T21:30:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6950000000000003e-06 2024-09-17T21:30:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.160945+0000", 2024-09-17T21:30:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041093076999999999 2024-09-17T21:30:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161071+0000", 2024-09-17T21:30:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012569799999999999 2024-09-17T21:30:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161283+0000", 2024-09-17T21:30:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021201999999999999 2024-09-17T21:30:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.179360+0000", 2024-09-17T21:30:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018077784999999999 2024-09-17T21:30:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184726+0000", 2024-09-17T21:30:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053652960000000003 2024-09-17T21:30:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184787+0000", 2024-09-17T21:30:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1402000000000006e-05 2024-09-17T21:30:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184836+0000", 2024-09-17T21:30:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9304000000000001e-05 2024-09-17T21:30:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50206 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.874257780000001, 2024-09-17T21:30:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.090361968000000001, 2024-09-17T21:30:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50206 2024-09-17T21:30:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156911+0000", 2024-09-17T21:30:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-17T21:30:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173670+0000", 2024-09-17T21:30:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016758907 2024-09-17T21:30:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173696+0000", 2024-09-17T21:30:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6329000000000002e-05 2024-09-17T21:30:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173708+0000", 2024-09-17T21:30:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2512e-05 2024-09-17T21:30:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173788+0000", 2024-09-17T21:30:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9332999999999994e-05 2024-09-17T21:30:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173918+0000", 2024-09-17T21:30:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013063300000000001 2024-09-17T21:30:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.176228+0000", 2024-09-17T21:30:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002309757 2024-09-17T21:30:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.245164+0000", 2024-09-17T21:30:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068936301000000005 2024-09-17T21:30:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247176+0000", 2024-09-17T21:30:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020117590000000001 2024-09-17T21:30:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247215+0000", 2024-09-17T21:30:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9094000000000002e-05 2024-09-17T21:30:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247270+0000", 2024-09-17T21:30:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4793999999999997e-05 2024-09-17T21:30:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50207 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [truncate 21556327] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.857408800000002, 2024-09-17T21:30:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.074055363999999999, 2024-09-17T21:30:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50207 2024-09-17T21:30:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173759+0000", 2024-09-17T21:30:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8160000000000001e-06 2024-09-17T21:30:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173763+0000", 2024-09-17T21:30:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5360000000000003e-06 2024-09-17T21:30:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173771+0000", 2024-09-17T21:30:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8849999999999999e-06 2024-09-17T21:30:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173777+0000", 2024-09-17T21:30:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7799999999999997e-06 2024-09-17T21:30:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221363+0000", 2024-09-17T21:30:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047585493 2024-09-17T21:30:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221430+0000", 2024-09-17T21:30:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7129000000000001e-05 2024-09-17T21:30:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221550+0000", 2024-09-17T21:30:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120658 2024-09-17T21:30:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246113+0000", 2024-09-17T21:30:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02456262 2024-09-17T21:30:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247744+0000", 2024-09-17T21:30:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001630692 2024-09-17T21:30:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247777+0000", 2024-09-17T21:30:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3015000000000002e-05 2024-09-17T21:30:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247812+0000", 2024-09-17T21:30:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.574e-05 2024-09-17T21:30:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50828 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.549296867999999, 2024-09-17T21:30:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047324185999999997, 2024-09-17T21:30:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50828 2024-09-17T21:30:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.46e-07 2024-09-17T21:30:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481871+0000", 2024-09-17T21:30:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T21:30:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481878+0000", 2024-09-17T21:30:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:30:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481882+0000", 2024-09-17T21:30:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:30:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529038+0000", 2024-09-17T21:30:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047156034999999999 2024-09-17T21:30:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529099+0000", 2024-09-17T21:30:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0316000000000003e-05 2024-09-17T21:30:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529193+0000", 2024-09-17T21:30:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4369000000000004e-05 2024-09-17T21:30:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50829 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 12845056~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.549278074, 2024-09-17T21:30:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047887596999999997, 2024-09-17T21:30:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50829 2024-09-17T21:30:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:30:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481890+0000", 2024-09-17T21:30:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.877e-06 2024-09-17T21:30:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481895+0000", 2024-09-17T21:30:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:30:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481900+0000", 2024-09-17T21:30:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4440000000000001e-06 2024-09-17T21:30:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529225+0000", 2024-09-17T21:30:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047325117999999999 2024-09-17T21:30:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529292+0000", 2024-09-17T21:30:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7100000000000005e-05 2024-09-17T21:30:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529775+0000", 2024-09-17T21:30:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048322799999999999 2024-09-17T21:30:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50830 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.549259671999998, 2024-09-17T21:30:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048062123999999998, 2024-09-17T21:30:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50830 2024-09-17T21:30:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481907+0000", 2024-09-17T21:30:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:30:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481909+0000", 2024-09-17T21:30:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T21:30:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481915+0000", 2024-09-17T21:30:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.02e-06 2024-09-17T21:30:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481920+0000", 2024-09-17T21:30:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6799999999999998e-06 2024-09-17T21:30:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529793+0000", 2024-09-17T21:30:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047873042999999997 2024-09-17T21:30:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529870+0000", 2024-09-17T21:30:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6967999999999999e-05 2024-09-17T21:30:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529968+0000", 2024-09-17T21:30:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8085999999999996e-05 2024-09-17T21:30:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50831 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 18087936~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.549239325999999, 2024-09-17T21:30:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048663304999999997, 2024-09-17T21:30:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50831 2024-09-17T21:30:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:30:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481929+0000", 2024-09-17T21:30:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2189999999999998e-06 2024-09-17T21:30:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481935+0000", 2024-09-17T21:30:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.383e-06 2024-09-17T21:30:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481939+0000", 2024-09-17T21:30:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2479999999999998e-06 2024-09-17T21:30:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530095+0000", 2024-09-17T21:30:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048156617999999998 2024-09-17T21:30:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530159+0000", 2024-09-17T21:30:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3394999999999999e-05 2024-09-17T21:30:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530590+0000", 2024-09-17T21:30:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043101199999999999 2024-09-17T21:30:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50832 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 19660800~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.549220679000001, 2024-09-17T21:30:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049339466999999998, 2024-09-17T21:30:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50832 2024-09-17T21:30:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481946+0000", 2024-09-17T21:30:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481951+0000", 2024-09-17T21:30:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1479999999999997e-06 2024-09-17T21:30:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481956+0000", 2024-09-17T21:30:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7339999999999996e-06 2024-09-17T21:30:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481961+0000", 2024-09-17T21:30:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4889999999999997e-06 2024-09-17T21:30:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530630+0000", 2024-09-17T21:30:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048668596000000001 2024-09-17T21:30:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530720+0000", 2024-09-17T21:30:44.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0163999999999999e-05 2024-09-17T21:30:44.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531285+0000", 2024-09-17T21:30:44.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056493199999999998 2024-09-17T21:30:44.095 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.095 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.095 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.095 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50833 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.096 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:44.096 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.549198535000002, 2024-09-17T21:30:44.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049908069999999999, 2024-09-17T21:30:44.096 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.096 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.096 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.096 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.096 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.096 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.096 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50833 2024-09-17T21:30:44.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.097 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:44.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:44.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481968+0000", 2024-09-17T21:30:44.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5600000000000001e-07 2024-09-17T21:30:44.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481970+0000", 2024-09-17T21:30:44.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:30:44.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481975+0000", 2024-09-17T21:30:44.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8499999999999999e-06 2024-09-17T21:30:44.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481980+0000", 2024-09-17T21:30:44.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-17T21:30:44.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531337+0000", 2024-09-17T21:30:44.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049356874000000002 2024-09-17T21:30:44.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531392+0000", 2024-09-17T21:30:44.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4826e-05 2024-09-17T21:30:44.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531875+0000", 2024-09-17T21:30:44.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048345099999999998 2024-09-17T21:30:44.101 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.101 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.101 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.101 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50834 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 24903680~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.101 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:44.101 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.549173766999999, 2024-09-17T21:30:44.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050534015000000002, 2024-09-17T21:30:44.101 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.102 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.102 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.102 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.102 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.102 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.102 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50834 2024-09-17T21:30:44.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.102 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:44.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:44.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481993+0000", 2024-09-17T21:30:44.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:30:44.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481995+0000", 2024-09-17T21:30:44.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.199e-06 2024-09-17T21:30:44.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482001+0000", 2024-09-17T21:30:44.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.772e-06 2024-09-17T21:30:44.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482006+0000", 2024-09-17T21:30:44.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0660000000000003e-06 2024-09-17T21:30:44.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531923+0000", 2024-09-17T21:30:44.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049917200000000002 2024-09-17T21:30:44.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532004+0000", 2024-09-17T21:30:44.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0758000000000002e-05 2024-09-17T21:30:44.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532526+0000", 2024-09-17T21:30:44.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00052258899999999995 2024-09-17T21:30:44.106 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.106 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.106 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.107 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50835 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 28573696~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.107 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:44.107 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.549153783999998, 2024-09-17T21:30:44.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051204701999999998, 2024-09-17T21:30:44.107 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.107 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.107 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.107 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.107 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.107 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.108 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50835 2024-09-17T21:30:44.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.108 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:44.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:44.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482013+0000", 2024-09-17T21:30:44.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:30:44.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482015+0000", 2024-09-17T21:30:44.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.272e-06 2024-09-17T21:30:44.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482020+0000", 2024-09-17T21:30:44.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3040000000000004e-06 2024-09-17T21:30:44.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482025+0000", 2024-09-17T21:30:44.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5650000000000003e-06 2024-09-17T21:30:44.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532569+0000", 2024-09-17T21:30:44.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050544661999999997 2024-09-17T21:30:44.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532649+0000", 2024-09-17T21:30:44.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9640000000000003e-05 2024-09-17T21:30:44.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533217+0000", 2024-09-17T21:30:44.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056784100000000003 2024-09-17T21:30:44.112 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.112 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.112 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.112 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50836 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 30146560~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.112 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:44.112 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.549134166999998, 2024-09-17T21:30:44.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051875589, 2024-09-17T21:30:44.113 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.113 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.113 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.113 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.113 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.113 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.113 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50836 2024-09-17T21:30:44.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.113 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:44.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:44.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:44.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3700000000000001e-07 2024-09-17T21:30:44.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482034+0000", 2024-09-17T21:30:44.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0059999999999999e-06 2024-09-17T21:30:44.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482040+0000", 2024-09-17T21:30:44.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:30:44.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482044+0000", 2024-09-17T21:30:44.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:30:44.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533260+0000", 2024-09-17T21:30:44.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051215416 2024-09-17T21:30:44.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533348+0000", 2024-09-17T21:30:44.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8695000000000004e-05 2024-09-17T21:30:44.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533907+0000", 2024-09-17T21:30:44.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055902199999999995 2024-09-17T21:30:44.117 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.117 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.117 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.118 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50844 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.118 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:44.118 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.548970740999998, 2024-09-17T21:30:44.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0475644, 2024-09-17T21:30:44.118 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.118 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.118 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.118 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.118 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.118 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.118 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50844 2024-09-17T21:30:44.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.119 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:44.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:44.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482196+0000", 2024-09-17T21:30:44.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.39e-07 2024-09-17T21:30:44.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482198+0000", 2024-09-17T21:30:44.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0099999999999998e-06 2024-09-17T21:30:44.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482204+0000", 2024-09-17T21:30:44.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.93e-06 2024-09-17T21:30:44.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482208+0000", 2024-09-17T21:30:44.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7439999999999998e-06 2024-09-17T21:30:44.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529318+0000", 2024-09-17T21:30:44.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047110051999999999 2024-09-17T21:30:44.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529396+0000", 2024-09-17T21:30:44.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7712000000000004e-05 2024-09-17T21:30:44.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529760+0000", 2024-09-17T21:30:44.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00036351299999999999 2024-09-17T21:30:44.122 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.122 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.122 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.123 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50845 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 21757952~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.123 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:44.123 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.548951637999998, 2024-09-17T21:30:44.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048084625999999998, 2024-09-17T21:30:44.123 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.123 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.123 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.123 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.123 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.123 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.124 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50845 2024-09-17T21:30:44.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.124 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:44.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:44.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482215+0000", 2024-09-17T21:30:44.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:30:44.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482216+0000", 2024-09-17T21:30:44.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.823e-06 2024-09-17T21:30:44.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482222+0000", 2024-09-17T21:30:44.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.507e-06 2024-09-17T21:30:44.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482227+0000", 2024-09-17T21:30:44.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6609999999999999e-06 2024-09-17T21:30:44.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529792+0000", 2024-09-17T21:30:44.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047565075999999998 2024-09-17T21:30:44.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529871+0000", 2024-09-17T21:30:44.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8855000000000003e-05 2024-09-17T21:30:44.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530299+0000", 2024-09-17T21:30:44.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000428307 2024-09-17T21:30:44.127 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.128 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.128 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.128 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50846 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 25427968~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.128 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:44.128 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.548929250999997, 2024-09-17T21:30:44.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048735142000000002, 2024-09-17T21:30:44.128 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.128 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.128 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.128 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.129 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.129 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.129 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50846 2024-09-17T21:30:44.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.129 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:44.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:44.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:44.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:30:44.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482239+0000", 2024-09-17T21:30:44.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.871e-06 2024-09-17T21:30:44.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482244+0000", 2024-09-17T21:30:44.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.49e-06 2024-09-17T21:30:44.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482249+0000", 2024-09-17T21:30:44.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6269999999999999e-06 2024-09-17T21:30:44.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530347+0000", 2024-09-17T21:30:44.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048097721000000003 2024-09-17T21:30:44.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530437+0000", 2024-09-17T21:30:44.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0604000000000002e-05 2024-09-17T21:30:44.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530972+0000", 2024-09-17T21:30:44.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053445000000000001 2024-09-17T21:30:44.133 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.133 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.133 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.133 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50847 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 27000832~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.133 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:44.133 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.548911355000001, 2024-09-17T21:30:44.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049308668, 2024-09-17T21:30:44.133 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.133 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.134 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.134 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.134 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.134 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.134 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50847 2024-09-17T21:30:44.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.134 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:44.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:44.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:44.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:30:44.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482257+0000", 2024-09-17T21:30:44.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.846e-06 2024-09-17T21:30:44.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482264+0000", 2024-09-17T21:30:44.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8170000000000001e-06 2024-09-17T21:30:44.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482283+0000", 2024-09-17T21:30:44.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9718e-05 2024-09-17T21:30:44.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531018+0000", 2024-09-17T21:30:44.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048734497000000002 2024-09-17T21:30:44.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531080+0000", 2024-09-17T21:30:44.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1795e-05 2024-09-17T21:30:44.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531563+0000", 2024-09-17T21:30:44.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048366499999999998 2024-09-17T21:30:44.138 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.138 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.138 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.138 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50848 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 30670848~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.138 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:44.138 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.548874812999998, 2024-09-17T21:30:44.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049893337000000003, 2024-09-17T21:30:44.139 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.139 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.139 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.139 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.139 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.139 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.139 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50848 2024-09-17T21:30:44.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.139 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:44.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:44.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482292+0000", 2024-09-17T21:30:44.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T21:30:44.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482294+0000", 2024-09-17T21:30:44.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3209999999999999e-06 2024-09-17T21:30:44.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482300+0000", 2024-09-17T21:30:44.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7189999999999998e-06 2024-09-17T21:30:44.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482305+0000", 2024-09-17T21:30:44.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T21:30:44.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531606+0000", 2024-09-17T21:30:44.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049301273999999999 2024-09-17T21:30:44.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531696+0000", 2024-09-17T21:30:44.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0161999999999995e-05 2024-09-17T21:30:44.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532184+0000", 2024-09-17T21:30:44.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048806400000000002 2024-09-17T21:30:44.143 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.143 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.143 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.144 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50849 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 32243712~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:44.144 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:44.144 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.548855371999998, 2024-09-17T21:30:44.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050444607000000002, 2024-09-17T21:30:44.144 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.144 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.144 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.144 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50849 2024-09-17T21:30:44.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.145 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:44.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:44.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482311+0000", 2024-09-17T21:30:44.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:30:44.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482316+0000", 2024-09-17T21:30:44.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1209999999999998e-06 2024-09-17T21:30:44.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482321+0000", 2024-09-17T21:30:44.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4779999999999999e-06 2024-09-17T21:30:44.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482326+0000", 2024-09-17T21:30:44.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7099999999999998e-06 2024-09-17T21:30:44.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532221+0000", 2024-09-17T21:30:44.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049894936000000001 2024-09-17T21:30:44.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532287+0000", 2024-09-17T21:30:44.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5914000000000006e-05 2024-09-17T21:30:44.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532755+0000", 2024-09-17T21:30:44.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000468107 2024-09-17T21:30:44.148 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.148 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.149 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.149 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.149 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.149 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:44.150 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:30:44.151 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:44.147+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:44.251 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:44.249+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:44.293 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:44.293 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:44.293 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:44.293 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:44.302 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:30:44.352 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:44.350+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:44.405 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:44.405 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:44.405 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:44.405 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:44.405 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:44.413 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:30:44.457 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:44.455+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:44.517 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:44.518 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:44.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:44.518 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:44.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.518 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "pg_update_log_missing(3.5 epoch 45/37 rep_tid 844 entries 45'73 (0'0) error 3:a65c40ae:::rbd_data.1120b21c3309.0000000000000126:head by client.4384.0:48900 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 45'70)", 2024-09-17T21:30:44.518 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:44.518 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.467480205999998, 2024-09-17T21:30:44.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019554365000000001, 2024-09-17T21:30:44.518 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.518 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.518 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T21:30:44.518 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:44.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:44.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047112+0000", 2024-09-17T21:30:44.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:30:44.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047121+0000", 2024-09-17T21:30:44.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4379999999999995e-06 2024-09-17T21:30:44.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047139+0000", 2024-09-17T21:30:44.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8235000000000001e-05 2024-09-17T21:30:44.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047303+0000", 2024-09-17T21:30:44.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016389400000000001 2024-09-17T21:30:44.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047396+0000", 2024-09-17T21:30:44.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3355000000000003e-05 2024-09-17T21:30:44.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066666+0000", 2024-09-17T21:30:44.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019269649999999999 2024-09-17T21:30:44.522 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.522 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.522 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.522 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48906 3.4 e45/25 3:2503fe27:::rbd_data.1120b21c3309.0000000000000127:head v 45'148, mlcod=45'146)", 2024-09-17T21:30:44.522 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:44.523 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.465354026999997, 2024-09-17T21:30:44.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018169739000000001, 2024-09-17T21:30:44.523 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.523 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.523 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:44.523 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:44.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:44.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:44.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:30:44.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049244+0000", 2024-09-17T21:30:44.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8390000000000003e-06 2024-09-17T21:30:44.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066411+0000", 2024-09-17T21:30:44.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017166818 2024-09-17T21:30:44.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066426+0000", 2024-09-17T21:30:44.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4888999999999999e-05 2024-09-17T21:30:44.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066611+0000", 2024-09-17T21:30:44.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018464499999999999 2024-09-17T21:30:44.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066630+0000", 2024-09-17T21:30:44.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9201000000000001e-05 2024-09-17T21:30:44.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:44.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067373+0000", 2024-09-17T21:30:44.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00074322600000000004 2024-09-17T21:30:44.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067407+0000", 2024-09-17T21:30:44.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4357999999999999e-05 2024-09-17T21:30:44.527 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.527 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.527 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.528 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50206 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'301, mlcod=45'299)", 2024-09-17T21:30:44.528 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:44.528 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.335877183000001, 2024-09-17T21:30:44.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068285810000000002, 2024-09-17T21:30:44.528 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.528 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.528 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:44.528 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:44.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:44.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178716+0000", 2024-09-17T21:30:44.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.438e-06 2024-09-17T21:30:44.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192044+0000", 2024-09-17T21:30:44.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013328370000000001 2024-09-17T21:30:44.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192069+0000", 2024-09-17T21:30:44.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4496000000000001e-05 2024-09-17T21:30:44.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192085+0000", 2024-09-17T21:30:44.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5758999999999999e-05 2024-09-17T21:30:44.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192167+0000", 2024-09-17T21:30:44.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2842000000000004e-05 2024-09-17T21:30:44.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192201+0000", 2024-09-17T21:30:44.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3127999999999999e-05 2024-09-17T21:30:44.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:44.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246938+0000", 2024-09-17T21:30:44.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054736982000000003 2024-09-17T21:30:44.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247000+0000", 2024-09-17T21:30:44.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2794999999999998e-05 2024-09-17T21:30:44.533 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.533 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.533 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.533 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50207 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'303, mlcod=45'299)", 2024-09-17T21:30:44.533 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:44.533 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.292930134000002, 2024-09-17T21:30:44.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026003433999999999, 2024-09-17T21:30:44.533 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.534 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.534 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:44.534 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:44.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:44.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221663+0000", 2024-09-17T21:30:44.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.271e-06 2024-09-17T21:30:44.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221672+0000", 2024-09-17T21:30:44.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6810000000000004e-06 2024-09-17T21:30:44.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221684+0000", 2024-09-17T21:30:44.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:30:44.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221691+0000", 2024-09-17T21:30:44.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:30:44.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221743+0000", 2024-09-17T21:30:44.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2868999999999998e-05 2024-09-17T21:30:44.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221765+0000", 2024-09-17T21:30:44.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1316999999999999e-05 2024-09-17T21:30:44.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:44.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247616+0000", 2024-09-17T21:30:44.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025851504000000001 2024-09-17T21:30:44.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247665+0000", 2024-09-17T21:30:44.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8739000000000002e-05 2024-09-17T21:30:44.538 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.538 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.538 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.538 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57808 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 196608~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:44.539 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:44.539 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.736631094, 2024-09-17T21:30:44.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017751475999999999, 2024-09-17T21:30:44.539 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.539 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.539 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.539 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.539 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.540 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.540 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57808 2024-09-17T21:30:44.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.540 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:44.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:44.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:44.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:30:44.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777968+0000", 2024-09-17T21:30:44.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8530000000000003e-06 2024-09-17T21:30:44.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777975+0000", 2024-09-17T21:30:44.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4410000000000004e-06 2024-09-17T21:30:44.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777980+0000", 2024-09-17T21:30:44.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7210000000000002e-06 2024-09-17T21:30:44.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795323+0000", 2024-09-17T21:30:44.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017343319999999999 2024-09-17T21:30:44.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795410+0000", 2024-09-17T21:30:44.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6567e-05 2024-09-17T21:30:44.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795712+0000", 2024-09-17T21:30:44.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030219200000000001 2024-09-17T21:30:44.544 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.544 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.544 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.544 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57809 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 458752~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:44.544 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:30:44.544 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.736605242, 2024-09-17T21:30:44.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018092869000000001, 2024-09-17T21:30:44.545 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.545 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.545 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.545 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57809 2024-09-17T21:30:44.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.545 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:30:44.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:30:44.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777987+0000", 2024-09-17T21:30:44.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T21:30:44.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777989+0000", 2024-09-17T21:30:44.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8649999999999999e-06 2024-09-17T21:30:44.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777994+0000", 2024-09-17T21:30:44.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4199999999999998e-06 2024-09-17T21:30:44.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778004+0000", 2024-09-17T21:30:44.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6900000000000004e-06 2024-09-17T21:30:44.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795745+0000", 2024-09-17T21:30:44.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017740859000000001 2024-09-17T21:30:44.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795815+0000", 2024-09-17T21:30:44.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0219e-05 2024-09-17T21:30:44.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796079+0000", 2024-09-17T21:30:44.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000264446 2024-09-17T21:30:44.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.549 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.550 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57810 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 720896~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:44.550 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:30:44.550 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.736581515000001, 2024-09-17T21:30:44.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018361253000000001, 2024-09-17T21:30:44.550 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.550 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.550 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.550 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57810 2024-09-17T21:30:44.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.551 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:30:44.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:30:44.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778011+0000", 2024-09-17T21:30:44.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:44.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778012+0000", 2024-09-17T21:30:44.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.872e-06 2024-09-17T21:30:44.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778018+0000", 2024-09-17T21:30:44.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8150000000000002e-06 2024-09-17T21:30:44.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778023+0000", 2024-09-17T21:30:44.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3939999999999998e-06 2024-09-17T21:30:44.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796106+0000", 2024-09-17T21:30:44.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018083446999999999 2024-09-17T21:30:44.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796159+0000", 2024-09-17T21:30:44.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3029999999999999e-05 2024-09-17T21:30:44.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796371+0000", 2024-09-17T21:30:44.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021231700000000001 2024-09-17T21:30:44.555 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.555 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.555 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.555 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57811 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:44.555 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:44.555 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.736563095000001, 2024-09-17T21:30:44.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018666035000000001, 2024-09-17T21:30:44.555 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.555 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.556 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.556 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.556 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.556 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.556 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57811 2024-09-17T21:30:44.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.556 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:44.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:44.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:44.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:30:44.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778031+0000", 2024-09-17T21:30:44.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.796e-06 2024-09-17T21:30:44.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778036+0000", 2024-09-17T21:30:44.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3290000000000001e-06 2024-09-17T21:30:44.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778041+0000", 2024-09-17T21:30:44.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5839999999999996e-06 2024-09-17T21:30:44.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796409+0000", 2024-09-17T21:30:44.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018368703 2024-09-17T21:30:44.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796483+0000", 2024-09-17T21:30:44.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3751999999999995e-05 2024-09-17T21:30:44.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796695+0000", 2024-09-17T21:30:44.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021153 2024-09-17T21:30:44.560 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.560 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.560 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.561 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57812 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1245184~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:44.561 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:44.561 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.736544198000001, 2024-09-17T21:30:44.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019000064000000001, 2024-09-17T21:30:44.561 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.561 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.561 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.561 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.561 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.562 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.562 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57812 2024-09-17T21:30:44.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.562 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:44.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:44.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:44.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:30:44.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778050+0000", 2024-09-17T21:30:44.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7e-06 2024-09-17T21:30:44.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778055+0000", 2024-09-17T21:30:44.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4049999999999999e-06 2024-09-17T21:30:44.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778059+0000", 2024-09-17T21:30:44.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1479999999999999e-06 2024-09-17T21:30:44.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796734+0000", 2024-09-17T21:30:44.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018674399000000001 2024-09-17T21:30:44.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796786+0000", 2024-09-17T21:30:44.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2879e-05 2024-09-17T21:30:44.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797048+0000", 2024-09-17T21:30:44.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026118400000000003 2024-09-17T21:30:44.566 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.566 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.566 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.566 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57813 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1507328~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:44.566 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:30:44.566 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.736526306000002, 2024-09-17T21:30:44.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019279392999999999, 2024-09-17T21:30:44.567 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.567 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.567 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.567 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.567 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.567 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.567 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57813 2024-09-17T21:30:44.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.567 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:30:44.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:30:44.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778066+0000", 2024-09-17T21:30:44.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6199999999999999e-07 2024-09-17T21:30:44.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778068+0000", 2024-09-17T21:30:44.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7990000000000001e-06 2024-09-17T21:30:44.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778074+0000", 2024-09-17T21:30:44.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:30:44.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778078+0000", 2024-09-17T21:30:44.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2579999999999997e-06 2024-09-17T21:30:44.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797077+0000", 2024-09-17T21:30:44.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018999180000000001 2024-09-17T21:30:44.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797123+0000", 2024-09-17T21:30:44.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5834999999999997e-05 2024-09-17T21:30:44.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797345+0000", 2024-09-17T21:30:44.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022191100000000001 2024-09-17T21:30:44.571 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.571 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.571 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.572 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57814 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:44.572 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:30:44.572 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.736507434, 2024-09-17T21:30:44.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019604947000000001, 2024-09-17T21:30:44.572 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.572 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.572 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.572 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.572 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.572 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.572 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57814 2024-09-17T21:30:44.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.573 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:30:44.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:30:44.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778085+0000", 2024-09-17T21:30:44.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:44.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778086+0000", 2024-09-17T21:30:44.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7850000000000001e-06 2024-09-17T21:30:44.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778093+0000", 2024-09-17T21:30:44.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6449999999999999e-06 2024-09-17T21:30:44.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778098+0000", 2024-09-17T21:30:44.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:30:44.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797383+0000", 2024-09-17T21:30:44.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019284879000000001 2024-09-17T21:30:44.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797431+0000", 2024-09-17T21:30:44.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8856000000000001e-05 2024-09-17T21:30:44.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797689+0000", 2024-09-17T21:30:44.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025785800000000002 2024-09-17T21:30:44.576 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.577 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.577 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.577 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57815 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2031616~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:44.577 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:44.577 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.736486765999999, 2024-09-17T21:30:44.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019910231, 2024-09-17T21:30:44.577 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.577 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.577 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.578 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.578 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.578 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.578 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57815 2024-09-17T21:30:44.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.578 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:44.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:44.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:44.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:30:44.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778118+0000", 2024-09-17T21:30:44.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2967e-05 2024-09-17T21:30:44.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778125+0000", 2024-09-17T21:30:44.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.601e-06 2024-09-17T21:30:44.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778130+0000", 2024-09-17T21:30:44.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1839999999999998e-06 2024-09-17T21:30:44.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797720+0000", 2024-09-17T21:30:44.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019589766000000002 2024-09-17T21:30:44.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797765+0000", 2024-09-17T21:30:44.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5281999999999998e-05 2024-09-17T21:30:44.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798015+0000", 2024-09-17T21:30:44.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025006200000000001 2024-09-17T21:30:44.582 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.582 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.582 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.582 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57816 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2293760~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:44.582 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:44.582 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.736454171999998, 2024-09-17T21:30:44.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020175322999999998, 2024-09-17T21:30:44.582 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.583 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.583 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.583 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57816 2024-09-17T21:30:44.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.583 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:44.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:44.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:44.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:30:44.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778140+0000", 2024-09-17T21:30:44.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T21:30:44.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778145+0000", 2024-09-17T21:30:44.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7320000000000003e-06 2024-09-17T21:30:44.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778154+0000", 2024-09-17T21:30:44.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6969999999999999e-06 2024-09-17T21:30:44.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798049+0000", 2024-09-17T21:30:44.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019894603 2024-09-17T21:30:44.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798106+0000", 2024-09-17T21:30:44.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7207000000000003e-05 2024-09-17T21:30:44.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798313+0000", 2024-09-17T21:30:44.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206861 2024-09-17T21:30:44.587 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.587 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.587 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.588 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57817 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2555904~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:44.588 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:44.588 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.736430926000001, 2024-09-17T21:30:44.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020466314999999999, 2024-09-17T21:30:44.588 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.588 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.588 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.588 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.588 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.588 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.588 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57817 2024-09-17T21:30:44.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.589 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:44.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:44.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:44.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2399999999999999e-07 2024-09-17T21:30:44.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778163+0000", 2024-09-17T21:30:44.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.204e-06 2024-09-17T21:30:44.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778169+0000", 2024-09-17T21:30:44.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:30:44.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778173+0000", 2024-09-17T21:30:44.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:30:44.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798342+0000", 2024-09-17T21:30:44.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020168416000000002 2024-09-17T21:30:44.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798388+0000", 2024-09-17T21:30:44.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6171000000000002e-05 2024-09-17T21:30:44.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798627+0000", 2024-09-17T21:30:44.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000239058 2024-09-17T21:30:44.593 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.593 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.593 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.593 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57818 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2818048~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:44.593 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:30:44.593 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.736412228999999, 2024-09-17T21:30:44.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020767240999999999, 2024-09-17T21:30:44.593 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.593 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.594 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.594 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.594 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.594 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.594 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57818 2024-09-17T21:30:44.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.594 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:30:44.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:30:44.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778180+0000", 2024-09-17T21:30:44.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:30:44.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778182+0000", 2024-09-17T21:30:44.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7269999999999999e-06 2024-09-17T21:30:44.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778188+0000", 2024-09-17T21:30:44.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6080000000000001e-06 2024-09-17T21:30:44.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778202+0000", 2024-09-17T21:30:44.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3369e-05 2024-09-17T21:30:44.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798660+0000", 2024-09-17T21:30:44.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020458171000000001 2024-09-17T21:30:44.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798704+0000", 2024-09-17T21:30:44.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4273000000000001e-05 2024-09-17T21:30:44.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798947+0000", 2024-09-17T21:30:44.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242651 2024-09-17T21:30:44.598 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.598 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.598 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.598 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57819 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:44.598 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:30:44.599 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.736381359999999, 2024-09-17T21:30:44.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020892340999999998, 2024-09-17T21:30:44.599 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.599 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.599 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.599 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.599 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.599 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.599 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57819 2024-09-17T21:30:44.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.599 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:30:44.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:30:44.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778211+0000", 2024-09-17T21:30:44.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:30:44.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778217+0000", 2024-09-17T21:30:44.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T21:30:44.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778224+0000", 2024-09-17T21:30:44.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3520000000000003e-06 2024-09-17T21:30:44.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778228+0000", 2024-09-17T21:30:44.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7480000000000001e-06 2024-09-17T21:30:44.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798977+0000", 2024-09-17T21:30:44.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020748571 2024-09-17T21:30:44.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799035+0000", 2024-09-17T21:30:44.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7908999999999997e-05 2024-09-17T21:30:44.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799103+0000", 2024-09-17T21:30:44.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7794999999999997e-05 2024-09-17T21:30:44.603 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.603 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.603 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.604 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57820 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:44.604 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:44.604 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.736356168, 2024-09-17T21:30:44.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021016658000000001, 2024-09-17T21:30:44.604 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.604 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.604 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.604 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.604 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.604 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.604 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57820 2024-09-17T21:30:44.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.605 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:44.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:44.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:44.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:30:44.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778238+0000", 2024-09-17T21:30:44.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.945e-06 2024-09-17T21:30:44.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778244+0000", 2024-09-17T21:30:44.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6520000000000003e-06 2024-09-17T21:30:44.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778248+0000", 2024-09-17T21:30:44.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5920000000000002e-06 2024-09-17T21:30:44.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799131+0000", 2024-09-17T21:30:44.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020883288999999999 2024-09-17T21:30:44.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799185+0000", 2024-09-17T21:30:44.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3223000000000002e-05 2024-09-17T21:30:44.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799252+0000", 2024-09-17T21:30:44.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7576999999999994e-05 2024-09-17T21:30:44.608 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.609 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.609 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.609 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57821 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:44.609 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:44.609 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.736337823, 2024-09-17T21:30:44.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021153528000000001, 2024-09-17T21:30:44.609 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.609 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.609 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.609 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.610 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.610 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.610 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57821 2024-09-17T21:30:44.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.610 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:44.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:44.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:44.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-07 2024-09-17T21:30:44.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778256+0000", 2024-09-17T21:30:44.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.604e-06 2024-09-17T21:30:44.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778262+0000", 2024-09-17T21:30:44.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7629999999999997e-06 2024-09-17T21:30:44.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778266+0000", 2024-09-17T21:30:44.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:30:44.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799284+0000", 2024-09-17T21:30:44.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021017312 2024-09-17T21:30:44.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799326+0000", 2024-09-17T21:30:44.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2639000000000003e-05 2024-09-17T21:30:44.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799407+0000", 2024-09-17T21:30:44.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1077999999999999e-05 2024-09-17T21:30:44.614 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.614 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.614 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.614 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57822 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:44.614 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:30:44.614 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.736319513000002, 2024-09-17T21:30:44.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021268966, 2024-09-17T21:30:44.614 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.615 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.615 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.615 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.615 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.615 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.615 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57822 2024-09-17T21:30:44.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.615 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:30:44.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:30:44.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778273+0000", 2024-09-17T21:30:44.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:30:44.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778274+0000", 2024-09-17T21:30:44.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.598e-06 2024-09-17T21:30:44.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778283+0000", 2024-09-17T21:30:44.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8279999999999992e-06 2024-09-17T21:30:44.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778289+0000", 2024-09-17T21:30:44.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5609999999999998e-06 2024-09-17T21:30:44.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799432+0000", 2024-09-17T21:30:44.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021143862999999999 2024-09-17T21:30:44.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799473+0000", 2024-09-17T21:30:44.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1075999999999998e-05 2024-09-17T21:30:44.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799541+0000", 2024-09-17T21:30:44.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7657999999999995e-05 2024-09-17T21:30:44.619 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.619 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.619 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.619 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57823 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:44.619 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:44.619 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.736296551999999, 2024-09-17T21:30:44.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021379770999999999, 2024-09-17T21:30:44.620 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:44.620 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:44.620 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:44.620 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:44.620 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:44.620 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:44.620 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57823 2024-09-17T21:30:44.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.620 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:44.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:44.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:44.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:44.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:44.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:44.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:44.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:44.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:30:44.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:44.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778297+0000", 2024-09-17T21:30:44.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:30:44.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:44.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778303+0000", 2024-09-17T21:30:44.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4559999999999999e-06 2024-09-17T21:30:44.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:44.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778307+0000", 2024-09-17T21:30:44.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6229999999999996e-06 2024-09-17T21:30:44.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:44.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799560+0000", 2024-09-17T21:30:44.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021252931999999999 2024-09-17T21:30:44.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:44.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799609+0000", 2024-09-17T21:30:44.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8301999999999999e-05 2024-09-17T21:30:44.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:44.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:44.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:44.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799675+0000", 2024-09-17T21:30:44.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6205999999999996e-05 2024-09-17T21:30:44.624 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.624 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.624 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.624 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:44.624 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:44.624 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:44.626 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:30:44.626 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:44.623+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:44.726 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:44.724+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:44.770 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:44.770 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:44.770 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:44.770 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:44.778 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:30:44.826 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T21:30:44.827 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:44.825+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:44.841 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-17T21:30:44.899 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone34 from 0xe356510 (overlap 0x7ad815) to 0x18919d 2024-09-17T21:30:44.927 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:44.925+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:44.950 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:44.950 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:44.950 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:44.950 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:44.950 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:44.963 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:30:45.029 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:45.026+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:45.083 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-17T21:30:45.252 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:45.250+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:45.253 INFO:teuthology.orchestra.run.smithi129.stdout:4796 write 0x8a0802b thru 0x8a0a24e (0x2224 bytes) 2024-09-17T21:30:45.253 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:45.253 INFO:teuthology.orchestra.run.smithi129.stdout:4799 trunc from 0xe356510 to 0xb8b061 2024-09-17T21:30:45.254 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:45.254 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:45.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:45.254 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:45.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:49361 3.19 e46/25 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:head v 46'135, mlcod=45'133)", 2024-09-17T21:30:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.380843405, 2024-09-17T21:30:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043477744999999998, 2024-09-17T21:30:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-07 2024-09-17T21:30:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769703+0000", 2024-09-17T21:30:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016134381 2024-09-17T21:30:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769724+0000", 2024-09-17T21:30:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0781999999999999e-05 2024-09-17T21:30:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769734+0000", 2024-09-17T21:30:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6870000000000006e-06 2024-09-17T21:30:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769797+0000", 2024-09-17T21:30:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3418999999999997e-05 2024-09-17T21:30:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769822+0000", 2024-09-17T21:30:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4652999999999999e-05 2024-09-17T21:30:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.796982+0000", 2024-09-17T21:30:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027160205 2024-09-17T21:30:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.797047+0000", 2024-09-17T21:30:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4455999999999994e-05 2024-09-17T21:30:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50805 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 17825792~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:30:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.639259911000003, 2024-09-17T21:30:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035591579999999998, 2024-09-17T21:30:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50805 2024-09-17T21:30:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:30:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:30:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495153+0000", 2024-09-17T21:30:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:30:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495155+0000", 2024-09-17T21:30:45.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1849999999999998e-06 2024-09-17T21:30:45.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495162+0000", 2024-09-17T21:30:45.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.64e-06 2024-09-17T21:30:45.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495166+0000", 2024-09-17T21:30:45.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:30:45.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530020+0000", 2024-09-17T21:30:45.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.034853824999999998 2024-09-17T21:30:45.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530076+0000", 2024-09-17T21:30:45.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5235999999999998e-05 2024-09-17T21:30:45.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530744+0000", 2024-09-17T21:30:45.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066835500000000003 2024-09-17T21:30:45.263 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.263 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.264 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.264 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50806 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19398656~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:45.264 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:45.264 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.639239443000001, 2024-09-17T21:30:45.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036408459999999997, 2024-09-17T21:30:45.264 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.264 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.264 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.264 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.264 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.264 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.265 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50806 2024-09-17T21:30:45.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.265 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:45.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:45.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:45.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.72e-07 2024-09-17T21:30:45.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495175+0000", 2024-09-17T21:30:45.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:30:45.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495182+0000", 2024-09-17T21:30:45.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8079999999999999e-06 2024-09-17T21:30:45.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495186+0000", 2024-09-17T21:30:45.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5549999999999996e-06 2024-09-17T21:30:45.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530791+0000", 2024-09-17T21:30:45.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035604963000000003 2024-09-17T21:30:45.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530855+0000", 2024-09-17T21:30:45.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3286000000000004e-05 2024-09-17T21:30:45.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531581+0000", 2024-09-17T21:30:45.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00072655699999999996 2024-09-17T21:30:45.268 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.268 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.268 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.268 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50807 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23068672~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:45.268 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:45.268 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.639220557999998, 2024-09-17T21:30:45.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037183314000000002, 2024-09-17T21:30:45.269 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.269 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.269 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.269 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.269 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.269 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.269 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50807 2024-09-17T21:30:45.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.269 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:45.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:45.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:45.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9999999999999999e-07 2024-09-17T21:30:45.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495194+0000", 2024-09-17T21:30:45.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.779e-06 2024-09-17T21:30:45.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495199+0000", 2024-09-17T21:30:45.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.468e-06 2024-09-17T21:30:45.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495203+0000", 2024-09-17T21:30:45.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0010000000000003e-06 2024-09-17T21:30:45.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531636+0000", 2024-09-17T21:30:45.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036432665000000003 2024-09-17T21:30:45.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531727+0000", 2024-09-17T21:30:45.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1607999999999994e-05 2024-09-17T21:30:45.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532375+0000", 2024-09-17T21:30:45.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00064749299999999996 2024-09-17T21:30:45.273 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.273 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.273 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.273 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50808 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24641536~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:45.273 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:30:45.273 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.639203815000002, 2024-09-17T21:30:45.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037870190999999997, 2024-09-17T21:30:45.273 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.273 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.273 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.273 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.274 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.274 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.274 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50808 2024-09-17T21:30:45.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.274 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:30:45.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:30:45.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495209+0000", 2024-09-17T21:30:45.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:30:45.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495215+0000", 2024-09-17T21:30:45.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1630000000000001e-06 2024-09-17T21:30:45.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495221+0000", 2024-09-17T21:30:45.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-17T21:30:45.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495226+0000", 2024-09-17T21:30:45.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8189999999999999e-06 2024-09-17T21:30:45.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532425+0000", 2024-09-17T21:30:45.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037199135000000001 2024-09-17T21:30:45.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532531+0000", 2024-09-17T21:30:45.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106314 2024-09-17T21:30:45.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533079+0000", 2024-09-17T21:30:45.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00054739700000000003 2024-09-17T21:30:45.277 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.277 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.277 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.278 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50809 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28311552~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:45.278 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:30:45.278 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.639180848999999, 2024-09-17T21:30:45.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038597605, 2024-09-17T21:30:45.278 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.278 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.278 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.278 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.278 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.278 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.278 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50809 2024-09-17T21:30:45.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.278 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:30:45.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:30:45.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495232+0000", 2024-09-17T21:30:45.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.39e-07 2024-09-17T21:30:45.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495234+0000", 2024-09-17T21:30:45.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T21:30:45.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495239+0000", 2024-09-17T21:30:45.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7969999999999997e-06 2024-09-17T21:30:45.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495244+0000", 2024-09-17T21:30:45.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999996e-06 2024-09-17T21:30:45.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533125+0000", 2024-09-17T21:30:45.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037881523 2024-09-17T21:30:45.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533237+0000", 2024-09-17T21:30:45.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000111879 2024-09-17T21:30:45.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533829+0000", 2024-09-17T21:30:45.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000591933 2024-09-17T21:30:45.282 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.282 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.282 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.282 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50810 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 29884416~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:45.282 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:30:45.282 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.639162931000001, 2024-09-17T21:30:45.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039149809000000001, 2024-09-17T21:30:45.282 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.282 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.283 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.283 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50810 2024-09-17T21:30:45.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.283 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:30:45.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:30:45.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495250+0000", 2024-09-17T21:30:45.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:30:45.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495251+0000", 2024-09-17T21:30:45.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.81e-06 2024-09-17T21:30:45.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495257+0000", 2024-09-17T21:30:45.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1399999999999999e-06 2024-09-17T21:30:45.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495261+0000", 2024-09-17T21:30:45.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4229999999999998e-06 2024-09-17T21:30:45.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533879+0000", 2024-09-17T21:30:45.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038618292999999998 2024-09-17T21:30:45.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533939+0000", 2024-09-17T21:30:45.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.012e-05 2024-09-17T21:30:45.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534399+0000", 2024-09-17T21:30:45.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00045968099999999998 2024-09-17T21:30:45.286 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.286 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.286 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.287 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50850 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 2359296~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:45.287 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:45.287 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.639144555999998, 2024-09-17T21:30:45.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039731491000000001, 2024-09-17T21:30:45.287 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.287 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.287 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.287 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50850 2024-09-17T21:30:45.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.288 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:45.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:45.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:45.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3200000000000001e-07 2024-09-17T21:30:45.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495270+0000", 2024-09-17T21:30:45.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.694e-06 2024-09-17T21:30:45.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495275+0000", 2024-09-17T21:30:45.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5890000000000002e-06 2024-09-17T21:30:45.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495279+0000", 2024-09-17T21:30:45.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-17T21:30:45.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534440+0000", 2024-09-17T21:30:45.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039160805999999999 2024-09-17T21:30:45.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534526+0000", 2024-09-17T21:30:45.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5327999999999993e-05 2024-09-17T21:30:45.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534999+0000", 2024-09-17T21:30:45.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00047353400000000001 2024-09-17T21:30:45.291 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.291 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.291 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.291 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50851 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 3932160~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:45.291 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:30:45.291 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.639126943999997, 2024-09-17T21:30:45.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040211991000000002, 2024-09-17T21:30:45.291 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.292 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.292 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.292 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.292 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.292 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.292 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50851 2024-09-17T21:30:45.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.292 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:30:45.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:30:45.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495286+0000", 2024-09-17T21:30:45.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:45.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495287+0000", 2024-09-17T21:30:45.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6640000000000001e-06 2024-09-17T21:30:45.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495293+0000", 2024-09-17T21:30:45.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6710000000000004e-06 2024-09-17T21:30:45.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495298+0000", 2024-09-17T21:30:45.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6700000000000002e-06 2024-09-17T21:30:45.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535017+0000", 2024-09-17T21:30:45.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039719059000000001 2024-09-17T21:30:45.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535067+0000", 2024-09-17T21:30:45.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0046000000000002e-05 2024-09-17T21:30:45.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535497+0000", 2024-09-17T21:30:45.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043050299999999999 2024-09-17T21:30:45.295 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.295 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.295 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.296 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50852 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 7602176~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:45.296 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:30:45.296 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.639108026999999, 2024-09-17T21:30:45.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041166337999999997, 2024-09-17T21:30:45.296 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.296 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.296 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.296 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.296 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.296 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.296 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50852 2024-09-17T21:30:45.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.297 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:30:45.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:30:45.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495305+0000", 2024-09-17T21:30:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1600000000000002e-07 2024-09-17T21:30:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495307+0000", 2024-09-17T21:30:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0980000000000001e-06 2024-09-17T21:30:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495312+0000", 2024-09-17T21:30:45.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6210000000000001e-06 2024-09-17T21:30:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495317+0000", 2024-09-17T21:30:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T21:30:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535531+0000", 2024-09-17T21:30:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040214580999999999 2024-09-17T21:30:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535618+0000", 2024-09-17T21:30:45.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6648000000000001e-05 2024-09-17T21:30:45.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536471+0000", 2024-09-17T21:30:45.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00085251599999999999 2024-09-17T21:30:45.300 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.300 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.300 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.300 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50853 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 9175040~204800] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:45.300 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:45.301 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.639089695999999, 2024-09-17T21:30:45.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041673496999999997, 2024-09-17T21:30:45.301 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.301 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.301 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.301 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.301 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.301 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.301 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50853 2024-09-17T21:30:45.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.301 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:45.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:45.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:45.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:30:45.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495334+0000", 2024-09-17T21:30:45.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.076e-05 2024-09-17T21:30:45.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495340+0000", 2024-09-17T21:30:45.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:30:45.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495345+0000", 2024-09-17T21:30:45.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:30:45.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536514+0000", 2024-09-17T21:30:45.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041168534999999999 2024-09-17T21:30:45.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536572+0000", 2024-09-17T21:30:45.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8742000000000001e-05 2024-09-17T21:30:45.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536996+0000", 2024-09-17T21:30:45.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042368899999999999 2024-09-17T21:30:45.305 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.305 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.305 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.305 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50854 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 12845056~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:45.305 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:45.305 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.639060637, 2024-09-17T21:30:45.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042777397000000002, 2024-09-17T21:30:45.306 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.306 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.306 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.306 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.306 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.306 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.306 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50854 2024-09-17T21:30:45.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.306 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:45.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:45.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:45.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.82e-07 2024-09-17T21:30:45.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495354+0000", 2024-09-17T21:30:45.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.091e-06 2024-09-17T21:30:45.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495360+0000", 2024-09-17T21:30:45.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.716e-06 2024-09-17T21:30:45.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495365+0000", 2024-09-17T21:30:45.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2939999999999996e-06 2024-09-17T21:30:45.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537020+0000", 2024-09-17T21:30:45.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041654419999999998 2024-09-17T21:30:45.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537105+0000", 2024-09-17T21:30:45.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5857000000000001e-05 2024-09-17T21:30:45.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538129+0000", 2024-09-17T21:30:45.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010235369999999999 2024-09-17T21:30:45.310 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.310 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.310 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.310 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50855 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 14417920~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:45.310 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:45.310 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.639040672, 2024-09-17T21:30:45.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043254325000000003, 2024-09-17T21:30:45.310 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.310 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.310 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.310 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.310 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.311 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.311 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50855 2024-09-17T21:30:45.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.311 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:45.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:45.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:45.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:30:45.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495374+0000", 2024-09-17T21:30:45.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.922e-06 2024-09-17T21:30:45.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495379+0000", 2024-09-17T21:30:45.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-17T21:30:45.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495384+0000", 2024-09-17T21:30:45.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-17T21:30:45.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538167+0000", 2024-09-17T21:30:45.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042782845 2024-09-17T21:30:45.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538218+0000", 2024-09-17T21:30:45.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1335000000000002e-05 2024-09-17T21:30:45.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538626+0000", 2024-09-17T21:30:45.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000407656 2024-09-17T21:30:45.314 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.314 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.314 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.315 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50856 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 18087936~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:45.315 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:45.315 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.639022216999997, 2024-09-17T21:30:45.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043911444000000001, 2024-09-17T21:30:45.315 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.315 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.315 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.315 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.315 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.315 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.315 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50856 2024-09-17T21:30:45.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.315 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:45.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:45.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:45.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0499999999999999e-07 2024-09-17T21:30:45.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495392+0000", 2024-09-17T21:30:45.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7430000000000001e-06 2024-09-17T21:30:45.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495398+0000", 2024-09-17T21:30:45.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3040000000000001e-06 2024-09-17T21:30:45.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495403+0000", 2024-09-17T21:30:45.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5600000000000004e-06 2024-09-17T21:30:45.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538660+0000", 2024-09-17T21:30:45.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043256832000000002 2024-09-17T21:30:45.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538746+0000", 2024-09-17T21:30:45.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6448999999999999e-05 2024-09-17T21:30:45.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539301+0000", 2024-09-17T21:30:45.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055525099999999999 2024-09-17T21:30:45.319 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.319 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.319 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.319 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50857 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19660800~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:45.319 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:30:45.319 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.639002812000001, 2024-09-17T21:30:45.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044408156999999997, 2024-09-17T21:30:45.319 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.320 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.320 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.320 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.320 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.320 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.320 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50857 2024-09-17T21:30:45.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.320 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:30:45.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:30:45.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495410+0000", 2024-09-17T21:30:45.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:30:45.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495412+0000", 2024-09-17T21:30:45.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9580000000000001e-06 2024-09-17T21:30:45.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495418+0000", 2024-09-17T21:30:45.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1909999999999997e-06 2024-09-17T21:30:45.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495423+0000", 2024-09-17T21:30:45.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2429999999999996e-06 2024-09-17T21:30:45.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539338+0000", 2024-09-17T21:30:45.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043914478 2024-09-17T21:30:45.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539393+0000", 2024-09-17T21:30:45.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5226999999999998e-05 2024-09-17T21:30:45.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539818+0000", 2024-09-17T21:30:45.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042469300000000002 2024-09-17T21:30:45.323 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.323 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.324 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.324 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50858 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23330816~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:45.324 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:45.324 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.638981715, 2024-09-17T21:30:45.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045010798999999997, 2024-09-17T21:30:45.324 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.324 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.324 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.325 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50858 2024-09-17T21:30:45.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.325 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:45.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:45.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:45.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:30:45.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495433+0000", 2024-09-17T21:30:45.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.787e-06 2024-09-17T21:30:45.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495438+0000", 2024-09-17T21:30:45.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:30:45.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495442+0000", 2024-09-17T21:30:45.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2640000000000002e-06 2024-09-17T21:30:45.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539852+0000", 2024-09-17T21:30:45.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044409592999999997 2024-09-17T21:30:45.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539940+0000", 2024-09-17T21:30:45.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7696999999999997e-05 2024-09-17T21:30:45.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540441+0000", 2024-09-17T21:30:45.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050164299999999997 2024-09-17T21:30:45.328 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.328 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.328 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.329 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50859 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24903680~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:45.329 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:30:45.329 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.638964037000001, 2024-09-17T21:30:45.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045571569999999999, 2024-09-17T21:30:45.329 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.329 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.329 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.329 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50859 2024-09-17T21:30:45.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.329 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:30:45.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:30:45.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495449+0000", 2024-09-17T21:30:45.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:30:45.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495450+0000", 2024-09-17T21:30:45.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7889999999999999e-06 2024-09-17T21:30:45.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495456+0000", 2024-09-17T21:30:45.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5679999999999999e-06 2024-09-17T21:30:45.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495460+0000", 2024-09-17T21:30:45.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4989999999999996e-06 2024-09-17T21:30:45.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540472+0000", 2024-09-17T21:30:45.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045011585999999999 2024-09-17T21:30:45.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540554+0000", 2024-09-17T21:30:45.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2443999999999999e-05 2024-09-17T21:30:45.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541020+0000", 2024-09-17T21:30:45.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00046534300000000001 2024-09-17T21:30:45.333 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.333 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.333 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.333 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50860 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28573696~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:45.333 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:45.333 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.638945683999999, 2024-09-17T21:30:45.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046167899999999998, 2024-09-17T21:30:45.334 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.334 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.334 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.334 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.334 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.334 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.334 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50860 2024-09-17T21:30:45.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.334 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:45.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:45.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:45.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:30:45.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495469+0000", 2024-09-17T21:30:45.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.99e-06 2024-09-17T21:30:45.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495475+0000", 2024-09-17T21:30:45.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:30:45.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495480+0000", 2024-09-17T21:30:45.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:30:45.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541050+0000", 2024-09-17T21:30:45.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045569853 2024-09-17T21:30:45.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541138+0000", 2024-09-17T21:30:45.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7997000000000005e-05 2024-09-17T21:30:45.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541634+0000", 2024-09-17T21:30:45.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00049685500000000004 2024-09-17T21:30:45.338 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.338 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.338 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.338 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50861 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 30146560~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:45.338 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:45.338 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.638926507000001, 2024-09-17T21:30:45.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046690553000000003, 2024-09-17T21:30:45.338 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.338 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.338 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.338 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.338 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.339 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.339 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50861 2024-09-17T21:30:45.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.339 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:45.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:45.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:45.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:30:45.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495488+0000", 2024-09-17T21:30:45.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:30:45.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495495+0000", 2024-09-17T21:30:45.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:30:45.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495500+0000", 2024-09-17T21:30:45.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8620000000000002e-06 2024-09-17T21:30:45.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541687+0000", 2024-09-17T21:30:45.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046186434999999998 2024-09-17T21:30:45.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541741+0000", 2024-09-17T21:30:45.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4228999999999999e-05 2024-09-17T21:30:45.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.542176+0000", 2024-09-17T21:30:45.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043531100000000002 2024-09-17T21:30:45.342 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.342 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.342 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.342 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97804 3.7 3:e725aa05:::rbd_data.1120c318df8c.0000000000000003:24 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:45.343 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:30:45.343 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.172980406, 2024-09-17T21:30:45.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035514416, 2024-09-17T21:30:45.343 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:45.343 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:45.343 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:45.343 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:45.343 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:45.343 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.343 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97804 2024-09-17T21:30:45.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.344 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:45.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:45.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:30:45.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:45.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:30:45.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:45.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:45.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:30:45.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:30:45.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:45.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961434+0000", 2024-09-17T21:30:45.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8959999999999999e-06 2024-09-17T21:30:45.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:45.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961439+0000", 2024-09-17T21:30:45.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T21:30:45.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961444+0000", 2024-09-17T21:30:45.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4970000000000003e-06 2024-09-17T21:30:45.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:45.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996466+0000", 2024-09-17T21:30:45.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035022438000000003 2024-09-17T21:30:45.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:45.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996556+0000", 2024-09-17T21:30:45.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9561000000000005e-05 2024-09-17T21:30:45.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:45.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:45.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:45.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996946+0000", 2024-09-17T21:30:45.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039028700000000001 2024-09-17T21:30:45.347 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.347 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.347 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.347 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:45.347 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:45.347 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:45.348 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:30:45.351 INFO:teuthology.orchestra.run.smithi129.stdout:4800 writesame 0xcb668aa thru 0xcb69ad9 data_size 0x2c(0x3230 bytes) 2024-09-17T21:30:45.352 INFO:teuthology.orchestra.run.smithi129.stdout:4801 read 0xc50a458 thru 0xc512f10 (0x8ab9 bytes) 2024-09-17T21:30:45.352 INFO:teuthology.orchestra.run.smithi129.stdout:4802 write 0xd10656f thru 0xd109c1b (0x36ad bytes) 2024-09-17T21:30:45.352 INFO:teuthology.orchestra.run.smithi129.stdout:4803 read 0x21a2f9f thru 0x21a7f89 (0x4feb bytes) 2024-09-17T21:30:45.352 INFO:teuthology.orchestra.run.smithi129.stdout:4806 trunc from 0xd109c1c to 0xbc0962e 2024-09-17T21:30:45.352 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:45.352 INFO:teuthology.orchestra.run.smithi129.stdout:4811 read 0x619b812 thru 0x61a1993 (0x6182 bytes) 2024-09-17T21:30:45.352 INFO:teuthology.orchestra.run.smithi129.stdout:4813 write 0x5faaaf8 thru 0x5fb17ee (0x6cf7 bytes) 2024-09-17T21:30:45.352 INFO:teuthology.orchestra.run.smithi129.stdout:4815 write 0x5ef602d thru 0x5ef7a3a (0x1a0e bytes) 2024-09-17T21:30:45.352 INFO:teuthology.orchestra.run.smithi129.stdout:4816 read 0x2c96af5 thru 0x2c9cc3c (0x6148 bytes) 2024-09-17T21:30:45.352 INFO:teuthology.orchestra.run.smithi129.stdout:4817 punch from 0x85a22d5 to 0x85aedf7, (0xcb22 bytes) 2024-09-17T21:30:45.352 INFO:teuthology.orchestra.run.smithi129.stdout:4818 write 0x3d0745a thru 0x3d107fa (0x93a1 bytes) 2024-09-17T21:30:45.352 INFO:teuthology.orchestra.run.smithi129.stdout:4819 writesame 0xc5ac42 thru 0xc5e3ab data_size 0x29(0x376a bytes) 2024-09-17T21:30:45.352 INFO:teuthology.orchestra.run.smithi129.stdout:4821 writesame 0x7dae130 thru 0x7dba77d data_size 0x3(0xc64e bytes) 2024-09-17T21:30:45.353 INFO:teuthology.orchestra.run.smithi129.stdout:4822 write 0xbf95b54 thru 0xbf9f2af (0x975c bytes) 2024-09-17T21:30:45.353 INFO:teuthology.orchestra.run.smithi129.stdout:4823 read 0x37ed84b thru 0x37f6a80 (0x9236 bytes) 2024-09-17T21:30:45.353 INFO:teuthology.orchestra.run.smithi129.stdout:4824 writesame 0x8b48d1f thru 0x8b5242e data_size 0x8(0x9710 bytes) 2024-09-17T21:30:45.353 INFO:teuthology.orchestra.run.smithi129.stdout:4825 writesame 0xae87f01 thru 0xae93976 data_size 0x2(0xba76 bytes) 2024-09-17T21:30:45.353 INFO:teuthology.orchestra.run.smithi129.stdout:4826 read 0xbab99cc thru 0xbac5c18 (0xc24d bytes) 2024-09-17T21:30:45.353 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:45.353 INFO:teuthology.orchestra.run.smithi129.stdout:4829 writesame 0x14dd783 thru 0x14df29f data_size 0x1(0x1b1d bytes) 2024-09-17T21:30:45.353 INFO:teuthology.orchestra.run.smithi129.stdout:4832 writesame 0x386cd8d thru 0x387491b data_size 0x1(0x7b8f bytes) 2024-09-17T21:30:45.353 INFO:teuthology.orchestra.run.smithi129.stdout:4835 trunc from 0xbf9f2b0 to 0x6b2365a 2024-09-17T21:30:45.353 INFO:teuthology.orchestra.run.smithi129.stdout:4836 punch from 0x48f354f to 0x48f415c, (0xc0d bytes) 2024-09-17T21:30:45.353 INFO:teuthology.orchestra.run.smithi129.stdout:4838 punch from 0xce87f1 to 0xcf41e5, (0xb9f4 bytes) 2024-09-17T21:30:45.353 INFO:teuthology.orchestra.run.smithi129.stdout:4839 writesame 0x44657d thru 0x451b99 data_size 0x1(0xb61d bytes) 2024-09-17T21:30:45.353 INFO:teuthology.orchestra.run.smithi129.stdout:4840 trunc from 0x6b2365a to 0xc70ab8c 2024-09-17T21:30:45.353 INFO:teuthology.orchestra.run.smithi129.stdout:4841 read 0xa3190b6 thru 0xa31fda2 (0x6ced bytes) 2024-09-17T21:30:45.354 INFO:teuthology.orchestra.run.smithi129.stdout:4842 read 0x159f8d5 thru 0x15aa48b (0xabb7 bytes) 2024-09-17T21:30:45.354 INFO:teuthology.orchestra.run.smithi129.stdout:4844 write 0xbcceb23 thru 0xbcd7339 (0x8817 bytes) 2024-09-17T21:30:45.354 INFO:teuthology.orchestra.run.smithi129.stdout:4846 punch from 0x4645d9b to 0x46509eb, (0xac50 bytes) 2024-09-17T21:30:45.354 INFO:teuthology.orchestra.run.smithi129.stdout:4847 read 0x825c44a thru 0x826ab64 (0xe71b bytes) 2024-09-17T21:30:45.354 INFO:teuthology.orchestra.run.smithi129.stdout:4848 punch from 0xc57cfbb to 0xc5819be, (0x4a03 bytes) 2024-09-17T21:30:45.354 INFO:teuthology.orchestra.run.smithi129.stdout:4849 trunc from 0xc70ab8c to 0x945c1eb 2024-09-17T21:30:45.354 INFO:teuthology.orchestra.run.smithi129.stdout:4850 read 0x2d8b321 thru 0x2d99390 (0xe070 bytes) 2024-09-17T21:30:45.354 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:45.354 INFO:teuthology.orchestra.run.smithi129.stdout:4852 punch from 0x60aaf79 to 0x60ae052, (0x30d9 bytes) 2024-09-17T21:30:45.355 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:45.353+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:45.357 INFO:teuthology.orchestra.run.smithi129.stdout:4854 write 0xc735592 thru 0xc74454b (0xefba bytes) 2024-09-17T21:30:45.362 INFO:teuthology.orchestra.run.smithi129.stdout:4855 read 0x978a0a thru 0x9865a0 (0xdb97 bytes) 2024-09-17T21:30:45.371 INFO:teuthology.orchestra.run.smithi129.stdout:4856 write 0xec5b4ce thru 0xec5b69c (0x1cf bytes) 2024-09-17T21:30:45.374 INFO:teuthology.orchestra.run.smithi129.stdout:4857 read 0xf04e16 thru 0xf0fdac (0xaf97 bytes) 2024-09-17T21:30:45.375 INFO:teuthology.orchestra.run.smithi129.stdout:4858 write 0x13d9d9e thru 0x13e0a15 (0x6c78 bytes) 2024-09-17T21:30:45.378 INFO:teuthology.orchestra.run.smithi129.stdout:4859 writesame 0xe669efd thru 0xe66abbf data_size 0x1b(0xcc3 bytes) 2024-09-17T21:30:45.381 INFO:teuthology.orchestra.run.smithi129.stdout:4861 write 0x600e65c thru 0x60148e4 (0x6289 bytes) 2024-09-17T21:30:45.385 INFO:teuthology.orchestra.run.smithi129.stdout:4862 punch from 0xd9e39ac to 0xd9e9f4b, (0x659f bytes) 2024-09-17T21:30:45.385 INFO:teuthology.orchestra.run.smithi129.stdout:4864 punch from 0x26e5f70 to 0x26f3789, (0xd819 bytes) 2024-09-17T21:30:45.395 INFO:teuthology.orchestra.run.smithi129.stdout:4865 trunc from 0xec5b69d to 0x5d4d853 2024-09-17T21:30:45.406 INFO:teuthology.orchestra.run.smithi129.stdout:4866 read 0x14de357 thru 0x14edaff (0xf7a9 bytes) 2024-09-17T21:30:45.407 INFO:teuthology.orchestra.run.smithi129.stdout:4867 read 0x28d6def thru 0x28e4dd5 (0xdfe7 bytes) 2024-09-17T21:30:45.407 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:45.407 INFO:teuthology.orchestra.run.smithi129.stdout:4869 punch from 0x4afb55f to 0x4afb807, (0x2a8 bytes) 2024-09-17T21:30:45.412 INFO:teuthology.orchestra.run.smithi129.stdout:4870 trunc from 0x5d4d853 to 0xa4b1406 2024-09-17T21:30:45.418 INFO:teuthology.orchestra.run.smithi129.stdout:4871 punch from 0x51dafdf to 0x51e6f3e, (0xbf5f bytes) 2024-09-17T21:30:45.418 INFO:teuthology.orchestra.run.smithi129.stdout:4873 read 0x7cc6b54 thru 0x7cd1442 (0xa8ef bytes) 2024-09-17T21:30:45.418 INFO:teuthology.orchestra.run.smithi129.stdout:4874 read 0x1a99255 thru 0x1aa8f55 (0xfd01 bytes) 2024-09-17T21:30:45.423 INFO:teuthology.orchestra.run.smithi129.stdout:4876 trunc from 0xa4b1406 to 0x62817e2 2024-09-17T21:30:45.428 INFO:teuthology.orchestra.run.smithi129.stdout:4877 punch from 0x868075 to 0x875d97, (0xdd22 bytes) 2024-09-17T21:30:45.429 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:45.429 INFO:teuthology.orchestra.run.smithi129.stdout:4879 clone 36 order 21 su 131072 sc 7 2024-09-17T21:30:45.456 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:45.454+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:45.459 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:45.459 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:45.459 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:45.459 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:45.467 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:30:45.557 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:45.555+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:45.611 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:45.611 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:45.611 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:45.611 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:45.611 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:45.619 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:30:45.660 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:45.658+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:45.733 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:30:45.764 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21342 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2082816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.084330993999998, 2024-09-17T21:30:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022083523000000001, 2024-09-17T21:30:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21342 2024-09-17T21:30:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:30:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676659+0000", 2024-09-17T21:30:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0950000000000001e-06 2024-09-17T21:30:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676662+0000", 2024-09-17T21:30:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.971e-06 2024-09-17T21:30:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676665+0000", 2024-09-17T21:30:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6189999999999998e-06 2024-09-17T21:30:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698568+0000", 2024-09-17T21:30:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021903549000000001 2024-09-17T21:30:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698668+0000", 2024-09-17T21:30:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9778999999999995e-05 2024-09-17T21:30:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698741+0000", 2024-09-17T21:30:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000006e-05 2024-09-17T21:30:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21343 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2095104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.084321056, 2024-09-17T21:30:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022504024000000001, 2024-09-17T21:30:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21343 2024-09-17T21:30:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:30:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676669+0000", 2024-09-17T21:30:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.003e-06 2024-09-17T21:30:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676672+0000", 2024-09-17T21:30:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9280000000000002e-06 2024-09-17T21:30:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676674+0000", 2024-09-17T21:30:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5469999999999999e-06 2024-09-17T21:30:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699015+0000", 2024-09-17T21:30:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022340563000000001 2024-09-17T21:30:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699102+0000", 2024-09-17T21:30:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7425999999999999e-05 2024-09-17T21:30:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699172+0000", 2024-09-17T21:30:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9346999999999999e-05 2024-09-17T21:30:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45576 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 39.056087255999998, 2024-09-17T21:30:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022024660000000001, 2024-09-17T21:30:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45576 2024-09-17T21:30:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704902+0000", 2024-09-17T21:30:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9100000000000004e-07 2024-09-17T21:30:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704905+0000", 2024-09-17T21:30:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6139999999999999e-06 2024-09-17T21:30:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704911+0000", 2024-09-17T21:30:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8619999999999997e-06 2024-09-17T21:30:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704917+0000", 2024-09-17T21:30:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:30:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726681+0000", 2024-09-17T21:30:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021764569000000001 2024-09-17T21:30:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726726+0000", 2024-09-17T21:30:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4218999999999998e-05 2024-09-17T21:30:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726926+0000", 2024-09-17T21:30:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000200466 2024-09-17T21:30:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45577 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 39.056063600000002, 2024-09-17T21:30:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022269694999999999, 2024-09-17T21:30:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45577 2024-09-17T21:30:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704926+0000", 2024-09-17T21:30:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:30:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704928+0000", 2024-09-17T21:30:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9989999999999998e-06 2024-09-17T21:30:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704934+0000", 2024-09-17T21:30:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:30:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704939+0000", 2024-09-17T21:30:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5909999999999998e-06 2024-09-17T21:30:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726954+0000", 2024-09-17T21:30:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022014885000000001 2024-09-17T21:30:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726995+0000", 2024-09-17T21:30:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1474000000000003e-05 2024-09-17T21:30:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727195+0000", 2024-09-17T21:30:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019968300000000001 2024-09-17T21:30:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45578 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 39.056042355999999, 2024-09-17T21:30:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022484006000000001, 2024-09-17T21:30:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45578 2024-09-17T21:30:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704947+0000", 2024-09-17T21:30:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1399999999999997e-07 2024-09-17T21:30:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704949+0000", 2024-09-17T21:30:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0959999999999999e-06 2024-09-17T21:30:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704956+0000", 2024-09-17T21:30:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9330000000000004e-06 2024-09-17T21:30:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704961+0000", 2024-09-17T21:30:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6759999999999998e-06 2024-09-17T21:30:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727213+0000", 2024-09-17T21:30:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022252568 2024-09-17T21:30:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727259+0000", 2024-09-17T21:30:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5238000000000003e-05 2024-09-17T21:30:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727430+0000", 2024-09-17T21:30:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000171881 2024-09-17T21:30:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45579 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 39.056020427999997, 2024-09-17T21:30:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022749080000000001, 2024-09-17T21:30:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45579 2024-09-17T21:30:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704969+0000", 2024-09-17T21:30:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:30:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704971+0000", 2024-09-17T21:30:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.097e-06 2024-09-17T21:30:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704977+0000", 2024-09-17T21:30:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6329999999999999e-06 2024-09-17T21:30:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704983+0000", 2024-09-17T21:30:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:30:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727457+0000", 2024-09-17T21:30:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022474521000000001 2024-09-17T21:30:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727498+0000", 2024-09-17T21:30:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0558e-05 2024-09-17T21:30:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727717+0000", 2024-09-17T21:30:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219805 2024-09-17T21:30:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45652 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 39.054874290000001, 2024-09-17T21:30:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022100465999999999, 2024-09-17T21:30:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45652 2024-09-17T21:30:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706115+0000", 2024-09-17T21:30:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:30:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706117+0000", 2024-09-17T21:30:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-06 2024-09-17T21:30:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706122+0000", 2024-09-17T21:30:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.288e-06 2024-09-17T21:30:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706136+0000", 2024-09-17T21:30:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3409e-05 2024-09-17T21:30:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728112+0000", 2024-09-17T21:30:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021976724999999999 2024-09-17T21:30:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728151+0000", 2024-09-17T21:30:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8318999999999997e-05 2024-09-17T21:30:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728215+0000", 2024-09-17T21:30:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4172999999999997e-05 2024-09-17T21:30:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45653 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 39.05484508, 2024-09-17T21:30:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022249517999999999, 2024-09-17T21:30:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45653 2024-09-17T21:30:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T21:30:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706147+0000", 2024-09-17T21:30:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4430000000000002e-06 2024-09-17T21:30:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706152+0000", 2024-09-17T21:30:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5350000000000004e-06 2024-09-17T21:30:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706157+0000", 2024-09-17T21:30:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8420000000000004e-06 2024-09-17T21:30:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728246+0000", 2024-09-17T21:30:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022089237000000001 2024-09-17T21:30:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728309+0000", 2024-09-17T21:30:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2356999999999999e-05 2024-09-17T21:30:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:30:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4596000000000001e-05 2024-09-17T21:30:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45657 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 36864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 39.054813815999999, 2024-09-17T21:30:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022218252000000001, 2024-09-17T21:30:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45657 2024-09-17T21:30:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8299999999999997e-07 2024-09-17T21:30:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706178+0000", 2024-09-17T21:30:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3700000000000002e-06 2024-09-17T21:30:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706184+0000", 2024-09-17T21:30:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.144e-06 2024-09-17T21:30:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706190+0000", 2024-09-17T21:30:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8590000000000001e-06 2024-09-17T21:30:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728240+0000", 2024-09-17T21:30:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022049804999999999 2024-09-17T21:30:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728307+0000", 2024-09-17T21:30:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7477999999999993e-05 2024-09-17T21:30:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:30:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6112999999999994e-05 2024-09-17T21:30:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45658 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 39.054790695999998, 2024-09-17T21:30:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022376455, 2024-09-17T21:30:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45658 2024-09-17T21:30:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:30:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706200+0000", 2024-09-17T21:30:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:30:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706206+0000", 2024-09-17T21:30:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7039999999999999e-06 2024-09-17T21:30:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706211+0000", 2024-09-17T21:30:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9429999999999999e-06 2024-09-17T21:30:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728418+0000", 2024-09-17T21:30:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022206760999999998 2024-09-17T21:30:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728497+0000", 2024-09-17T21:30:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9493999999999995e-05 2024-09-17T21:30:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728574+0000", 2024-09-17T21:30:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7180000000000003e-05 2024-09-17T21:30:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45707 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 212992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 39.053086630000003, 2024-09-17T21:30:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022235550999999999, 2024-09-17T21:30:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45707 2024-09-17T21:30:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:30:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707905+0000", 2024-09-17T21:30:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2120000000000002e-06 2024-09-17T21:30:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707911+0000", 2024-09-17T21:30:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1929999999999998e-06 2024-09-17T21:30:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707915+0000", 2024-09-17T21:30:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:30:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729897+0000", 2024-09-17T21:30:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021981186 2024-09-17T21:30:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729944+0000", 2024-09-17T21:30:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571000000000002e-05 2024-09-17T21:30:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730138+0000", 2024-09-17T21:30:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000193451 2024-09-17T21:30:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45708 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 39.053066498, 2024-09-17T21:30:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022533742999999998, 2024-09-17T21:30:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45708 2024-09-17T21:30:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707923+0000", 2024-09-17T21:30:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:30:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707932+0000", 2024-09-17T21:30:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6430000000000007e-06 2024-09-17T21:30:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707939+0000", 2024-09-17T21:30:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3539999999999996e-06 2024-09-17T21:30:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707944+0000", 2024-09-17T21:30:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3249999999999998e-06 2024-09-17T21:30:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730176+0000", 2024-09-17T21:30:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022232393 2024-09-17T21:30:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730263+0000", 2024-09-17T21:30:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6638999999999995e-05 2024-09-17T21:30:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730456+0000", 2024-09-17T21:30:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019304899999999999 2024-09-17T21:30:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45752 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 39.052811278, 2024-09-17T21:30:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022426878000000001, 2024-09-17T21:30:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45752 2024-09-17T21:30:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708178+0000", 2024-09-17T21:30:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:30:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708181+0000", 2024-09-17T21:30:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.909e-06 2024-09-17T21:30:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708187+0000", 2024-09-17T21:30:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4999999999999996e-06 2024-09-17T21:30:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708193+0000", 2024-09-17T21:30:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:30:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730484+0000", 2024-09-17T21:30:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022291765000000002 2024-09-17T21:30:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730541+0000", 2024-09-17T21:30:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6483000000000001e-05 2024-09-17T21:30:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730604+0000", 2024-09-17T21:30:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3429000000000005e-05 2024-09-17T21:30:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45761 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 39.052783368, 2024-09-17T21:30:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022614408999999999, 2024-09-17T21:30:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45761 2024-09-17T21:30:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708206+0000", 2024-09-17T21:30:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4700000000000001e-07 2024-09-17T21:30:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708208+0000", 2024-09-17T21:30:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1509999999999998e-06 2024-09-17T21:30:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708214+0000", 2024-09-17T21:30:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.49e-06 2024-09-17T21:30:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708219+0000", 2024-09-17T21:30:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8119999999999998e-06 2024-09-17T21:30:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730641+0000", 2024-09-17T21:30:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022422765000000001 2024-09-17T21:30:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730731+0000", 2024-09-17T21:30:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9598999999999994e-05 2024-09-17T21:30:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730820+0000", 2024-09-17T21:30:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8844999999999994e-05 2024-09-17T21:30:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45778 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 39.052039764, 2024-09-17T21:30:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022018544000000001, 2024-09-17T21:30:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45778 2024-09-17T21:30:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708949+0000", 2024-09-17T21:30:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708950+0000", 2024-09-17T21:30:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-17T21:30:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709307+0000", 2024-09-17T21:30:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000356848 2024-09-17T21:30:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709316+0000", 2024-09-17T21:30:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1430000000000004e-06 2024-09-17T21:30:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709328+0000", 2024-09-17T21:30:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2629999999999999e-05 2024-09-17T21:30:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730861+0000", 2024-09-17T21:30:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021532057 2024-09-17T21:30:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730905+0000", 2024-09-17T21:30:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4907000000000002e-05 2024-09-17T21:30:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730968+0000", 2024-09-17T21:30:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2108999999999998e-05 2024-09-17T21:30:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45785 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 39.051561581000001, 2024-09-17T21:30:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022044113000000001, 2024-09-17T21:30:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45785 2024-09-17T21:30:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709428+0000", 2024-09-17T21:30:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:30:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709430+0000", 2024-09-17T21:30:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:30:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709451+0000", 2024-09-17T21:30:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1372000000000001e-05 2024-09-17T21:30:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709472+0000", 2024-09-17T21:30:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1007999999999999e-05 2024-09-17T21:30:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731293+0000", 2024-09-17T21:30:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021821320000000002 2024-09-17T21:30:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731378+0000", 2024-09-17T21:30:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5059e-05 2024-09-17T21:30:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731471+0000", 2024-09-17T21:30:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2867000000000004e-05 2024-09-17T21:30:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45786 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 39.051502784, 2024-09-17T21:30:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022181821000000001, 2024-09-17T21:30:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45786 2024-09-17T21:30:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709487+0000", 2024-09-17T21:30:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1299999999999999e-07 2024-09-17T21:30:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709491+0000", 2024-09-17T21:30:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2089999999999999e-06 2024-09-17T21:30:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709498+0000", 2024-09-17T21:30:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3980000000000002e-06 2024-09-17T21:30:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709504+0000", 2024-09-17T21:30:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7219999999999996e-06 2024-09-17T21:30:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731499+0000", 2024-09-17T21:30:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021995297 2024-09-17T21:30:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731591+0000", 2024-09-17T21:30:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1373999999999996e-05 2024-09-17T21:30:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731668+0000", 2024-09-17T21:30:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7107999999999994e-05 2024-09-17T21:30:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50202 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'139, mlcod=45'137)", 2024-09-17T21:30:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:30:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 34.636766915000003, 2024-09-17T21:30:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059659441000000001, 2024-09-17T21:30:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:30:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:30:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124224+0000", 2024-09-17T21:30:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5000000000000002e-06 2024-09-17T21:30:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147385+0000", 2024-09-17T21:30:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023160544000000002 2024-09-17T21:30:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147411+0000", 2024-09-17T21:30:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6095999999999999e-05 2024-09-17T21:30:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147438+0000", 2024-09-17T21:30:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6982999999999999e-05 2024-09-17T21:30:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147504+0000", 2024-09-17T21:30:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6156999999999997e-05 2024-09-17T21:30:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147531+0000", 2024-09-17T21:30:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6741999999999999e-05 2024-09-17T21:30:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183820+0000", 2024-09-17T21:30:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036289090000000003 2024-09-17T21:30:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183881+0000", 2024-09-17T21:30:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1328999999999995e-05 2024-09-17T21:30:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50203 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'141, mlcod=45'137)", 2024-09-17T21:30:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:30:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 34.599504172000003, 2024-09-17T21:30:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023138823999999999, 2024-09-17T21:30:45.850 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.850 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.850 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:45.850 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:30:45.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:30:45.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161486+0000", 2024-09-17T21:30:45.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5290000000000001e-06 2024-09-17T21:30:45.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161496+0000", 2024-09-17T21:30:45.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0063e-05 2024-09-17T21:30:45.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161514+0000", 2024-09-17T21:30:45.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7728e-05 2024-09-17T21:30:45.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161524+0000", 2024-09-17T21:30:45.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0339e-05 2024-09-17T21:30:45.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170872+0000", 2024-09-17T21:30:45.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093473640000000004 2024-09-17T21:30:45.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170899+0000", 2024-09-17T21:30:45.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7321000000000002e-05 2024-09-17T21:30:45.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:45.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184580+0000", 2024-09-17T21:30:45.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013680988999999999 2024-09-17T21:30:45.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184623+0000", 2024-09-17T21:30:45.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3491000000000001e-05 2024-09-17T21:30:45.853 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.853 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.854 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.854 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:88193 3.9 3:97498da5:::rbd_data.1120ea662e44.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-17T21:30:45.854 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:30:45.854 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.4373232619999996, 2024-09-17T21:30:45.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026278261000000001, 2024-09-17T21:30:45.854 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:45.854 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:45.854 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:45.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:45.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:45.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:45.855 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 88193 2024-09-17T21:30:45.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.855 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:45.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:45.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:30:45.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:45.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:30:45.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:45.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:45.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323666+0000", 2024-09-17T21:30:45.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:30:45.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:45.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323670+0000", 2024-09-17T21:30:45.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.771e-06 2024-09-17T21:30:45.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:45.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323680+0000", 2024-09-17T21:30:45.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9469999999999998e-06 2024-09-17T21:30:45.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:45.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323685+0000", 2024-09-17T21:30:45.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9919999999999998e-06 2024-09-17T21:30:45.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:45.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323728+0000", 2024-09-17T21:30:45.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2811000000000001e-05 2024-09-17T21:30:45.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:45.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323776+0000", 2024-09-17T21:30:45.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8310000000000003e-05 2024-09-17T21:30:45.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:45.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.328583+0000", 2024-09-17T21:30:45.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048072899999999997 2024-09-17T21:30:45.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:45.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349861+0000", 2024-09-17T21:30:45.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021277695999999999 2024-09-17T21:30:45.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:45.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349877+0000", 2024-09-17T21:30:45.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:30:45.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:45.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349911+0000", 2024-09-17T21:30:45.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3844999999999999e-05 2024-09-17T21:30:45.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:45.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:45.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:45.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349944+0000", 2024-09-17T21:30:45.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2663000000000002e-05 2024-09-17T21:30:45.859 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.859 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.859 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.860 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:45.860 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:45.860 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:45.860 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:30:45.861 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:45.775+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:45.863 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:30:45.863 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:45.859+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:45.863 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T21:30:45.864 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T21:30:45.864 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:45.864 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:45.884 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:30:45.884 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:30:45.902 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:45.902 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:45.913 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone35 from 0x62817e2 (overlap 0xb8b061) to 0x5c5830 2024-09-17T21:30:45.922 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:45.922 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:45.942 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:45.942 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:45.960 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:45.960 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:45.962 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:45.960+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:45.977 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:45.977 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:45.996 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:45.996 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:46.015 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:46.016 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:46.016 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-17T21:30:46.023 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:46.023 INFO:teuthology.orchestra.run.smithi129.stdout:4883 read 0x5f9efbf thru 0x5fae87d (0xf8bf bytes) 2024-09-17T21:30:46.024 INFO:teuthology.orchestra.run.smithi129.stdout:4884 punch from 0x48df194 to 0x48e9487, (0xa2f3 bytes) 2024-09-17T21:30:46.025 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:30:46.038 INFO:teuthology.orchestra.run.smithi129.stdout:4885 writesame 0x926a56a thru 0x9278da6 data_size 0x1(0xe83d bytes) 2024-09-17T21:30:46.051 INFO:teuthology.orchestra.run.smithi129.stdout:4887 trunc from 0x9278da7 to 0x9ff2c2 2024-09-17T21:30:46.069 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:46.067+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:46.080 INFO:teuthology.orchestra.run.smithi129.stdout:4888 punch from 0x8d7274 to 0x8dac32, (0x39be bytes) 2024-09-17T21:30:46.095 INFO:teuthology.orchestra.run.smithi129.stdout:4890 write 0x664673d thru 0x6648a4f (0x2313 bytes) 2024-09-17T21:30:46.109 INFO:teuthology.orchestra.run.smithi129.stdout:4891 write 0xa16f6f8 thru 0xa178888 (0x9191 bytes) 2024-09-17T21:30:46.112 INFO:teuthology.orchestra.run.smithi129.stdout:4892 write 0x79b6905 thru 0x79baa96 (0x4192 bytes) 2024-09-17T21:30:46.116 INFO:teuthology.orchestra.run.smithi129.stdout:4893 read 0x31747be thru 0x317def3 (0x9736 bytes) 2024-09-17T21:30:46.116 INFO:teuthology.orchestra.run.smithi129.stdout:4894 read 0x89f2454 thru 0x89ffbd4 (0xd781 bytes) 2024-09-17T21:30:46.116 INFO:teuthology.orchestra.run.smithi129.stdout:4895 read 0x71a65f5 thru 0x71a89dc (0x23e8 bytes) 2024-09-17T21:30:46.116 INFO:teuthology.orchestra.run.smithi129.stdout:4896 flatten 2024-09-17T21:30:46.134 INFO:teuthology.orchestra.run.smithi129.stdout:4897 trunc from 0xa178889 to 0x1e578fd 2024-09-17T21:30:46.139 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:46.139 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:46.149 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:30:46.161 INFO:teuthology.orchestra.run.smithi129.stdout:4898 writesame 0xac04052 thru 0xac0c703 data_size 0x15(0x86b2 bytes) 2024-09-17T21:30:46.161 INFO:teuthology.orchestra.run.smithi129.stdout:4899 write 0xaa08616 thru 0xaa0d6f0 (0x50db bytes) 2024-09-17T21:30:46.165 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:46.176 INFO:teuthology.orchestra.run.smithi129.stdout:4901 trunc from 0xac0c704 to 0x1f1d3e4 2024-09-17T21:30:46.190 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:46.188+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:46.209 INFO:teuthology.orchestra.run.smithi129.stdout:4902 write 0xe2c9429 thru 0xe2d6729 (0xd301 bytes) 2024-09-17T21:30:46.213 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:46.214 INFO:teuthology.orchestra.run.smithi129.stdout:4904 write 0x6fc966b thru 0x6fcff6a (0x6900 bytes) 2024-09-17T21:30:46.223 INFO:teuthology.orchestra.run.smithi129.stdout:4905 write 0xe7cf451 thru 0xe7d567d (0x622d bytes) 2024-09-17T21:30:46.228 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:46.228 INFO:teuthology.orchestra.run.smithi129.stdout:4908 read 0xa9bcb8a thru 0xa9bdf75 (0x13ec bytes) 2024-09-17T21:30:46.234 INFO:teuthology.orchestra.run.smithi129.stdout:4910 trunc from 0xe7d567e to 0x932bfc1 2024-09-17T21:30:46.247 INFO:teuthology.orchestra.run.smithi129.stdout:4911 write 0x81c8fb3 thru 0x81d207b (0x90c9 bytes) 2024-09-17T21:30:46.251 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:46.251 INFO:teuthology.orchestra.run.smithi129.stdout:4914 punch from 0xfd6f52 to 0xfd8d65, (0x1e13 bytes) 2024-09-17T21:30:46.251 INFO:teuthology.orchestra.run.smithi129.stdout:4915 writesame 0x2f0100d thru 0x2f09413 data_size 0x1(0x8407 bytes) 2024-09-17T21:30:46.265 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:46.263+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:46.269 INFO:teuthology.orchestra.run.smithi129.stdout:4916 read 0x63a4a55 thru 0x63b2866 (0xde12 bytes) 2024-09-17T21:30:46.269 INFO:teuthology.orchestra.run.smithi129.stdout:4917 read 0x480acff thru 0x4810342 (0x5644 bytes) 2024-09-17T21:30:46.273 INFO:teuthology.orchestra.run.smithi129.stdout:4919 trunc from 0x932bfc1 to 0xc709969 2024-09-17T21:30:46.281 INFO:teuthology.orchestra.run.smithi129.stdout:4920 trunc from 0xc709969 to 0xa9d2171 2024-09-17T21:30:46.288 INFO:teuthology.orchestra.run.smithi129.stdout:4921 write 0x1f388e4 thru 0x1f457a7 (0xcec4 bytes) 2024-09-17T21:30:46.301 INFO:teuthology.orchestra.run.smithi129.stdout:4924 write 0xd6af59c thru 0xd6be18b (0xebf0 bytes) 2024-09-17T21:30:46.301 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:46.302 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:46.305 INFO:teuthology.orchestra.run.smithi129.stdout:4926 writesame 0xd28fc88 thru 0xd29c570 data_size 0x1(0xc8e9 bytes) 2024-09-17T21:30:46.311 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:30:46.331 INFO:teuthology.orchestra.run.smithi129.stdout:4927 write 0xa046e64 thru 0xa054061 (0xd1fe bytes) 2024-09-17T21:30:46.335 INFO:teuthology.orchestra.run.smithi129.stdout:4928 write 0xbe693ba thru 0xbe6b7d0 (0x2417 bytes) 2024-09-17T21:30:46.338 INFO:teuthology.orchestra.run.smithi129.stdout:4929 punch from 0x7c9dbfb to 0x7ca70ba, (0x94bf bytes) 2024-09-17T21:30:46.338 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:46.339 INFO:teuthology.orchestra.run.smithi129.stdout:4931 read 0x8721fb1 thru 0x8723155 (0x11a5 bytes) 2024-09-17T21:30:46.339 INFO:teuthology.orchestra.run.smithi129.stdout:4933 write 0xb1680b3 thru 0xb177caa (0xfbf8 bytes) 2024-09-17T21:30:46.342 INFO:teuthology.orchestra.run.smithi129.stdout:4935 write 0x99e52f5 thru 0x99f11e2 (0xbeee bytes) 2024-09-17T21:30:46.351 INFO:teuthology.orchestra.run.smithi129.stdout:4937 write 0xdcdb3dd thru 0xdce430c (0x8f30 bytes) 2024-09-17T21:30:46.354 INFO:teuthology.orchestra.run.smithi129.stdout:4938 read 0x4c58f2b thru 0x4c687a3 (0xf879 bytes) 2024-09-17T21:30:46.355 INFO:teuthology.orchestra.run.smithi129.stdout:4939 punch from 0x24d72be to 0x24e1068, (0x9daa bytes) 2024-09-17T21:30:46.355 INFO:teuthology.orchestra.run.smithi129.stdout:4941 punch from 0xd0654ab to 0xd06f9e9, (0xa53e bytes) 2024-09-17T21:30:46.355 INFO:teuthology.orchestra.run.smithi129.stdout:4942 write 0xa651ed7 thru 0xa653a77 (0x1ba1 bytes) 2024-09-17T21:30:46.357 INFO:teuthology.orchestra.run.smithi129.stdout:4945 write 0xb7fe30b thru 0xb802345 (0x403b bytes) 2024-09-17T21:30:46.362 INFO:teuthology.orchestra.run.smithi129.stdout:4947 read 0xc2e9aeb thru 0xc2f48f3 (0xae09 bytes) 2024-09-17T21:30:46.362 INFO:teuthology.orchestra.run.smithi129.stdout:4949 punch from 0x2081cce to 0x208ea3f, (0xcd71 bytes) 2024-09-17T21:30:46.362 INFO:teuthology.orchestra.run.smithi129.stdout:4951 write 0x5267b5f thru 0x5275f0b (0xe3ad bytes) 2024-09-17T21:30:46.366 INFO:teuthology.orchestra.run.smithi129.stdout:4953 read 0xa15a333 thru 0xa15d466 (0x3134 bytes) 2024-09-17T21:30:46.366 INFO:teuthology.orchestra.run.smithi129.stdout:4956 write 0x461429d thru 0x461b2e2 (0x7046 bytes) 2024-09-17T21:30:46.366 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:46.363+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:46.369 INFO:teuthology.orchestra.run.smithi129.stdout:4957 punch from 0x858c545 to 0x858fdb2, (0x386d bytes) 2024-09-17T21:30:46.369 INFO:teuthology.orchestra.run.smithi129.stdout:4958 writesame 0x9a07299 thru 0x9a1347f data_size 0x1(0xc1e7 bytes) 2024-09-17T21:30:46.387 INFO:teuthology.orchestra.run.smithi129.stdout:4959 write 0x2e2ec4b thru 0x2e32b42 (0x3ef8 bytes) 2024-09-17T21:30:46.403 INFO:teuthology.orchestra.run.smithi129.stdout:4962 trunc from 0xdce430d to 0x25a261b 2024-09-17T21:30:46.423 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:46.423 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:46.425 INFO:teuthology.orchestra.run.smithi129.stdout:4964 writesame 0x51aa2d9 thru 0x51ad926 data_size 0x15(0x364e bytes) 2024-09-17T21:30:46.425 INFO:teuthology.orchestra.run.smithi129.stdout:4965 writesame 0x8ae00b thru 0x8bab16 data_size 0x73(0xcb0c bytes) 2024-09-17T21:30:46.433 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:30:46.436 INFO:teuthology.orchestra.run.smithi129.stdout:4967 writesame 0x740b03d thru 0x740c72b data_size 0x13(0x16ef bytes) 2024-09-17T21:30:46.436 INFO:teuthology.orchestra.run.smithi129.stdout:4968 read 0x3f722a0 thru 0x3f7344c (0x11ad bytes) 2024-09-17T21:30:46.437 INFO:teuthology.orchestra.run.smithi129.stdout:4970 trunc from 0x740c72c to 0x6481653 2024-09-17T21:30:46.444 INFO:teuthology.orchestra.run.smithi129.stdout:4972 read 0x43df98b thru 0x43ebfe5 (0xc65b bytes) 2024-09-17T21:30:46.444 INFO:teuthology.orchestra.run.smithi129.stdout:4973 punch from 0x188b0ea to 0x1898bc3, (0xdad9 bytes) 2024-09-17T21:30:46.444 INFO:teuthology.orchestra.run.smithi129.stdout:4974 writesame 0x456c406 thru 0x456f102 data_size 0xb(0x2cfd bytes) 2024-09-17T21:30:46.448 INFO:teuthology.orchestra.run.smithi129.stdout:4975 punch from 0x12c2447 to 0x12cc65a, (0xa213 bytes) 2024-09-17T21:30:46.448 INFO:teuthology.orchestra.run.smithi129.stdout:4978 punch from 0x6132213 to 0x61408e0, (0xe6cd bytes) 2024-09-17T21:30:46.463 INFO:teuthology.orchestra.run.smithi129.stdout:4979 write 0xcc1b8b5 thru 0xcc22ad5 (0x7221 bytes) 2024-09-17T21:30:46.467 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:46.464+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:46.474 INFO:teuthology.orchestra.run.smithi129.stdout:4980 trunc from 0xcc22ad6 to 0x6e142bc 2024-09-17T21:30:46.483 INFO:teuthology.orchestra.run.smithi129.stdout:4981 write 0x18381da thru 0x1839213 (0x103a bytes) 2024-09-17T21:30:46.487 INFO:teuthology.orchestra.run.smithi129.stdout:4982 read 0x6bdeebb thru 0x6be79a7 (0x8aed bytes) 2024-09-17T21:30:46.487 INFO:teuthology.orchestra.run.smithi129.stdout:4983 trunc from 0x6e142bc to 0x6b98fe0 2024-09-17T21:30:46.503 INFO:teuthology.orchestra.run.smithi129.stdout:4984 write 0xa5f91cc thru 0xa606d4e (0xdb83 bytes) 2024-09-17T21:30:46.510 INFO:teuthology.orchestra.run.smithi129.stdout:4986 punch from 0x31a9139 to 0x31af0a2, (0x5f69 bytes) 2024-09-17T21:30:46.510 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:46.510 INFO:teuthology.orchestra.run.smithi129.stdout:4989 read 0x34802c1 thru 0x348f52a (0xf26a bytes) 2024-09-17T21:30:46.510 INFO:teuthology.orchestra.run.smithi129.stdout:4990 read 0x5b71e69 thru 0x5b76014 (0x41ac bytes) 2024-09-17T21:30:46.511 INFO:teuthology.orchestra.run.smithi129.stdout:4991 punch from 0x6ba1118 to 0x6bae368, (0xd250 bytes) 2024-09-17T21:30:46.511 INFO:teuthology.orchestra.run.smithi129.stdout:4992 read 0x2fb2760 thru 0x2fb4b92 (0x2433 bytes) 2024-09-17T21:30:46.516 INFO:teuthology.orchestra.run.smithi129.stdout:4994 trunc from 0xa606d4f to 0x4b57636 2024-09-17T21:30:46.530 INFO:teuthology.orchestra.run.smithi129.stdout:4995 read 0x1eb4dbe thru 0x1ebe3e9 (0x962c bytes) 2024-09-17T21:30:46.530 INFO:teuthology.orchestra.run.smithi129.stdout:4996 punch from 0x42e6dbc to 0x42f5a3c, (0xec80 bytes) 2024-09-17T21:30:46.530 INFO:teuthology.orchestra.run.smithi129.stdout:4997 writesame 0x22681b6 thru 0x22726db data_size 0x1(0xa526 bytes) 2024-09-17T21:30:46.543 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:46.543 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:46.551 INFO:teuthology.orchestra.run.smithi129.stdout:4998 read 0x2e85962 thru 0x2e8a814 (0x4eb3 bytes) 2024-09-17T21:30:46.551 INFO:teuthology.orchestra.run.smithi129.stdout:5000 punch from 0xba4bc9 to 0xba6d3f, (0x2176 bytes) 2024-09-17T21:30:46.552 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:30:46.561 INFO:teuthology.orchestra.run.smithi129.stdout:5001 trunc from 0x4b57636 to 0xd804c4f 2024-09-17T21:30:46.567 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:46.565+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:46.580 INFO:teuthology.orchestra.run.smithi129.stdout:5002 trunc from 0xd804c4f to 0x32b88b5 2024-09-17T21:30:46.593 INFO:teuthology.orchestra.run.smithi129.stdout:5004 punch from 0x67da17 to 0x683896, (0x5e7f bytes) 2024-09-17T21:30:46.611 INFO:teuthology.orchestra.run.smithi129.stdout:5005 writesame 0xbb072e1 thru 0xbb0b269 data_size 0x1(0x3f89 bytes) 2024-09-17T21:30:46.611 INFO:teuthology.orchestra.run.smithi129.stdout:5006 read 0x680e205 thru 0x6812a21 (0x481d bytes) 2024-09-17T21:30:46.618 INFO:teuthology.orchestra.run.smithi129.stdout:5007 trunc from 0xbb0b26a to 0x552a66b 2024-09-17T21:30:46.625 INFO:teuthology.orchestra.run.smithi129.stdout:5008 read 0x409d380 thru 0x40a8334 (0xafb5 bytes) 2024-09-17T21:30:46.625 INFO:teuthology.orchestra.run.smithi129.stdout:5009 read 0x31c26c8 thru 0x31cbbac (0x94e5 bytes) 2024-09-17T21:30:46.625 INFO:teuthology.orchestra.run.smithi129.stdout:5010 read 0x36f2877 thru 0x36fb515 (0x8c9f bytes) 2024-09-17T21:30:46.625 INFO:teuthology.orchestra.run.smithi129.stdout:5011 clone 37 order 18 su 16384 sc 9 2024-09-17T21:30:46.658 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:46.658 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:46.667 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:30:46.668 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:46.666+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:46.770 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:46.767+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:46.770 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:46.771 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:46.779 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:30:46.845 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is unset 2024-09-17T21:30:46.845 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T21:30:46.862 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T21:30:46.870 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:46.868+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:46.887 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:46.887 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:46.896 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:30:46.980 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:46.978+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:47.002 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-17T21:30:47.014 INFO:teuthology.orchestra.run.smithi129.stdout:5012 read 0x1ae8289 thru 0x1aec858 (0x45d0 bytes) 2024-09-17T21:30:47.018 INFO:teuthology.orchestra.run.smithi129.stdout:5013 trunc from 0x552a66b to 0x3147b81 2024-09-17T21:30:47.047 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:47.047 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:47.047 INFO:teuthology.orchestra.run.smithi129.stdout:5014 write 0x5df33de thru 0x5dff7a7 (0xc3ca bytes) 2024-09-17T21:30:47.057 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:30:47.060 INFO:teuthology.orchestra.run.smithi129.stdout:5015 writesame 0x79ae893 thru 0x79af5c3 data_size 0x1(0xd31 bytes) 2024-09-17T21:30:47.060 INFO:teuthology.orchestra.run.smithi129.stdout:5016 read 0x168ee98 thru 0x1695feb (0x7154 bytes) 2024-09-17T21:30:47.060 INFO:teuthology.orchestra.run.smithi129.stdout:5018 read 0x72a5786 thru 0x72ae82d (0x90a8 bytes) 2024-09-17T21:30:47.060 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:47.060 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:47.061 INFO:teuthology.orchestra.run.smithi129.stdout:5022 write 0x5c25228 thru 0x5c31764 (0xc53d bytes) 2024-09-17T21:30:47.066 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:47.066 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:47.066 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:47.067 INFO:teuthology.orchestra.run.smithi129.stdout:5029 read 0x12d2387 thru 0x12db800 (0x947a bytes) 2024-09-17T21:30:47.067 INFO:teuthology.orchestra.run.smithi129.stdout:5030 read 0x557f2d7 thru 0x558d37a (0xe0a4 bytes) 2024-09-17T21:30:47.067 INFO:teuthology.orchestra.run.smithi129.stdout:5031 punch from 0x46c27e to 0x472aa6, (0x6828 bytes) 2024-09-17T21:30:47.075 INFO:teuthology.orchestra.run.smithi129.stdout:5033 writesame 0xa7a8428 thru 0xa7ab7eb data_size 0x2(0x33c4 bytes) 2024-09-17T21:30:47.075 INFO:teuthology.orchestra.run.smithi129.stdout:5036 read 0x666db9e thru 0x667680a (0x8c6d bytes) 2024-09-17T21:30:47.075 INFO:teuthology.orchestra.run.smithi129.stdout:5037 write 0x19d97e0 thru 0x19dd638 (0x3e59 bytes) 2024-09-17T21:30:47.080 INFO:teuthology.orchestra.run.smithi129.stdout:5038 read 0x27bd2b6 thru 0x27c19aa (0x46f5 bytes) 2024-09-17T21:30:47.080 INFO:teuthology.orchestra.run.smithi129.stdout:5039 write 0x309ad9b thru 0x309c452 (0x16b8 bytes) 2024-09-17T21:30:47.082 INFO:teuthology.orchestra.run.smithi129.stdout:5041 punch from 0x2f6cbbc to 0x2f75338, (0x877c bytes) 2024-09-17T21:30:47.082 INFO:teuthology.orchestra.run.smithi129.stdout:5042 read 0x33f49f3 thru 0x33f559d (0xbab bytes) 2024-09-17T21:30:47.082 INFO:teuthology.orchestra.run.smithi129.stdout:5043 read 0x67c5acf thru 0x67cc8c7 (0x6df9 bytes) 2024-09-17T21:30:47.087 INFO:teuthology.orchestra.run.smithi129.stdout:5044 write 0xa88ba3f thru 0xa898cdd (0xd29f bytes) 2024-09-17T21:30:47.097 INFO:teuthology.orchestra.run.smithi129.stdout:5045 write 0x19e8ae4 thru 0x19f1265 (0x8782 bytes) 2024-09-17T21:30:47.099 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:47.097+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:47.108 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:47.108 INFO:teuthology.orchestra.run.smithi129.stdout:5047 read 0x1310f0 thru 0x139bb0 (0x8ac1 bytes) 2024-09-17T21:30:47.121 INFO:teuthology.orchestra.run.smithi129.stdout:5048 trunc from 0xa898cde to 0x12ae7de 2024-09-17T21:30:47.156 INFO:teuthology.orchestra.run.smithi129.stdout:5049 read 0xdf7d38 thru 0xdff31e (0x75e7 bytes) 2024-09-17T21:30:47.156 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:47.160 INFO:teuthology.orchestra.run.smithi129.stdout:5051 trunc from 0x12ae7de to 0x47ffd4e 2024-09-17T21:30:47.172 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:47.170+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:47.181 INFO:teuthology.orchestra.run.smithi129.stdout:5052 write 0xd27d225 thru 0xd28a3e0 (0xd1bc bytes) 2024-09-17T21:30:47.188 INFO:teuthology.orchestra.run.smithi129.stdout:5053 read 0x640a464 thru 0x64125bd (0x815a bytes) 2024-09-17T21:30:47.188 INFO:teuthology.orchestra.run.smithi129.stdout:5054 trunc from 0xd28a3e1 to 0xcf5f4bc 2024-09-17T21:30:47.189 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:47.189 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:47.199 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:30:47.201 INFO:teuthology.orchestra.run.smithi129.stdout:5056 read 0x73b0049 thru 0x73bb52a (0xb4e2 bytes) 2024-09-17T21:30:47.201 INFO:teuthology.orchestra.run.smithi129.stdout:5057 read 0xbf4d80a thru 0xbf51163 (0x395a bytes) 2024-09-17T21:30:47.201 INFO:teuthology.orchestra.run.smithi129.stdout:5058 punch from 0xcad5833 to 0xcad6a9c, (0x1269 bytes) 2024-09-17T21:30:47.201 INFO:teuthology.orchestra.run.smithi129.stdout:5060 writesame 0xacab4b0 thru 0xacad39e data_size 0x1(0x1eef bytes) 2024-09-17T21:30:47.206 INFO:teuthology.orchestra.run.smithi129.stdout:5062 read 0x1dd9188 thru 0x1de4a4f (0xb8c8 bytes) 2024-09-17T21:30:47.206 INFO:teuthology.orchestra.run.smithi129.stdout:5063 read 0x1ecb57f thru 0x1eda6e8 (0xf16a bytes) 2024-09-17T21:30:47.206 INFO:teuthology.orchestra.run.smithi129.stdout:5064 read 0x3f2827e thru 0x3f37e73 (0xfbf6 bytes) 2024-09-17T21:30:47.213 INFO:teuthology.orchestra.run.smithi129.stdout:5065 write 0xe20dd56 thru 0xe21798e (0x9c39 bytes) 2024-09-17T21:30:47.219 INFO:teuthology.orchestra.run.smithi129.stdout:5066 read 0x2e80ba6 thru 0x2e8981d (0x8c78 bytes) 2024-09-17T21:30:47.219 INFO:teuthology.orchestra.run.smithi129.stdout:5070 read 0xaf78aa2 thru 0xaf8529a (0xc7f9 bytes) 2024-09-17T21:30:47.219 INFO:teuthology.orchestra.run.smithi129.stdout:5073 write 0x4c0eae1 thru 0x4c0ebdc (0xfc bytes) 2024-09-17T21:30:47.222 INFO:teuthology.orchestra.run.smithi129.stdout:5074 read 0xdbe7b66 thru 0xdbec502 (0x499d bytes) 2024-09-17T21:30:47.227 INFO:teuthology.orchestra.run.smithi129.stdout:5075 write 0xe6cad59 thru 0xe6d0f54 (0x61fc bytes) 2024-09-17T21:30:47.233 INFO:teuthology.orchestra.run.smithi129.stdout:5076 read 0xab51e59 thru 0xab60298 (0xe440 bytes) 2024-09-17T21:30:47.233 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:47.233 INFO:teuthology.orchestra.run.smithi129.stdout:5078 punch from 0x9604d15 to 0x960b97e, (0x6c69 bytes) 2024-09-17T21:30:47.233 INFO:teuthology.orchestra.run.smithi129.stdout:5082 trunc from 0xe6d0f55 to 0xeb187a0 2024-09-17T21:30:47.249 INFO:teuthology.orchestra.run.smithi129.stdout:5083 write 0xedcc04d thru 0xedd936c (0xd320 bytes) 2024-09-17T21:30:47.256 INFO:teuthology.orchestra.run.smithi129.stdout:5084 writesame 0x9a39d87 thru 0x9a432d8 data_size 0x45(0x9552 bytes) 2024-09-17T21:30:47.271 INFO:teuthology.orchestra.run.smithi129.stdout:5086 trunc from 0xedd936d to 0x6cc6397 2024-09-17T21:30:47.273 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:47.271+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:47.285 INFO:teuthology.orchestra.run.smithi129.stdout:5088 punch from 0x26a93da to 0x26a9e26, (0xa4c bytes) 2024-09-17T21:30:47.286 INFO:teuthology.orchestra.run.smithi129.stdout:5089 read 0x2e21511 thru 0x2e2d1d2 (0xbcc2 bytes) 2024-09-17T21:30:47.286 INFO:teuthology.orchestra.run.smithi129.stdout:5090 write 0x36ce68a thru 0x36de65f (0xffd6 bytes) 2024-09-17T21:30:47.294 INFO:teuthology.orchestra.run.smithi129.stdout:5092 punch from 0x1b22a09 to 0x1b2d2dc, (0xa8d3 bytes) 2024-09-17T21:30:47.294 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:47.294 INFO:teuthology.orchestra.run.smithi129.stdout:5094 read 0x4151645 thru 0x415581c (0x41d8 bytes) 2024-09-17T21:30:47.294 INFO:teuthology.orchestra.run.smithi129.stdout:5095 read 0x33a641 thru 0x343b26 (0x94e6 bytes) 2024-09-17T21:30:47.301 INFO:teuthology.orchestra.run.smithi129.stdout:5096 writesame 0x7fb7476 thru 0x7fc15b8 data_size 0x21(0xa143 bytes) 2024-09-17T21:30:47.302 INFO:teuthology.orchestra.run.smithi129.stdout:5098 writesame 0x6fcfceb thru 0x6fd8e4c data_size 0x3(0x9162 bytes) 2024-09-17T21:30:47.324 INFO:teuthology.orchestra.run.smithi129.stdout:5102 writesame 0xdd5ba9e thru 0xdd65e31 data_size 0x4c(0xa394 bytes) 2024-09-17T21:30:47.324 INFO:teuthology.orchestra.run.smithi129.stdout:5103 writesame 0xae4207d thru 0xae43bd0 data_size 0x21(0x1b54 bytes) 2024-09-17T21:30:47.325 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:47.325 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:47.328 INFO:teuthology.orchestra.run.smithi129.stdout:5105 write 0x347c6b1 thru 0x3488ac9 (0xc419 bytes) 2024-09-17T21:30:47.335 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:30:47.339 INFO:teuthology.orchestra.run.smithi129.stdout:5106 clone 38 order 18 su 65536 sc 3 2024-09-17T21:30:47.374 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:47.372+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:47.442 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:47.442 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:47.450 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:30:47.493 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:47.491+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:47.552 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:47.552 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:47.561 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:30:47.576 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:47.573+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:47.676 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:47.674+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:47.705 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:47.705 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:47.714 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:30:47.777 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:47.774+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:47.817 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:47.817 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:47.826 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:30:47.858 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T21:30:47.874 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-17T21:30:47.877 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:47.875+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:47.922 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone37 from 0xdd65e32 (overlap 0x12ae7de) to 0x0 2024-09-17T21:30:47.978 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:47.976+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:47.999 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:47.999 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:48.012 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:30:48.015 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:30:48.079 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:48.077+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:48.129 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-17T21:30:48.176 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:48.176 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:48.180 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:48.180 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:48.180 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:48.178+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:48.190 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:30:48.193 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:30:48.194 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:30:48.212 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:48.212 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:48.228 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:48.229 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:48.233 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:48.233 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:48.233 INFO:teuthology.orchestra.run.smithi129.stdout:5113 punch from 0xb375f69 to 0xb38307d, (0xd114 bytes) 2024-09-17T21:30:48.237 INFO:teuthology.orchestra.run.smithi129.stdout:5114 read 0x840c5c8 thru 0x8412b5c (0x6595 bytes) 2024-09-17T21:30:48.243 INFO:teuthology.orchestra.run.smithi129.stdout:5115 trunc from 0xdd65e32 to 0x9a159bd 2024-09-17T21:30:48.247 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:48.248 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:48.255 INFO:teuthology.orchestra.run.smithi129.stdout:5116 write 0x703b4b1 thru 0x704357a (0x80ca bytes) 2024-09-17T21:30:48.263 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:48.267 INFO:teuthology.orchestra.run.smithi129.stdout:5120 trunc from 0x9a159bd to 0xca1fde4 2024-09-17T21:30:48.273 INFO:teuthology.orchestra.run.smithi129.stdout:5122 write 0x9021dc thru 0x911367 (0xf18c bytes) 2024-09-17T21:30:48.281 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:48.279+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:48.282 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:48.282 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:48.282 INFO:teuthology.orchestra.run.smithi129.stdout:5123 writesame 0x766ecfe thru 0x7673b6e data_size 0x1(0x4e71 bytes) 2024-09-17T21:30:48.293 INFO:teuthology.orchestra.run.smithi129.stdout:5125 read 0x6ee4f2b thru 0x6ee62ca (0x13a0 bytes) 2024-09-17T21:30:48.293 INFO:teuthology.orchestra.run.smithi129.stdout:5127 punch from 0x3bda45e to 0x3bdb54e, (0x10f0 bytes) 2024-09-17T21:30:48.293 INFO:teuthology.orchestra.run.smithi129.stdout:5128 writesame 0x984fb7c thru 0x985ec1b data_size 0x28(0xf0a0 bytes) 2024-09-17T21:30:48.298 INFO:teuthology.orchestra.run.smithi129.stdout:5130 write 0x4b62f1c thru 0x4b69840 (0x6925 bytes) 2024-09-17T21:30:48.303 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:48.303 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:48.311 INFO:teuthology.orchestra.run.smithi129.stdout:5132 writesame 0xee3dde6 thru 0xee4c717 data_size 0x13(0xe932 bytes) 2024-09-17T21:30:48.311 INFO:teuthology.orchestra.run.smithi129.stdout:5133 write 0xd8c7e22 thru 0xd8ccbb5 (0x4d94 bytes) 2024-09-17T21:30:48.315 INFO:teuthology.orchestra.run.smithi129.stdout:5135 writesame 0x8d1ef57 thru 0x8d2083e data_size 0x4(0x18e8 bytes) 2024-09-17T21:30:48.320 INFO:teuthology.orchestra.run.smithi129.stdout:5136 punch from 0xcf8afdb to 0xcf8b095, (0xba bytes) 2024-09-17T21:30:48.320 INFO:teuthology.orchestra.run.smithi129.stdout:5137 read 0xc1ffd66 thru 0xc20e21a (0xe4b5 bytes) 2024-09-17T21:30:48.320 INFO:teuthology.orchestra.run.smithi129.stdout:5139 write 0xd3e6ccb thru 0xd3f1a6f (0xada5 bytes) 2024-09-17T21:30:48.321 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:48.321 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:48.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:48.322 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:48.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.322 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "pg_update_log_missing(3.5 epoch 45/37 rep_tid 844 entries 45'73 (0'0) error 3:a65c40ae:::rbd_data.1120b21c3309.0000000000000126:head by client.4384.0:48900 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 45'70)", 2024-09-17T21:30:48.322 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:48.322 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.271706684000002, 2024-09-17T21:30:48.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019554365000000001, 2024-09-17T21:30:48.322 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.322 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.322 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T21:30:48.322 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:48.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:48.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047112+0000", 2024-09-17T21:30:48.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:30:48.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047121+0000", 2024-09-17T21:30:48.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4379999999999995e-06 2024-09-17T21:30:48.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047139+0000", 2024-09-17T21:30:48.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8235000000000001e-05 2024-09-17T21:30:48.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047303+0000", 2024-09-17T21:30:48.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016389400000000001 2024-09-17T21:30:48.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047396+0000", 2024-09-17T21:30:48.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3355000000000003e-05 2024-09-17T21:30:48.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066666+0000", 2024-09-17T21:30:48.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019269649999999999 2024-09-17T21:30:48.325 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.325 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.325 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.326 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48906 3.4 e45/25 3:2503fe27:::rbd_data.1120b21c3309.0000000000000127:head v 45'148, mlcod=45'146)", 2024-09-17T21:30:48.326 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:48.326 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.269580505, 2024-09-17T21:30:48.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018169739000000001, 2024-09-17T21:30:48.326 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.326 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.326 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:48.326 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:48.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:48.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:48.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:30:48.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049244+0000", 2024-09-17T21:30:48.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8390000000000003e-06 2024-09-17T21:30:48.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066411+0000", 2024-09-17T21:30:48.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017166818 2024-09-17T21:30:48.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066426+0000", 2024-09-17T21:30:48.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4888999999999999e-05 2024-09-17T21:30:48.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066611+0000", 2024-09-17T21:30:48.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018464499999999999 2024-09-17T21:30:48.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066630+0000", 2024-09-17T21:30:48.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9201000000000001e-05 2024-09-17T21:30:48.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:48.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067373+0000", 2024-09-17T21:30:48.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00074322600000000004 2024-09-17T21:30:48.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067407+0000", 2024-09-17T21:30:48.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4357999999999999e-05 2024-09-17T21:30:48.330 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.330 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.330 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.330 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50206 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'301, mlcod=45'299)", 2024-09-17T21:30:48.330 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:48.330 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.140103660999998, 2024-09-17T21:30:48.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068285810000000002, 2024-09-17T21:30:48.331 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.331 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.331 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:48.331 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:48.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:48.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178716+0000", 2024-09-17T21:30:48.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.438e-06 2024-09-17T21:30:48.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192044+0000", 2024-09-17T21:30:48.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013328370000000001 2024-09-17T21:30:48.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192069+0000", 2024-09-17T21:30:48.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4496000000000001e-05 2024-09-17T21:30:48.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192085+0000", 2024-09-17T21:30:48.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5758999999999999e-05 2024-09-17T21:30:48.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192167+0000", 2024-09-17T21:30:48.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2842000000000004e-05 2024-09-17T21:30:48.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192201+0000", 2024-09-17T21:30:48.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3127999999999999e-05 2024-09-17T21:30:48.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:48.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246938+0000", 2024-09-17T21:30:48.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054736982000000003 2024-09-17T21:30:48.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247000+0000", 2024-09-17T21:30:48.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2794999999999998e-05 2024-09-17T21:30:48.335 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.335 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.335 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.336 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50207 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'303, mlcod=45'299)", 2024-09-17T21:30:48.336 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:48.336 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.097156611999999, 2024-09-17T21:30:48.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026003433999999999, 2024-09-17T21:30:48.336 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.336 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.336 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:48.336 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:48.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:48.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221663+0000", 2024-09-17T21:30:48.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.271e-06 2024-09-17T21:30:48.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221672+0000", 2024-09-17T21:30:48.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6810000000000004e-06 2024-09-17T21:30:48.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221684+0000", 2024-09-17T21:30:48.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:30:48.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221691+0000", 2024-09-17T21:30:48.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:30:48.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221743+0000", 2024-09-17T21:30:48.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2868999999999998e-05 2024-09-17T21:30:48.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221765+0000", 2024-09-17T21:30:48.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1316999999999999e-05 2024-09-17T21:30:48.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:48.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247616+0000", 2024-09-17T21:30:48.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025851504000000001 2024-09-17T21:30:48.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247665+0000", 2024-09-17T21:30:48.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8739000000000002e-05 2024-09-17T21:30:48.340 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.340 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.340 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.341 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57808 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 196608~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:48.341 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:48.341 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.540857572, 2024-09-17T21:30:48.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017751475999999999, 2024-09-17T21:30:48.341 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.341 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.341 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.341 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.341 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.341 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.341 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57808 2024-09-17T21:30:48.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.342 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:48.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:48.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:48.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:30:48.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777968+0000", 2024-09-17T21:30:48.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8530000000000003e-06 2024-09-17T21:30:48.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777975+0000", 2024-09-17T21:30:48.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4410000000000004e-06 2024-09-17T21:30:48.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777980+0000", 2024-09-17T21:30:48.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7210000000000002e-06 2024-09-17T21:30:48.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795323+0000", 2024-09-17T21:30:48.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017343319999999999 2024-09-17T21:30:48.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795410+0000", 2024-09-17T21:30:48.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6567e-05 2024-09-17T21:30:48.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795712+0000", 2024-09-17T21:30:48.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030219200000000001 2024-09-17T21:30:48.345 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.345 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.345 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.345 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57809 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 458752~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:48.345 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:30:48.345 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.54083172, 2024-09-17T21:30:48.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018092869000000001, 2024-09-17T21:30:48.345 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.346 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.346 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.346 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57809 2024-09-17T21:30:48.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.346 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:30:48.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:30:48.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777987+0000", 2024-09-17T21:30:48.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T21:30:48.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777989+0000", 2024-09-17T21:30:48.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8649999999999999e-06 2024-09-17T21:30:48.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777994+0000", 2024-09-17T21:30:48.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4199999999999998e-06 2024-09-17T21:30:48.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778004+0000", 2024-09-17T21:30:48.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6900000000000004e-06 2024-09-17T21:30:48.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795745+0000", 2024-09-17T21:30:48.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017740859000000001 2024-09-17T21:30:48.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795815+0000", 2024-09-17T21:30:48.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0219e-05 2024-09-17T21:30:48.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796079+0000", 2024-09-17T21:30:48.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000264446 2024-09-17T21:30:48.349 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.349 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.350 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.350 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57810 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 720896~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:48.350 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:30:48.350 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.540807993000001, 2024-09-17T21:30:48.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018361253000000001, 2024-09-17T21:30:48.350 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.350 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.350 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.351 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57810 2024-09-17T21:30:48.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.351 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:30:48.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:30:48.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778011+0000", 2024-09-17T21:30:48.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:48.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778012+0000", 2024-09-17T21:30:48.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.872e-06 2024-09-17T21:30:48.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778018+0000", 2024-09-17T21:30:48.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8150000000000002e-06 2024-09-17T21:30:48.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778023+0000", 2024-09-17T21:30:48.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3939999999999998e-06 2024-09-17T21:30:48.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796106+0000", 2024-09-17T21:30:48.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018083446999999999 2024-09-17T21:30:48.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796159+0000", 2024-09-17T21:30:48.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3029999999999999e-05 2024-09-17T21:30:48.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796371+0000", 2024-09-17T21:30:48.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021231700000000001 2024-09-17T21:30:48.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.354 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.354 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57811 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:48.355 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:48.355 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.540789573000001, 2024-09-17T21:30:48.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018666035000000001, 2024-09-17T21:30:48.355 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.355 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.355 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.355 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57811 2024-09-17T21:30:48.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.355 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:48.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:48.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:48.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:30:48.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778031+0000", 2024-09-17T21:30:48.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.796e-06 2024-09-17T21:30:48.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778036+0000", 2024-09-17T21:30:48.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3290000000000001e-06 2024-09-17T21:30:48.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778041+0000", 2024-09-17T21:30:48.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5839999999999996e-06 2024-09-17T21:30:48.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796409+0000", 2024-09-17T21:30:48.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018368703 2024-09-17T21:30:48.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796483+0000", 2024-09-17T21:30:48.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3751999999999995e-05 2024-09-17T21:30:48.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796695+0000", 2024-09-17T21:30:48.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021153 2024-09-17T21:30:48.359 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.359 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.359 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.359 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57812 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1245184~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:48.359 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:48.359 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.540770676000001, 2024-09-17T21:30:48.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019000064000000001, 2024-09-17T21:30:48.360 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.360 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.360 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.360 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.360 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.360 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.360 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57812 2024-09-17T21:30:48.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.360 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:48.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:48.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:48.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:30:48.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778050+0000", 2024-09-17T21:30:48.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7e-06 2024-09-17T21:30:48.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778055+0000", 2024-09-17T21:30:48.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4049999999999999e-06 2024-09-17T21:30:48.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778059+0000", 2024-09-17T21:30:48.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1479999999999999e-06 2024-09-17T21:30:48.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796734+0000", 2024-09-17T21:30:48.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018674399000000001 2024-09-17T21:30:48.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796786+0000", 2024-09-17T21:30:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2879e-05 2024-09-17T21:30:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797048+0000", 2024-09-17T21:30:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026118400000000003 2024-09-17T21:30:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57813 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1507328~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:30:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.540752783999999, 2024-09-17T21:30:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019279392999999999, 2024-09-17T21:30:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57813 2024-09-17T21:30:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:30:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:30:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778066+0000", 2024-09-17T21:30:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6199999999999999e-07 2024-09-17T21:30:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778068+0000", 2024-09-17T21:30:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7990000000000001e-06 2024-09-17T21:30:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778074+0000", 2024-09-17T21:30:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:30:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778078+0000", 2024-09-17T21:30:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2579999999999997e-06 2024-09-17T21:30:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797077+0000", 2024-09-17T21:30:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018999180000000001 2024-09-17T21:30:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797123+0000", 2024-09-17T21:30:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5834999999999997e-05 2024-09-17T21:30:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797345+0000", 2024-09-17T21:30:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022191100000000001 2024-09-17T21:30:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57814 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:30:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.540733912, 2024-09-17T21:30:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019604947000000001, 2024-09-17T21:30:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57814 2024-09-17T21:30:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:30:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:30:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778085+0000", 2024-09-17T21:30:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778086+0000", 2024-09-17T21:30:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7850000000000001e-06 2024-09-17T21:30:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778093+0000", 2024-09-17T21:30:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6449999999999999e-06 2024-09-17T21:30:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778098+0000", 2024-09-17T21:30:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:30:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797383+0000", 2024-09-17T21:30:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019284879000000001 2024-09-17T21:30:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797431+0000", 2024-09-17T21:30:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8856000000000001e-05 2024-09-17T21:30:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797689+0000", 2024-09-17T21:30:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025785800000000002 2024-09-17T21:30:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57815 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2031616~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.540713243999999, 2024-09-17T21:30:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019910231, 2024-09-17T21:30:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57815 2024-09-17T21:30:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:30:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778118+0000", 2024-09-17T21:30:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2967e-05 2024-09-17T21:30:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778125+0000", 2024-09-17T21:30:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.601e-06 2024-09-17T21:30:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778130+0000", 2024-09-17T21:30:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1839999999999998e-06 2024-09-17T21:30:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797720+0000", 2024-09-17T21:30:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019589766000000002 2024-09-17T21:30:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797765+0000", 2024-09-17T21:30:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5281999999999998e-05 2024-09-17T21:30:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798015+0000", 2024-09-17T21:30:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025006200000000001 2024-09-17T21:30:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57816 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2293760~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.540680649999999, 2024-09-17T21:30:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020175322999999998, 2024-09-17T21:30:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57816 2024-09-17T21:30:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:30:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778140+0000", 2024-09-17T21:30:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T21:30:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778145+0000", 2024-09-17T21:30:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7320000000000003e-06 2024-09-17T21:30:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778154+0000", 2024-09-17T21:30:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6969999999999999e-06 2024-09-17T21:30:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798049+0000", 2024-09-17T21:30:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019894603 2024-09-17T21:30:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798106+0000", 2024-09-17T21:30:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7207000000000003e-05 2024-09-17T21:30:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798313+0000", 2024-09-17T21:30:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206861 2024-09-17T21:30:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57817 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2555904~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.540657404000001, 2024-09-17T21:30:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020466314999999999, 2024-09-17T21:30:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57817 2024-09-17T21:30:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2399999999999999e-07 2024-09-17T21:30:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778163+0000", 2024-09-17T21:30:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.204e-06 2024-09-17T21:30:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778169+0000", 2024-09-17T21:30:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:30:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778173+0000", 2024-09-17T21:30:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:30:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798342+0000", 2024-09-17T21:30:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020168416000000002 2024-09-17T21:30:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798388+0000", 2024-09-17T21:30:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6171000000000002e-05 2024-09-17T21:30:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798627+0000", 2024-09-17T21:30:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000239058 2024-09-17T21:30:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57818 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2818048~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:30:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.540638706999999, 2024-09-17T21:30:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020767240999999999, 2024-09-17T21:30:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57818 2024-09-17T21:30:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:30:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:30:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778180+0000", 2024-09-17T21:30:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:30:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778182+0000", 2024-09-17T21:30:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7269999999999999e-06 2024-09-17T21:30:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778188+0000", 2024-09-17T21:30:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6080000000000001e-06 2024-09-17T21:30:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778202+0000", 2024-09-17T21:30:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3369e-05 2024-09-17T21:30:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798660+0000", 2024-09-17T21:30:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020458171000000001 2024-09-17T21:30:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798704+0000", 2024-09-17T21:30:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4273000000000001e-05 2024-09-17T21:30:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798947+0000", 2024-09-17T21:30:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242651 2024-09-17T21:30:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57819 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:30:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.540607838, 2024-09-17T21:30:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020892340999999998, 2024-09-17T21:30:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57819 2024-09-17T21:30:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:30:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:30:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778211+0000", 2024-09-17T21:30:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:30:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778217+0000", 2024-09-17T21:30:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T21:30:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778224+0000", 2024-09-17T21:30:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3520000000000003e-06 2024-09-17T21:30:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778228+0000", 2024-09-17T21:30:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7480000000000001e-06 2024-09-17T21:30:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798977+0000", 2024-09-17T21:30:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020748571 2024-09-17T21:30:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799035+0000", 2024-09-17T21:30:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7908999999999997e-05 2024-09-17T21:30:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799103+0000", 2024-09-17T21:30:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7794999999999997e-05 2024-09-17T21:30:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57820 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.540582646000001, 2024-09-17T21:30:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021016658000000001, 2024-09-17T21:30:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57820 2024-09-17T21:30:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:30:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778238+0000", 2024-09-17T21:30:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.945e-06 2024-09-17T21:30:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778244+0000", 2024-09-17T21:30:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6520000000000003e-06 2024-09-17T21:30:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778248+0000", 2024-09-17T21:30:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5920000000000002e-06 2024-09-17T21:30:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799131+0000", 2024-09-17T21:30:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020883288999999999 2024-09-17T21:30:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799185+0000", 2024-09-17T21:30:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3223000000000002e-05 2024-09-17T21:30:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799252+0000", 2024-09-17T21:30:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7576999999999994e-05 2024-09-17T21:30:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57821 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.540564301, 2024-09-17T21:30:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021153528000000001, 2024-09-17T21:30:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57821 2024-09-17T21:30:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-07 2024-09-17T21:30:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778256+0000", 2024-09-17T21:30:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.604e-06 2024-09-17T21:30:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778262+0000", 2024-09-17T21:30:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7629999999999997e-06 2024-09-17T21:30:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778266+0000", 2024-09-17T21:30:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:30:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799284+0000", 2024-09-17T21:30:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021017312 2024-09-17T21:30:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799326+0000", 2024-09-17T21:30:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2639000000000003e-05 2024-09-17T21:30:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799407+0000", 2024-09-17T21:30:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1077999999999999e-05 2024-09-17T21:30:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57822 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:30:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.540545990999998, 2024-09-17T21:30:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021268966, 2024-09-17T21:30:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57822 2024-09-17T21:30:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:30:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:30:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778273+0000", 2024-09-17T21:30:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:30:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778274+0000", 2024-09-17T21:30:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.598e-06 2024-09-17T21:30:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778283+0000", 2024-09-17T21:30:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8279999999999992e-06 2024-09-17T21:30:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778289+0000", 2024-09-17T21:30:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5609999999999998e-06 2024-09-17T21:30:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799432+0000", 2024-09-17T21:30:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021143862999999999 2024-09-17T21:30:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799473+0000", 2024-09-17T21:30:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1075999999999998e-05 2024-09-17T21:30:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799541+0000", 2024-09-17T21:30:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7657999999999995e-05 2024-09-17T21:30:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57823 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.540523029999999, 2024-09-17T21:30:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021379770999999999, 2024-09-17T21:30:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57823 2024-09-17T21:30:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:30:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778297+0000", 2024-09-17T21:30:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:30:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778303+0000", 2024-09-17T21:30:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4559999999999999e-06 2024-09-17T21:30:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778307+0000", 2024-09-17T21:30:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6229999999999996e-06 2024-09-17T21:30:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799560+0000", 2024-09-17T21:30:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021252931999999999 2024-09-17T21:30:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799609+0000", 2024-09-17T21:30:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8301999999999999e-05 2024-09-17T21:30:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799675+0000", 2024-09-17T21:30:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6205999999999996e-05 2024-09-17T21:30:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.419 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:48.420 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:48.420 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:48.420 INFO:teuthology.orchestra.run.smithi129.stdout:5140 read 0xadd681d thru 0xade044d (0x9c31 bytes) 2024-09-17T21:30:48.421 INFO:teuthology.orchestra.run.smithi129.stdout:5141 writesame 0x5e65657 thru 0x5e7493f data_size 0x1(0xf2e9 bytes) 2024-09-17T21:30:48.421 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:30:48.423 INFO:teuthology.orchestra.run.smithi129.stdout:5142 write 0x93345dc thru 0x933ccda (0x86ff bytes) 2024-09-17T21:30:48.424 INFO:teuthology.orchestra.run.smithi129.stdout:5144 trunc from 0xee4c718 to 0x2b7db84 2024-09-17T21:30:48.424 INFO:teuthology.orchestra.run.smithi129.stdout:5146 writesame 0xb2975cb thru 0xb2a2c09 data_size 0x2b(0xb63f bytes) 2024-09-17T21:30:48.424 INFO:teuthology.orchestra.run.smithi129.stdout:5147 write 0x671992d thru 0x6722c8a (0x935e bytes) 2024-09-17T21:30:48.424 INFO:teuthology.orchestra.run.smithi129.stdout:5149 read 0x4e7aeb1 thru 0x4e8a826 (0xf976 bytes) 2024-09-17T21:30:48.424 INFO:teuthology.orchestra.run.smithi129.stdout:5150 write 0x8c50d94 thru 0x8c52a4c (0x1cb9 bytes) 2024-09-17T21:30:48.424 INFO:teuthology.orchestra.run.smithi129.stdout:5151 read 0x7eab8f thru 0x7f93b3 (0xe825 bytes) 2024-09-17T21:30:48.424 INFO:teuthology.orchestra.run.smithi129.stdout:5152 punch from 0x5b78de8 to 0x5b82912, (0x9b2a bytes) 2024-09-17T21:30:48.424 INFO:teuthology.orchestra.run.smithi129.stdout:5153 read 0x1db7d62 thru 0x1dbc72d (0x49cc bytes) 2024-09-17T21:30:48.424 INFO:teuthology.orchestra.run.smithi129.stdout:5154 read 0x763fd90 thru 0x76437ff (0x3a70 bytes) 2024-09-17T21:30:48.424 INFO:teuthology.orchestra.run.smithi129.stdout:5156 write 0xd353572 thru 0xd35ce0f (0x989e bytes) 2024-09-17T21:30:48.425 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:48.418+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:48.425 INFO:teuthology.orchestra.run.smithi129.stdout:5157 write 0x87eb808 thru 0x87f8bbe (0xd3b7 bytes) 2024-09-17T21:30:48.437 INFO:teuthology.orchestra.run.smithi129.stdout:5159 write 0xd818d2e thru 0xd81d32e (0x4601 bytes) 2024-09-17T21:30:48.440 INFO:teuthology.orchestra.run.smithi129.stdout:5160 writesame 0xcdbeb4 thru 0xceb345 data_size 0xf(0xf492 bytes) 2024-09-17T21:30:48.446 INFO:teuthology.orchestra.run.smithi129.stdout:5161 read 0xa889efb thru 0xa8938da (0x99e0 bytes) 2024-09-17T21:30:48.446 INFO:teuthology.orchestra.run.smithi129.stdout:5162 writesame 0xbcb4019 thru 0xbcbcefd data_size 0x1(0x8ee5 bytes) 2024-09-17T21:30:48.469 INFO:teuthology.orchestra.run.smithi129.stdout:5163 write 0xe1c088b thru 0xe1c0b45 (0x2bb bytes) 2024-09-17T21:30:48.473 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:48.473 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:48.473 INFO:teuthology.orchestra.run.smithi129.stdout:5167 write 0xa4bcbe5 thru 0xa4cc429 (0xf845 bytes) 2024-09-17T21:30:48.483 INFO:teuthology.orchestra.run.smithi129.stdout:5168 trunc from 0xe1c0b46 to 0xa468c65 2024-09-17T21:30:48.495 INFO:teuthology.orchestra.run.smithi129.stdout:5169 read 0x2bddbe7 thru 0x2be2335 (0x474f bytes) 2024-09-17T21:30:48.495 INFO:teuthology.orchestra.run.smithi129.stdout:5170 flatten 2024-09-17T21:30:48.521 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:48.518+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:48.566 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:48.566 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:48.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.566 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:102357 3.5 3:a59e58b9:::rbd_data.1120274f269d.000000000000004a:28 [read 65536~16384 out=16384b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-17T21:30:48.566 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:48.562695+0000", 2024-09-17T21:30:48.567 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.00077785799999999998, 2024-09-17T21:30:48.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000801616, 2024-09-17T21:30:48.568 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.568 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.568 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.568 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.568 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.569 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.569 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 102357 2024-09-17T21:30:48.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.569 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.562695+0000", 2024-09-17T21:30:48.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.562695+0000", 2024-09-17T21:30:48.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.562696+0000", 2024-09-17T21:30:48.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5400000000000001e-07 2024-09-17T21:30:48.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.562698+0000", 2024-09-17T21:30:48.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.605e-06 2024-09-17T21:30:48.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.562705+0000", 2024-09-17T21:30:48.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1509999999999998e-06 2024-09-17T21:30:48.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.562711+0000", 2024-09-17T21:30:48.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4739999999999996e-06 2024-09-17T21:30:48.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563338+0000", 2024-09-17T21:30:48.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00062720500000000004 2024-09-17T21:30:48.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563421+0000", 2024-09-17T21:30:48.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3248e-05 2024-09-17T21:30:48.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563507+0000", 2024-09-17T21:30:48.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5381000000000001e-05 2024-09-17T21:30:48.579 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.579 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.579 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.580 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:102364 3.c 3.2831ba2c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-17T21:30:48.580 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:48.563514+0000", 2024-09-17T21:30:48.580 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4294967295.9999585, 2024-09-17T21:30:48.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6969000000000003e-05, 2024-09-17T21:30:48.581 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.581 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.581 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T21:30:48.581 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.581 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.582 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.582 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 102364 2024-09-17T21:30:48.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.582 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563514+0000", 2024-09-17T21:30:48.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563514+0000", 2024-09-17T21:30:48.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563515+0000", 2024-09-17T21:30:48.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.16e-07 2024-09-17T21:30:48.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563519+0000", 2024-09-17T21:30:48.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9179999999999996e-06 2024-09-17T21:30:48.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563527+0000", 2024-09-17T21:30:48.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8120000000000008e-06 2024-09-17T21:30:48.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563533+0000", 2024-09-17T21:30:48.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4049999999999997e-06 2024-09-17T21:30:48.586 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.586 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.586 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.586 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:102367 3.5 3.9d1a79a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-17T21:30:48.586 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:48.563566+0000", 2024-09-17T21:30:48.586 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4294967295.999907, 2024-09-17T21:30:48.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2633000000000002e-05, 2024-09-17T21:30:48.587 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.587 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.587 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T21:30:48.587 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.587 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.587 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.587 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 102367 2024-09-17T21:30:48.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.587 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563566+0000", 2024-09-17T21:30:48.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563566+0000", 2024-09-17T21:30:48.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563567+0000", 2024-09-17T21:30:48.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7400000000000003e-07 2024-09-17T21:30:48.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563570+0000", 2024-09-17T21:30:48.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1300000000000001e-06 2024-09-17T21:30:48.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563578+0000", 2024-09-17T21:30:48.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-17T21:30:48.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563584+0000", 2024-09-17T21:30:48.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.229e-06 2024-09-17T21:30:48.590 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.590 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.590 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.590 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:102368 3.1e 3.74958d9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-17T21:30:48.590 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:48.563598+0000", 2024-09-17T21:30:48.590 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4294967295.9998755, 2024-09-17T21:30:48.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6972999999999997e-05, 2024-09-17T21:30:48.591 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.591 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.591 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T21:30:48.591 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.591 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.591 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.591 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 102368 2024-09-17T21:30:48.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.591 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563598+0000", 2024-09-17T21:30:48.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563598+0000", 2024-09-17T21:30:48.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563598+0000", 2024-09-17T21:30:48.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:30:48.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563600+0000", 2024-09-17T21:30:48.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4279999999999999e-06 2024-09-17T21:30:48.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563610+0000", 2024-09-17T21:30:48.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5950000000000005e-06 2024-09-17T21:30:48.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.563616+0000", 2024-09-17T21:30:48.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9390000000000002e-06 2024-09-17T21:30:48.594 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.594 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.594 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.594 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:102298 3.b 3:d4920476:::rbd_data.11206b8c0882.0000000000000021:head [call rbd.sparse_copyup in=262184b,set-alloc-hint object_size 262144 write_size 262144,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T21:30:48.594 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:48.559332+0000", 2024-09-17T21:30:48.594 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0041406560000000004, 2024-09-17T21:30:48.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043907599999999996, 2024-09-17T21:30:48.595 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.595 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.595 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:30:48.595 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.595 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.595 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.596 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 102298 2024-09-17T21:30:48.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.596 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.559332+0000", 2024-09-17T21:30:48.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.559332+0000", 2024-09-17T21:30:48.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.559333+0000", 2024-09-17T21:30:48.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.06e-06 2024-09-17T21:30:48.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.559398+0000", 2024-09-17T21:30:48.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4783999999999995e-05 2024-09-17T21:30:48.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.559415+0000", 2024-09-17T21:30:48.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6385999999999999e-05 2024-09-17T21:30:48.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.559423+0000", 2024-09-17T21:30:48.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9740000000000002e-06 2024-09-17T21:30:48.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.559475+0000", 2024-09-17T21:30:48.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2803999999999998e-05 2024-09-17T21:30:48.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.561191+0000", 2024-09-17T21:30:48.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0017161430000000001 2024-09-17T21:30:48.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-17T21:30:48.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.561426+0000", 2024-09-17T21:30:48.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000234921 2024-09-17T21:30:48.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:48.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:48.562408+0000", 2024-09-17T21:30:48.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00098177000000000008 2024-09-17T21:30:48.601 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.601 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.601 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.601 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.601 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:30:48.601 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 5 2024-09-17T21:30:48.601 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:48.603 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:30:48.604 INFO:teuthology.orchestra.run.smithi129.stdout:5171 punch from 0x3dee7ab to 0x3dfa120, (0xb975 bytes) 2024-09-17T21:30:48.604 INFO:teuthology.orchestra.run.smithi129.stdout:5172 writesame 0x266047 thru 0x267717 data_size 0xb(0x16d1 bytes) 2024-09-17T21:30:48.604 INFO:teuthology.orchestra.run.smithi129.stdout:5173 writesame 0x1605ff thru 0x162558 data_size 0x1(0x1f5a bytes) 2024-09-17T21:30:48.604 INFO:teuthology.orchestra.run.smithi129.stdout:5174 writesame 0x2b847e9 thru 0x2b92e49 data_size 0x3(0xe661 bytes) 2024-09-17T21:30:48.604 INFO:teuthology.orchestra.run.smithi129.stdout:5175 read 0x68f291c thru 0x68fcc56 (0xa33b bytes) 2024-09-17T21:30:48.604 INFO:teuthology.orchestra.run.smithi129.stdout:5177 read 0xa1273dd thru 0xa12c1ff (0x4e23 bytes) 2024-09-17T21:30:48.604 INFO:teuthology.orchestra.run.smithi129.stdout:5179 read 0x26ceac0 thru 0x26d4bc0 (0x6101 bytes) 2024-09-17T21:30:48.604 INFO:teuthology.orchestra.run.smithi129.stdout:5180 clone 39 order 20 su 32768 sc 14 2024-09-17T21:30:48.622 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:48.619+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:48.723 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:48.720+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:48.746 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:48.746 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:48.746 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:48.746 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:48.746 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:48.754 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:30:48.824 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:48.821+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:48.859 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:48.859 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:48.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:48.859 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:48.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.859 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:49361 3.19 e46/25 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:head v 46'135, mlcod=45'133)", 2024-09-17T21:30:48.860 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:48.860 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.102984599999999, 2024-09-17T21:30:48.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043477744999999998, 2024-09-17T21:30:48.860 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.860 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.860 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:48.860 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:48.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:48.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:48.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-07 2024-09-17T21:30:48.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769703+0000", 2024-09-17T21:30:48.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016134381 2024-09-17T21:30:48.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769724+0000", 2024-09-17T21:30:48.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0781999999999999e-05 2024-09-17T21:30:48.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769734+0000", 2024-09-17T21:30:48.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6870000000000006e-06 2024-09-17T21:30:48.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769797+0000", 2024-09-17T21:30:48.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3418999999999997e-05 2024-09-17T21:30:48.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769822+0000", 2024-09-17T21:30:48.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4652999999999999e-05 2024-09-17T21:30:48.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:48.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.796982+0000", 2024-09-17T21:30:48.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027160205 2024-09-17T21:30:48.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.797047+0000", 2024-09-17T21:30:48.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4455999999999994e-05 2024-09-17T21:30:48.865 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.865 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.865 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.865 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50805 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 17825792~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:48.865 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:30:48.865 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.361401106000002, 2024-09-17T21:30:48.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035591579999999998, 2024-09-17T21:30:48.865 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.865 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.865 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.866 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.866 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.866 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.866 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50805 2024-09-17T21:30:48.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.866 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:30:48.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:30:48.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495153+0000", 2024-09-17T21:30:48.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:30:48.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495155+0000", 2024-09-17T21:30:48.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1849999999999998e-06 2024-09-17T21:30:48.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495162+0000", 2024-09-17T21:30:48.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.64e-06 2024-09-17T21:30:48.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495166+0000", 2024-09-17T21:30:48.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:30:48.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530020+0000", 2024-09-17T21:30:48.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.034853824999999998 2024-09-17T21:30:48.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530076+0000", 2024-09-17T21:30:48.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5235999999999998e-05 2024-09-17T21:30:48.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530744+0000", 2024-09-17T21:30:48.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066835500000000003 2024-09-17T21:30:48.869 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.869 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.869 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.869 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50806 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19398656~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:48.870 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:48.870 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.361380638, 2024-09-17T21:30:48.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036408459999999997, 2024-09-17T21:30:48.870 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.870 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.870 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.870 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50806 2024-09-17T21:30:48.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.870 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:48.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:48.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:48.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.72e-07 2024-09-17T21:30:48.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495175+0000", 2024-09-17T21:30:48.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:30:48.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495182+0000", 2024-09-17T21:30:48.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8079999999999999e-06 2024-09-17T21:30:48.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495186+0000", 2024-09-17T21:30:48.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5549999999999996e-06 2024-09-17T21:30:48.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530791+0000", 2024-09-17T21:30:48.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035604963000000003 2024-09-17T21:30:48.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530855+0000", 2024-09-17T21:30:48.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3286000000000004e-05 2024-09-17T21:30:48.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531581+0000", 2024-09-17T21:30:48.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00072655699999999996 2024-09-17T21:30:48.874 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.874 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.874 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.875 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50807 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23068672~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:48.875 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:48.875 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.361361752999997, 2024-09-17T21:30:48.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037183314000000002, 2024-09-17T21:30:48.875 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.875 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.875 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.875 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.875 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.875 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.875 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50807 2024-09-17T21:30:48.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.876 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:48.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:48.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:48.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9999999999999999e-07 2024-09-17T21:30:48.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495194+0000", 2024-09-17T21:30:48.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.779e-06 2024-09-17T21:30:48.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495199+0000", 2024-09-17T21:30:48.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.468e-06 2024-09-17T21:30:48.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495203+0000", 2024-09-17T21:30:48.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0010000000000003e-06 2024-09-17T21:30:48.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531636+0000", 2024-09-17T21:30:48.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036432665000000003 2024-09-17T21:30:48.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531727+0000", 2024-09-17T21:30:48.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1607999999999994e-05 2024-09-17T21:30:48.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532375+0000", 2024-09-17T21:30:48.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00064749299999999996 2024-09-17T21:30:48.879 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.879 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.879 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.880 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50808 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24641536~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:48.880 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:30:48.880 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.361345010000001, 2024-09-17T21:30:48.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037870190999999997, 2024-09-17T21:30:48.880 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.880 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.880 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.880 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.881 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50808 2024-09-17T21:30:48.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.881 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:30:48.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:30:48.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495209+0000", 2024-09-17T21:30:48.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:30:48.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495215+0000", 2024-09-17T21:30:48.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1630000000000001e-06 2024-09-17T21:30:48.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495221+0000", 2024-09-17T21:30:48.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-17T21:30:48.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495226+0000", 2024-09-17T21:30:48.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8189999999999999e-06 2024-09-17T21:30:48.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532425+0000", 2024-09-17T21:30:48.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037199135000000001 2024-09-17T21:30:48.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532531+0000", 2024-09-17T21:30:48.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106314 2024-09-17T21:30:48.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533079+0000", 2024-09-17T21:30:48.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00054739700000000003 2024-09-17T21:30:48.884 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.884 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.885 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.885 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50809 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28311552~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:48.885 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:30:48.885 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.361322043999998, 2024-09-17T21:30:48.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038597605, 2024-09-17T21:30:48.885 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.885 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.885 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.885 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.885 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.885 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.886 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50809 2024-09-17T21:30:48.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.886 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:30:48.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:30:48.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495232+0000", 2024-09-17T21:30:48.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.39e-07 2024-09-17T21:30:48.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495234+0000", 2024-09-17T21:30:48.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T21:30:48.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495239+0000", 2024-09-17T21:30:48.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7969999999999997e-06 2024-09-17T21:30:48.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495244+0000", 2024-09-17T21:30:48.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999996e-06 2024-09-17T21:30:48.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533125+0000", 2024-09-17T21:30:48.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037881523 2024-09-17T21:30:48.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533237+0000", 2024-09-17T21:30:48.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000111879 2024-09-17T21:30:48.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533829+0000", 2024-09-17T21:30:48.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000591933 2024-09-17T21:30:48.889 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.889 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.889 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.889 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50810 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 29884416~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:48.889 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:30:48.890 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.361304126, 2024-09-17T21:30:48.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039149809000000001, 2024-09-17T21:30:48.890 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.890 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.890 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.890 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.890 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.890 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.890 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50810 2024-09-17T21:30:48.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.890 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:30:48.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:30:48.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495250+0000", 2024-09-17T21:30:48.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:30:48.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495251+0000", 2024-09-17T21:30:48.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.81e-06 2024-09-17T21:30:48.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495257+0000", 2024-09-17T21:30:48.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1399999999999999e-06 2024-09-17T21:30:48.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495261+0000", 2024-09-17T21:30:48.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4229999999999998e-06 2024-09-17T21:30:48.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533879+0000", 2024-09-17T21:30:48.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038618292999999998 2024-09-17T21:30:48.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533939+0000", 2024-09-17T21:30:48.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.012e-05 2024-09-17T21:30:48.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534399+0000", 2024-09-17T21:30:48.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00045968099999999998 2024-09-17T21:30:48.894 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.894 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.894 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.894 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50850 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 2359296~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:48.894 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:48.894 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.361285750999997, 2024-09-17T21:30:48.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039731491000000001, 2024-09-17T21:30:48.894 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.894 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.895 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.895 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.895 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.895 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.895 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50850 2024-09-17T21:30:48.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.895 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:48.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:48.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:48.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3200000000000001e-07 2024-09-17T21:30:48.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495270+0000", 2024-09-17T21:30:48.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.694e-06 2024-09-17T21:30:48.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495275+0000", 2024-09-17T21:30:48.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5890000000000002e-06 2024-09-17T21:30:48.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495279+0000", 2024-09-17T21:30:48.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-17T21:30:48.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534440+0000", 2024-09-17T21:30:48.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039160805999999999 2024-09-17T21:30:48.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534526+0000", 2024-09-17T21:30:48.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5327999999999993e-05 2024-09-17T21:30:48.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534999+0000", 2024-09-17T21:30:48.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00047353400000000001 2024-09-17T21:30:48.898 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.898 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.898 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.899 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50851 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 3932160~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:48.899 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:30:48.899 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.361268139000003, 2024-09-17T21:30:48.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040211991000000002, 2024-09-17T21:30:48.899 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.899 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.899 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.899 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.899 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.899 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.899 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50851 2024-09-17T21:30:48.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.899 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:30:48.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:30:48.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495286+0000", 2024-09-17T21:30:48.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:48.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495287+0000", 2024-09-17T21:30:48.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6640000000000001e-06 2024-09-17T21:30:48.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495293+0000", 2024-09-17T21:30:48.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6710000000000004e-06 2024-09-17T21:30:48.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495298+0000", 2024-09-17T21:30:48.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6700000000000002e-06 2024-09-17T21:30:48.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535017+0000", 2024-09-17T21:30:48.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039719059000000001 2024-09-17T21:30:48.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535067+0000", 2024-09-17T21:30:48.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0046000000000002e-05 2024-09-17T21:30:48.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535497+0000", 2024-09-17T21:30:48.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043050299999999999 2024-09-17T21:30:48.903 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.903 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.903 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.903 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50852 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 7602176~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:48.903 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:30:48.904 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.361249221999998, 2024-09-17T21:30:48.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041166337999999997, 2024-09-17T21:30:48.904 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.904 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.904 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.904 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50852 2024-09-17T21:30:48.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.904 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:30:48.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:30:48.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495305+0000", 2024-09-17T21:30:48.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1600000000000002e-07 2024-09-17T21:30:48.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495307+0000", 2024-09-17T21:30:48.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0980000000000001e-06 2024-09-17T21:30:48.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495312+0000", 2024-09-17T21:30:48.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6210000000000001e-06 2024-09-17T21:30:48.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495317+0000", 2024-09-17T21:30:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T21:30:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535531+0000", 2024-09-17T21:30:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040214580999999999 2024-09-17T21:30:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535618+0000", 2024-09-17T21:30:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6648000000000001e-05 2024-09-17T21:30:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536471+0000", 2024-09-17T21:30:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00085251599999999999 2024-09-17T21:30:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50853 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 9175040~204800] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:48.908 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.361230890999998, 2024-09-17T21:30:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041673496999999997, 2024-09-17T21:30:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50853 2024-09-17T21:30:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:30:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:30:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495334+0000", 2024-09-17T21:30:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.076e-05 2024-09-17T21:30:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495340+0000", 2024-09-17T21:30:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:30:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495345+0000", 2024-09-17T21:30:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:30:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536514+0000", 2024-09-17T21:30:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041168534999999999 2024-09-17T21:30:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536572+0000", 2024-09-17T21:30:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8742000000000001e-05 2024-09-17T21:30:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536996+0000", 2024-09-17T21:30:48.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042368899999999999 2024-09-17T21:30:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50854 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 12845056~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.361201831999999, 2024-09-17T21:30:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042777397000000002, 2024-09-17T21:30:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50854 2024-09-17T21:30:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:30:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.82e-07 2024-09-17T21:30:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495354+0000", 2024-09-17T21:30:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.091e-06 2024-09-17T21:30:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495360+0000", 2024-09-17T21:30:48.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.716e-06 2024-09-17T21:30:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495365+0000", 2024-09-17T21:30:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2939999999999996e-06 2024-09-17T21:30:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537020+0000", 2024-09-17T21:30:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041654419999999998 2024-09-17T21:30:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537105+0000", 2024-09-17T21:30:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5857000000000001e-05 2024-09-17T21:30:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538129+0000", 2024-09-17T21:30:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010235369999999999 2024-09-17T21:30:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50855 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 14417920~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:48.917 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.361181866999999, 2024-09-17T21:30:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043254325000000003, 2024-09-17T21:30:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50855 2024-09-17T21:30:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:30:48.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:30:48.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495374+0000", 2024-09-17T21:30:48.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.922e-06 2024-09-17T21:30:48.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495379+0000", 2024-09-17T21:30:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-17T21:30:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495384+0000", 2024-09-17T21:30:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-17T21:30:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538167+0000", 2024-09-17T21:30:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042782845 2024-09-17T21:30:48.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538218+0000", 2024-09-17T21:30:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1335000000000002e-05 2024-09-17T21:30:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538626+0000", 2024-09-17T21:30:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000407656 2024-09-17T21:30:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50856 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 18087936~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.361163412000003, 2024-09-17T21:30:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043911444000000001, 2024-09-17T21:30:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50856 2024-09-17T21:30:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.923 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:30:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0499999999999999e-07 2024-09-17T21:30:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495392+0000", 2024-09-17T21:30:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7430000000000001e-06 2024-09-17T21:30:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495398+0000", 2024-09-17T21:30:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3040000000000001e-06 2024-09-17T21:30:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495403+0000", 2024-09-17T21:30:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5600000000000004e-06 2024-09-17T21:30:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538660+0000", 2024-09-17T21:30:48.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043256832000000002 2024-09-17T21:30:48.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538746+0000", 2024-09-17T21:30:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6448999999999999e-05 2024-09-17T21:30:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539301+0000", 2024-09-17T21:30:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055525099999999999 2024-09-17T21:30:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.929 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50857 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19660800~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:30:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.361144007, 2024-09-17T21:30:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044408156999999997, 2024-09-17T21:30:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50857 2024-09-17T21:30:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:30:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:30:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495410+0000", 2024-09-17T21:30:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:30:48.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495412+0000", 2024-09-17T21:30:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9580000000000001e-06 2024-09-17T21:30:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495418+0000", 2024-09-17T21:30:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1909999999999997e-06 2024-09-17T21:30:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495423+0000", 2024-09-17T21:30:48.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2429999999999996e-06 2024-09-17T21:30:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539338+0000", 2024-09-17T21:30:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043914478 2024-09-17T21:30:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539393+0000", 2024-09-17T21:30:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5226999999999998e-05 2024-09-17T21:30:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539818+0000", 2024-09-17T21:30:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042469300000000002 2024-09-17T21:30:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50858 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23330816~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.361122909999999, 2024-09-17T21:30:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045010798999999997, 2024-09-17T21:30:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.934 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50858 2024-09-17T21:30:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:30:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:30:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495433+0000", 2024-09-17T21:30:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.787e-06 2024-09-17T21:30:48.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495438+0000", 2024-09-17T21:30:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:30:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495442+0000", 2024-09-17T21:30:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2640000000000002e-06 2024-09-17T21:30:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539852+0000", 2024-09-17T21:30:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044409592999999997 2024-09-17T21:30:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539940+0000", 2024-09-17T21:30:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7696999999999997e-05 2024-09-17T21:30:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540441+0000", 2024-09-17T21:30:48.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050164299999999997 2024-09-17T21:30:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50859 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24903680~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:30:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.361105232, 2024-09-17T21:30:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045571569999999999, 2024-09-17T21:30:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.939 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50859 2024-09-17T21:30:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:30:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:30:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495449+0000", 2024-09-17T21:30:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:30:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495450+0000", 2024-09-17T21:30:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7889999999999999e-06 2024-09-17T21:30:48.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495456+0000", 2024-09-17T21:30:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5679999999999999e-06 2024-09-17T21:30:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495460+0000", 2024-09-17T21:30:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4989999999999996e-06 2024-09-17T21:30:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540472+0000", 2024-09-17T21:30:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045011585999999999 2024-09-17T21:30:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540554+0000", 2024-09-17T21:30:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2443999999999999e-05 2024-09-17T21:30:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541020+0000", 2024-09-17T21:30:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00046534300000000001 2024-09-17T21:30:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.943 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50860 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28573696~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.361086878999998, 2024-09-17T21:30:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046167899999999998, 2024-09-17T21:30:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50860 2024-09-17T21:30:48.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:30:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:30:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495469+0000", 2024-09-17T21:30:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.99e-06 2024-09-17T21:30:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495475+0000", 2024-09-17T21:30:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:30:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495480+0000", 2024-09-17T21:30:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:30:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541050+0000", 2024-09-17T21:30:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045569853 2024-09-17T21:30:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541138+0000", 2024-09-17T21:30:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7997000000000005e-05 2024-09-17T21:30:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541634+0000", 2024-09-17T21:30:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00049685500000000004 2024-09-17T21:30:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50861 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 30146560~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:48.948 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.361067702, 2024-09-17T21:30:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046690553000000003, 2024-09-17T21:30:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50861 2024-09-17T21:30:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:30:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:30:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495488+0000", 2024-09-17T21:30:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:30:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495495+0000", 2024-09-17T21:30:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:30:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495500+0000", 2024-09-17T21:30:48.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8620000000000002e-06 2024-09-17T21:30:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541687+0000", 2024-09-17T21:30:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046186434999999998 2024-09-17T21:30:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541741+0000", 2024-09-17T21:30:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4228999999999999e-05 2024-09-17T21:30:48.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.542176+0000", 2024-09-17T21:30:48.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043531100000000002 2024-09-17T21:30:48.953 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.953 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.953 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97804 3.7 3:e725aa05:::rbd_data.1120c318df8c.0000000000000003:24 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:30:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.895121601, 2024-09-17T21:30:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035514416, 2024-09-17T21:30:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97804 2024-09-17T21:30:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.954 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:30:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:30:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:30:48.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:30:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961434+0000", 2024-09-17T21:30:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8959999999999999e-06 2024-09-17T21:30:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961439+0000", 2024-09-17T21:30:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T21:30:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:48.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961444+0000", 2024-09-17T21:30:48.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4970000000000003e-06 2024-09-17T21:30:48.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:48.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996466+0000", 2024-09-17T21:30:48.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035022438000000003 2024-09-17T21:30:48.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:48.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996556+0000", 2024-09-17T21:30:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9561000000000005e-05 2024-09-17T21:30:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996946+0000", 2024-09-17T21:30:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039028700000000001 2024-09-17T21:30:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:48.958 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:48.958 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:48.959 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:30:48.960 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.6 to 1 (10000) 2024-09-17T21:30:48.961 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:30:48.962 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-17T21:30:48.962 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:48.958+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:49.033 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-17T21:30:49.034 INFO:teuthology.orchestra.run.smithi129.stdout:5181 write 0x297b882 thru 0x297f757 (0x3ed6 bytes) 2024-09-17T21:30:49.042 INFO:teuthology.orchestra.run.smithi129.stdout:5182 punch from 0x7885335 to 0x7892899, (0xd564 bytes) 2024-09-17T21:30:49.042 INFO:teuthology.orchestra.run.smithi129.stdout:5183 read 0x4af671 thru 0x4b808e (0x8a1e bytes) 2024-09-17T21:30:49.049 INFO:teuthology.orchestra.run.smithi129.stdout:5184 writesame 0xadde9fb thru 0xade8dd4 data_size 0x3(0xa3da bytes) 2024-09-17T21:30:49.049 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:49.049 INFO:teuthology.orchestra.run.smithi129.stdout:5186 read 0x81e2ce9 thru 0x81e8649 (0x5961 bytes) 2024-09-17T21:30:49.049 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:49.056 INFO:teuthology.orchestra.run.smithi129.stdout:5189 write 0xbc72bfd thru 0xbc80c5a (0xe05e bytes) 2024-09-17T21:30:49.061 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:49.058+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:49.071 INFO:teuthology.orchestra.run.smithi129.stdout:5191 writesame 0xdab4863 thru 0xdabdf40 data_size 0x29(0x96de bytes) 2024-09-17T21:30:49.071 INFO:teuthology.orchestra.run.smithi129.stdout:5192 write 0x963e176 thru 0x96478fc (0x9787 bytes) 2024-09-17T21:30:49.072 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:49.072 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:49.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.072 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:102644 3.1d 3:b93a2489:::rbd_object_map.1120836ca0fb:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-17T21:30:49.073 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:49.070262+0000", 2024-09-17T21:30:49.073 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00075715600000000004, 2024-09-17T21:30:49.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00077556999999999995, 2024-09-17T21:30:49.073 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.073 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.073 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:30:49.073 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.073 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.073 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.073 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 102644 2024-09-17T21:30:49.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.073 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:49.070262+0000", 2024-09-17T21:30:49.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:49.070262+0000", 2024-09-17T21:30:49.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:49.070263+0000", 2024-09-17T21:30:49.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8100000000000002e-07 2024-09-17T21:30:49.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:49.070266+0000", 2024-09-17T21:30:49.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3790000000000002e-06 2024-09-17T21:30:49.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:49.070274+0000", 2024-09-17T21:30:49.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9410000000000007e-06 2024-09-17T21:30:49.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:49.070278+0000", 2024-09-17T21:30:49.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:30:49.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:49.070326+0000", 2024-09-17T21:30:49.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.782e-05 2024-09-17T21:30:49.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:49.070404+0000", 2024-09-17T21:30:49.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7976e-05 2024-09-17T21:30:49.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-17T21:30:49.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:49.070736+0000", 2024-09-17T21:30:49.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000332317 2024-09-17T21:30:49.077 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.077 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.077 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.077 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:102645 3.1d 3.91245c9d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-17T21:30:49.077 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:49.070287+0000", 2024-09-17T21:30:49.077 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.000731903, 2024-09-17T21:30:49.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00078754400000000005, 2024-09-17T21:30:49.077 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.077 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.077 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T21:30:49.078 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.078 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.078 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.078 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 102645 2024-09-17T21:30:49.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.078 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:49.070287+0000", 2024-09-17T21:30:49.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:49.070287+0000", 2024-09-17T21:30:49.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:49.070288+0000", 2024-09-17T21:30:49.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1e-07 2024-09-17T21:30:49.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:49.070289+0000", 2024-09-17T21:30:49.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.759e-06 2024-09-17T21:30:49.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:49.070293+0000", 2024-09-17T21:30:49.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7450000000000001e-06 2024-09-17T21:30:49.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:49.070296+0000", 2024-09-17T21:30:49.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:30:49.080 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.080 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.080 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.080 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.080 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-17T21:30:49.081 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 2 2024-09-17T21:30:49.081 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:49.081 INFO:teuthology.orchestra.run.smithi129.stdout:5193 flatten 2024-09-17T21:30:49.084 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:30:49.134 INFO:teuthology.orchestra.run.smithi129.stdout:5194 read 0x45bc825 thru 0x45c4ea8 (0x8684 bytes) 2024-09-17T21:30:49.134 INFO:teuthology.orchestra.run.smithi129.stdout:5195 read 0xc3a6673 thru 0xc3ab1fd (0x4b8b bytes) 2024-09-17T21:30:49.134 INFO:teuthology.orchestra.run.smithi129.stdout:5197 write 0x4f524a5 thru 0x4f612b2 (0xee0e bytes) 2024-09-17T21:30:49.142 INFO:teuthology.orchestra.run.smithi129.stdout:5198 read 0x10a6eac thru 0x10b1245 (0xa39a bytes) 2024-09-17T21:30:49.142 INFO:teuthology.orchestra.run.smithi129.stdout:5199 write 0xc355ddc thru 0xc35d482 (0x76a7 bytes) 2024-09-17T21:30:49.150 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:49.150 INFO:teuthology.orchestra.run.smithi129.stdout:5201 write 0x92d1bc6 thru 0x92e1785 (0xfbc0 bytes) 2024-09-17T21:30:49.161 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:49.159+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:49.162 INFO:teuthology.orchestra.run.smithi129.stdout:5203 writesame 0x2200474 thru 0x220fd32 data_size 0xb(0xf8bf bytes) 2024-09-17T21:30:49.166 INFO:teuthology.orchestra.run.smithi129.stdout:5205 write 0x26d6b03 thru 0x26e0fcc (0xa4ca bytes) 2024-09-17T21:30:49.170 INFO:teuthology.orchestra.run.smithi129.stdout:5206 write 0x3f8bd6e thru 0x3f98488 (0xc71b bytes) 2024-09-17T21:30:49.190 INFO:teuthology.orchestra.run.smithi129.stdout:5207 trunc from 0xdabdf41 to 0x3462e02 2024-09-17T21:30:49.207 INFO:teuthology.orchestra.run.smithi129.stdout:5208 read 0x119c87f thru 0x119cda6 (0x528 bytes) 2024-09-17T21:30:49.207 INFO:teuthology.orchestra.run.smithi129.stdout:5209 punch from 0xf38334 to 0xf3b9ec, (0x36b8 bytes) 2024-09-17T21:30:49.207 INFO:teuthology.orchestra.run.smithi129.stdout:5211 read 0x1c17be9 thru 0x1c20bbb (0x8fd3 bytes) 2024-09-17T21:30:49.208 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:49.208 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:49.208 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:49.208 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:49.208 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:49.214 INFO:teuthology.orchestra.run.smithi129.stdout:5214 write 0x3e2f9bb thru 0x3e377a9 (0x7def bytes) 2024-09-17T21:30:49.218 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:30:49.219 INFO:teuthology.orchestra.run.smithi129.stdout:5217 writesame 0x2665116 thru 0x267500b data_size 0x7a(0xfef6 bytes) 2024-09-17T21:30:49.221 INFO:teuthology.orchestra.run.smithi129.stdout:5219 punch from 0x2e9eaa to 0x2f308d, (0x91e3 bytes) 2024-09-17T21:30:49.221 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:49.230 INFO:teuthology.orchestra.run.smithi129.stdout:5221 trunc from 0x3e377aa to 0xa600b1f 2024-09-17T21:30:49.235 INFO:teuthology.orchestra.run.smithi129.stdout:5222 write 0x1f2b9b4 thru 0x1f37ca1 (0xc2ee bytes) 2024-09-17T21:30:49.240 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:49.240 INFO:teuthology.orchestra.run.smithi129.stdout:5226 punch from 0xabc74a to 0xabf5bf, (0x2e75 bytes) 2024-09-17T21:30:49.247 INFO:teuthology.orchestra.run.smithi129.stdout:5228 trunc from 0xa600b1f to 0x4ef9d48 2024-09-17T21:30:49.253 INFO:teuthology.orchestra.run.smithi129.stdout:5233 write 0x27cef3c thru 0x27dcd63 (0xde28 bytes) 2024-09-17T21:30:49.258 INFO:teuthology.orchestra.run.smithi129.stdout:5234 punch from 0x45b83ca to 0x45c2dbf, (0xa9f5 bytes) 2024-09-17T21:30:49.259 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:49.263 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:49.261+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:49.266 INFO:teuthology.orchestra.run.smithi129.stdout:5236 write 0x6fd3cee thru 0x6fd7fde (0x42f1 bytes) 2024-09-17T21:30:49.279 INFO:teuthology.orchestra.run.smithi129.stdout:5237 write 0x9223790 thru 0x922dbd1 (0xa442 bytes) 2024-09-17T21:30:49.291 INFO:teuthology.orchestra.run.smithi129.stdout:5239 writesame 0xaad95a9 thru 0xaae4663 data_size 0xb(0xb0bb bytes) 2024-09-17T21:30:49.291 INFO:teuthology.orchestra.run.smithi129.stdout:5242 punch from 0x1c526f1 to 0x1c59c0c, (0x751b bytes) 2024-09-17T21:30:49.291 INFO:teuthology.orchestra.run.smithi129.stdout:5243 read 0x6a9c5bd thru 0x6aa6c44 (0xa688 bytes) 2024-09-17T21:30:49.291 INFO:teuthology.orchestra.run.smithi129.stdout:5244 read 0x687b3c5 thru 0x688ae3e (0xfa7a bytes) 2024-09-17T21:30:49.291 INFO:teuthology.orchestra.run.smithi129.stdout:5245 writesame 0x25dc46 thru 0x2675b9 data_size 0x1c(0x9974 bytes) 2024-09-17T21:30:49.293 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:49.293 INFO:teuthology.orchestra.run.smithi129.stdout:5247 write 0x1f82e58 thru 0x1f89a02 (0x6bab bytes) 2024-09-17T21:30:49.294 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:49.295 INFO:teuthology.orchestra.run.smithi129.stdout:5249 read 0x1a68d43 thru 0x1a6bd33 (0x2ff1 bytes) 2024-09-17T21:30:49.302 INFO:teuthology.orchestra.run.smithi129.stdout:5252 trunc from 0xaae4664 to 0x3ecc941 2024-09-17T21:30:49.314 INFO:teuthology.orchestra.run.smithi129.stdout:5253 write 0x1d9b471 thru 0x1da1bc2 (0x6752 bytes) 2024-09-17T21:30:49.324 INFO:teuthology.orchestra.run.smithi129.stdout:5255 trunc from 0x3ecc941 to 0xa06e9e6 2024-09-17T21:30:49.330 INFO:teuthology.orchestra.run.smithi129.stdout:5257 read 0x9d2f304 thru 0x9d3bfdf (0xccdc bytes) 2024-09-17T21:30:49.330 INFO:teuthology.orchestra.run.smithi129.stdout:5258 punch from 0x34c5326 to 0x34d3a78, (0xe752 bytes) 2024-09-17T21:30:49.330 INFO:teuthology.orchestra.run.smithi129.stdout:5260 read 0x3e7c522 thru 0x3e7c929 (0x408 bytes) 2024-09-17T21:30:49.330 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:49.332 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21342 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2082816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.652859239000001, 2024-09-17T21:30:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022083523000000001, 2024-09-17T21:30:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21342 2024-09-17T21:30:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:30:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:30:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676659+0000", 2024-09-17T21:30:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0950000000000001e-06 2024-09-17T21:30:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676662+0000", 2024-09-17T21:30:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.971e-06 2024-09-17T21:30:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676665+0000", 2024-09-17T21:30:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6189999999999998e-06 2024-09-17T21:30:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698568+0000", 2024-09-17T21:30:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021903549000000001 2024-09-17T21:30:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698668+0000", 2024-09-17T21:30:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9778999999999995e-05 2024-09-17T21:30:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698741+0000", 2024-09-17T21:30:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000006e-05 2024-09-17T21:30:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21343 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2095104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:30:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.652849301000003, 2024-09-17T21:30:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022504024000000001, 2024-09-17T21:30:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21343 2024-09-17T21:30:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:30:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:30:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676669+0000", 2024-09-17T21:30:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.003e-06 2024-09-17T21:30:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676672+0000", 2024-09-17T21:30:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9280000000000002e-06 2024-09-17T21:30:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676674+0000", 2024-09-17T21:30:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5469999999999999e-06 2024-09-17T21:30:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699015+0000", 2024-09-17T21:30:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022340563000000001 2024-09-17T21:30:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699102+0000", 2024-09-17T21:30:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7425999999999999e-05 2024-09-17T21:30:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699172+0000", 2024-09-17T21:30:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9346999999999999e-05 2024-09-17T21:30:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45576 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.624615501000001, 2024-09-17T21:30:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022024660000000001, 2024-09-17T21:30:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45576 2024-09-17T21:30:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:30:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704902+0000", 2024-09-17T21:30:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9100000000000004e-07 2024-09-17T21:30:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704905+0000", 2024-09-17T21:30:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6139999999999999e-06 2024-09-17T21:30:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704911+0000", 2024-09-17T21:30:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8619999999999997e-06 2024-09-17T21:30:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704917+0000", 2024-09-17T21:30:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:30:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726681+0000", 2024-09-17T21:30:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021764569000000001 2024-09-17T21:30:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726726+0000", 2024-09-17T21:30:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4218999999999998e-05 2024-09-17T21:30:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726926+0000", 2024-09-17T21:30:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000200466 2024-09-17T21:30:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45577 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.624591844999998, 2024-09-17T21:30:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022269694999999999, 2024-09-17T21:30:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45577 2024-09-17T21:30:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:30:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704926+0000", 2024-09-17T21:30:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:30:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704928+0000", 2024-09-17T21:30:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9989999999999998e-06 2024-09-17T21:30:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704934+0000", 2024-09-17T21:30:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:30:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704939+0000", 2024-09-17T21:30:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5909999999999998e-06 2024-09-17T21:30:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726954+0000", 2024-09-17T21:30:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022014885000000001 2024-09-17T21:30:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726995+0000", 2024-09-17T21:30:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1474000000000003e-05 2024-09-17T21:30:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727195+0000", 2024-09-17T21:30:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019968300000000001 2024-09-17T21:30:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45578 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.624570601000002, 2024-09-17T21:30:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022484006000000001, 2024-09-17T21:30:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45578 2024-09-17T21:30:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:30:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704947+0000", 2024-09-17T21:30:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1399999999999997e-07 2024-09-17T21:30:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704949+0000", 2024-09-17T21:30:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0959999999999999e-06 2024-09-17T21:30:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704956+0000", 2024-09-17T21:30:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9330000000000004e-06 2024-09-17T21:30:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704961+0000", 2024-09-17T21:30:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6759999999999998e-06 2024-09-17T21:30:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727213+0000", 2024-09-17T21:30:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022252568 2024-09-17T21:30:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727259+0000", 2024-09-17T21:30:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5238000000000003e-05 2024-09-17T21:30:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727430+0000", 2024-09-17T21:30:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000171881 2024-09-17T21:30:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45579 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.624548673, 2024-09-17T21:30:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022749080000000001, 2024-09-17T21:30:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45579 2024-09-17T21:30:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:30:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704969+0000", 2024-09-17T21:30:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:30:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704971+0000", 2024-09-17T21:30:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.097e-06 2024-09-17T21:30:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704977+0000", 2024-09-17T21:30:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6329999999999999e-06 2024-09-17T21:30:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704983+0000", 2024-09-17T21:30:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:30:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727457+0000", 2024-09-17T21:30:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022474521000000001 2024-09-17T21:30:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727498+0000", 2024-09-17T21:30:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0558e-05 2024-09-17T21:30:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727717+0000", 2024-09-17T21:30:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219805 2024-09-17T21:30:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45652 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.623402534999997, 2024-09-17T21:30:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022100465999999999, 2024-09-17T21:30:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45652 2024-09-17T21:30:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:30:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706115+0000", 2024-09-17T21:30:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:30:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706117+0000", 2024-09-17T21:30:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-06 2024-09-17T21:30:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706122+0000", 2024-09-17T21:30:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.288e-06 2024-09-17T21:30:49.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706136+0000", 2024-09-17T21:30:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3409e-05 2024-09-17T21:30:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728112+0000", 2024-09-17T21:30:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021976724999999999 2024-09-17T21:30:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728151+0000", 2024-09-17T21:30:49.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8318999999999997e-05 2024-09-17T21:30:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728215+0000", 2024-09-17T21:30:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4172999999999997e-05 2024-09-17T21:30:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45653 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:49.368 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.623373325000003, 2024-09-17T21:30:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022249517999999999, 2024-09-17T21:30:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45653 2024-09-17T21:30:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:30:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T21:30:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706147+0000", 2024-09-17T21:30:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4430000000000002e-06 2024-09-17T21:30:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706152+0000", 2024-09-17T21:30:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5350000000000004e-06 2024-09-17T21:30:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706157+0000", 2024-09-17T21:30:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8420000000000004e-06 2024-09-17T21:30:49.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728246+0000", 2024-09-17T21:30:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022089237000000001 2024-09-17T21:30:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728309+0000", 2024-09-17T21:30:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2356999999999999e-05 2024-09-17T21:30:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:30:49.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4596000000000001e-05 2024-09-17T21:30:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45657 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 36864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.623342061000002, 2024-09-17T21:30:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022218252000000001, 2024-09-17T21:30:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.373 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45657 2024-09-17T21:30:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:30:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8299999999999997e-07 2024-09-17T21:30:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706178+0000", 2024-09-17T21:30:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3700000000000002e-06 2024-09-17T21:30:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706184+0000", 2024-09-17T21:30:49.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.144e-06 2024-09-17T21:30:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706190+0000", 2024-09-17T21:30:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8590000000000001e-06 2024-09-17T21:30:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728240+0000", 2024-09-17T21:30:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022049804999999999 2024-09-17T21:30:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728307+0000", 2024-09-17T21:30:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7477999999999993e-05 2024-09-17T21:30:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:30:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6112999999999994e-05 2024-09-17T21:30:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45658 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:49.377 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.623318941000001, 2024-09-17T21:30:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022376455, 2024-09-17T21:30:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45658 2024-09-17T21:30:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:30:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:30:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706200+0000", 2024-09-17T21:30:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:30:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706206+0000", 2024-09-17T21:30:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7039999999999999e-06 2024-09-17T21:30:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706211+0000", 2024-09-17T21:30:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9429999999999999e-06 2024-09-17T21:30:49.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728418+0000", 2024-09-17T21:30:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022206760999999998 2024-09-17T21:30:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728497+0000", 2024-09-17T21:30:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9493999999999995e-05 2024-09-17T21:30:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728574+0000", 2024-09-17T21:30:49.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7180000000000003e-05 2024-09-17T21:30:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45707 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 212992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.621614874999999, 2024-09-17T21:30:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022235550999999999, 2024-09-17T21:30:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.382 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45707 2024-09-17T21:30:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:30:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:30:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707905+0000", 2024-09-17T21:30:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2120000000000002e-06 2024-09-17T21:30:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707911+0000", 2024-09-17T21:30:49.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1929999999999998e-06 2024-09-17T21:30:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707915+0000", 2024-09-17T21:30:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:30:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729897+0000", 2024-09-17T21:30:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021981186 2024-09-17T21:30:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729944+0000", 2024-09-17T21:30:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571000000000002e-05 2024-09-17T21:30:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730138+0000", 2024-09-17T21:30:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000193451 2024-09-17T21:30:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45708 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:49.386 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.621594743000003, 2024-09-17T21:30:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022533742999999998, 2024-09-17T21:30:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45708 2024-09-17T21:30:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:49.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:30:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707923+0000", 2024-09-17T21:30:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:30:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707932+0000", 2024-09-17T21:30:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6430000000000007e-06 2024-09-17T21:30:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707939+0000", 2024-09-17T21:30:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3539999999999996e-06 2024-09-17T21:30:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707944+0000", 2024-09-17T21:30:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3249999999999998e-06 2024-09-17T21:30:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730176+0000", 2024-09-17T21:30:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022232393 2024-09-17T21:30:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730263+0000", 2024-09-17T21:30:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6638999999999995e-05 2024-09-17T21:30:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730456+0000", 2024-09-17T21:30:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019304899999999999 2024-09-17T21:30:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.390 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45752 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.621339523000003, 2024-09-17T21:30:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022426878000000001, 2024-09-17T21:30:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.391 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45752 2024-09-17T21:30:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:30:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708178+0000", 2024-09-17T21:30:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:30:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708181+0000", 2024-09-17T21:30:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.909e-06 2024-09-17T21:30:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708187+0000", 2024-09-17T21:30:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4999999999999996e-06 2024-09-17T21:30:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708193+0000", 2024-09-17T21:30:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:30:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730484+0000", 2024-09-17T21:30:49.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022291765000000002 2024-09-17T21:30:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730541+0000", 2024-09-17T21:30:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6483000000000001e-05 2024-09-17T21:30:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730604+0000", 2024-09-17T21:30:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3429000000000005e-05 2024-09-17T21:30:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45761 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.621311613000003, 2024-09-17T21:30:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022614408999999999, 2024-09-17T21:30:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45761 2024-09-17T21:30:49.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:30:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708206+0000", 2024-09-17T21:30:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4700000000000001e-07 2024-09-17T21:30:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708208+0000", 2024-09-17T21:30:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1509999999999998e-06 2024-09-17T21:30:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708214+0000", 2024-09-17T21:30:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.49e-06 2024-09-17T21:30:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708219+0000", 2024-09-17T21:30:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8119999999999998e-06 2024-09-17T21:30:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730641+0000", 2024-09-17T21:30:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022422765000000001 2024-09-17T21:30:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730731+0000", 2024-09-17T21:30:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9598999999999994e-05 2024-09-17T21:30:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730820+0000", 2024-09-17T21:30:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8844999999999994e-05 2024-09-17T21:30:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45785 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.620089825999997, 2024-09-17T21:30:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022044113000000001, 2024-09-17T21:30:49.400 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45785 2024-09-17T21:30:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:30:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709428+0000", 2024-09-17T21:30:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:30:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709430+0000", 2024-09-17T21:30:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:30:49.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709451+0000", 2024-09-17T21:30:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1372000000000001e-05 2024-09-17T21:30:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709472+0000", 2024-09-17T21:30:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1007999999999999e-05 2024-09-17T21:30:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731293+0000", 2024-09-17T21:30:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021821320000000002 2024-09-17T21:30:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731378+0000", 2024-09-17T21:30:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5059e-05 2024-09-17T21:30:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731471+0000", 2024-09-17T21:30:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2867000000000004e-05 2024-09-17T21:30:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.404 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45786 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:30:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.620031029000003, 2024-09-17T21:30:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022181821000000001, 2024-09-17T21:30:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.405 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45786 2024-09-17T21:30:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:30:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709487+0000", 2024-09-17T21:30:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1299999999999999e-07 2024-09-17T21:30:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709491+0000", 2024-09-17T21:30:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2089999999999999e-06 2024-09-17T21:30:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709498+0000", 2024-09-17T21:30:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3980000000000002e-06 2024-09-17T21:30:49.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709504+0000", 2024-09-17T21:30:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7219999999999996e-06 2024-09-17T21:30:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731499+0000", 2024-09-17T21:30:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021995297 2024-09-17T21:30:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731591+0000", 2024-09-17T21:30:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1373999999999996e-05 2024-09-17T21:30:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731668+0000", 2024-09-17T21:30:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7107999999999994e-05 2024-09-17T21:30:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50202 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'139, mlcod=45'137)", 2024-09-17T21:30:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:30:49.409 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.205295159999999, 2024-09-17T21:30:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059659441000000001, 2024-09-17T21:30:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:30:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:30:49.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124224+0000", 2024-09-17T21:30:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5000000000000002e-06 2024-09-17T21:30:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147385+0000", 2024-09-17T21:30:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023160544000000002 2024-09-17T21:30:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147411+0000", 2024-09-17T21:30:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6095999999999999e-05 2024-09-17T21:30:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147438+0000", 2024-09-17T21:30:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6982999999999999e-05 2024-09-17T21:30:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147504+0000", 2024-09-17T21:30:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6156999999999997e-05 2024-09-17T21:30:49.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147531+0000", 2024-09-17T21:30:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6741999999999999e-05 2024-09-17T21:30:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183820+0000", 2024-09-17T21:30:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036289090000000003 2024-09-17T21:30:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183881+0000", 2024-09-17T21:30:49.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1328999999999995e-05 2024-09-17T21:30:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50203 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'141, mlcod=45'137)", 2024-09-17T21:30:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:30:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.168032416999999, 2024-09-17T21:30:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023138823999999999, 2024-09-17T21:30:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:30:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:30:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161486+0000", 2024-09-17T21:30:49.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5290000000000001e-06 2024-09-17T21:30:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161496+0000", 2024-09-17T21:30:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0063e-05 2024-09-17T21:30:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161514+0000", 2024-09-17T21:30:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7728e-05 2024-09-17T21:30:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161524+0000", 2024-09-17T21:30:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0339e-05 2024-09-17T21:30:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170872+0000", 2024-09-17T21:30:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093473640000000004 2024-09-17T21:30:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170899+0000", 2024-09-17T21:30:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7321000000000002e-05 2024-09-17T21:30:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184580+0000", 2024-09-17T21:30:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013680988999999999 2024-09-17T21:30:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184623+0000", 2024-09-17T21:30:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3491000000000001e-05 2024-09-17T21:30:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:88193 3.9 3:97498da5:::rbd_data.1120ea662e44.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-17T21:30:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:30:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.0058515069999991, 2024-09-17T21:30:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026278261000000001, 2024-09-17T21:30:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 88193 2024-09-17T21:30:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:30:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:30:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323666+0000", 2024-09-17T21:30:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:30:49.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323670+0000", 2024-09-17T21:30:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.771e-06 2024-09-17T21:30:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323680+0000", 2024-09-17T21:30:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9469999999999998e-06 2024-09-17T21:30:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323685+0000", 2024-09-17T21:30:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9919999999999998e-06 2024-09-17T21:30:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323728+0000", 2024-09-17T21:30:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2811000000000001e-05 2024-09-17T21:30:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323776+0000", 2024-09-17T21:30:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8310000000000003e-05 2024-09-17T21:30:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.328583+0000", 2024-09-17T21:30:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048072899999999997 2024-09-17T21:30:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349861+0000", 2024-09-17T21:30:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021277695999999999 2024-09-17T21:30:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349877+0000", 2024-09-17T21:30:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:30:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349911+0000", 2024-09-17T21:30:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3844999999999999e-05 2024-09-17T21:30:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349944+0000", 2024-09-17T21:30:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2663000000000002e-05 2024-09-17T21:30:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:101384 3.9 3:90f2a32a:::rbd_data.11206b8c0882.0000000000000178:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T21:30:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:30:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.002981908, 2024-09-17T21:30:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025286194000000001, 2024-09-17T21:30:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 101384 2024-09-17T21:30:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:30:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:30:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326536+0000", 2024-09-17T21:30:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4799999999999997e-07 2024-09-17T21:30:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326541+0000", 2024-09-17T21:30:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8489999999999997e-06 2024-09-17T21:30:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326552+0000", 2024-09-17T21:30:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0912e-05 2024-09-17T21:30:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326557+0000", 2024-09-17T21:30:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0359999999999997e-06 2024-09-17T21:30:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326603+0000", 2024-09-17T21:30:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6428999999999999e-05 2024-09-17T21:30:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326686+0000", 2024-09-17T21:30:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2521999999999994e-05 2024-09-17T21:30:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.332698+0000", 2024-09-17T21:30:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060124599999999999 2024-09-17T21:30:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351744+0000", 2024-09-17T21:30:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019045838999999998 2024-09-17T21:30:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351754+0000", 2024-09-17T21:30:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0380999999999999e-05 2024-09-17T21:30:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351791+0000", 2024-09-17T21:30:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6678999999999999e-05 2024-09-17T21:30:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351821+0000", 2024-09-17T21:30:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0139000000000001e-05 2024-09-17T21:30:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.430 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:49.432 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:30:49.433 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:49.429+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:49.433 INFO:teuthology.orchestra.run.smithi129.stdout:5265 writesame 0xd9a66c4 thru 0xd9a87bf data_size 0x2(0x20fc bytes) 2024-09-17T21:30:49.433 INFO:teuthology.orchestra.run.smithi129.stdout:5267 read 0x5d12cbf thru 0x5d20d61 (0xe0a3 bytes) 2024-09-17T21:30:49.434 INFO:teuthology.orchestra.run.smithi129.stdout:5268 punch from 0xc6aeb76 to 0xc6bd368, (0xe7f2 bytes) 2024-09-17T21:30:49.434 INFO:teuthology.orchestra.run.smithi129.stdout:5269 write 0xdb6623 thru 0xdbba65 (0x5443 bytes) 2024-09-17T21:30:49.434 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:49.434 INFO:teuthology.orchestra.run.smithi129.stdout:5271 writesame 0x339299c thru 0x3398e6b data_size 0x8(0x64d0 bytes) 2024-09-17T21:30:49.434 INFO:teuthology.orchestra.run.smithi129.stdout:5272 write 0xab24a42 thru 0xab2cb84 (0x8143 bytes) 2024-09-17T21:30:49.434 INFO:teuthology.orchestra.run.smithi129.stdout:5273 write 0xd3d92cd thru 0xd3e1ad3 (0x8807 bytes) 2024-09-17T21:30:49.434 INFO:teuthology.orchestra.run.smithi129.stdout:5274 punch from 0x12b8b31 to 0x12bb73f, (0x2c0e bytes) 2024-09-17T21:30:49.434 INFO:teuthology.orchestra.run.smithi129.stdout:5275 write 0xb8e6e0b thru 0xb8e7c0f (0xe05 bytes) 2024-09-17T21:30:49.434 INFO:teuthology.orchestra.run.smithi129.stdout:5276 trunc from 0xd9a87c0 to 0xb0e689 2024-09-17T21:30:49.434 INFO:teuthology.orchestra.run.smithi129.stdout:5277 write 0xe960cbf thru 0xe967811 (0x6b53 bytes) 2024-09-17T21:30:49.438 INFO:teuthology.orchestra.run.smithi129.stdout:5279 trunc from 0xe967812 to 0xe742bdf 2024-09-17T21:30:49.447 INFO:teuthology.orchestra.run.smithi129.stdout:5281 clone 40 order 21 su 65536 sc 8 2024-09-17T21:30:49.532 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:49.530+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:49.576 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:49.576 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:49.576 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:49.576 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:49.585 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:30:49.633 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:49.631+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:49.688 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:49.688 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:49.688 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:49.688 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:49.688 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:49.698 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:30:49.734 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:49.732+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:49.803 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:49.803 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:49.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:49.803 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:49.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.804 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10799 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.804 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:30:49.804 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.734828155000002, 2024-09-17T21:30:49.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039122745, 2024-09-17T21:30:49.804 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.804 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.804 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:49.804 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.804 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.804 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.805 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10799 2024-09-17T21:30:49.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.805 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:30:49.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:30:49.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065469+0000", 2024-09-17T21:30:49.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:30:49.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065477+0000", 2024-09-17T21:30:49.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1319999999999994e-06 2024-09-17T21:30:49.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065483+0000", 2024-09-17T21:30:49.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2040000000000002e-06 2024-09-17T21:30:49.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065487+0000", 2024-09-17T21:30:49.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.42e-06 2024-09-17T21:30:49.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099984+0000", 2024-09-17T21:30:49.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034496484000000001 2024-09-17T21:30:49.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100065+0000", 2024-09-17T21:30:49.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1200999999999997e-05 2024-09-17T21:30:49.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:49.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100422+0000", 2024-09-17T21:30:49.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035657700000000001 2024-09-17T21:30:49.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:49.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104214+0000", 2024-09-17T21:30:49.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003792045 2024-09-17T21:30:49.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:49.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104505+0000", 2024-09-17T21:30:49.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029141399999999999 2024-09-17T21:30:49.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:49.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104556+0000", 2024-09-17T21:30:49.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1388999999999999e-05 2024-09-17T21:30:49.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104591+0000", 2024-09-17T21:30:49.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4455999999999997e-05 2024-09-17T21:30:49.811 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.811 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.811 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.811 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12187 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.811 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:49.811 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.660735424000002, 2024-09-17T21:30:49.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038825750999999999, 2024-09-17T21:30:49.812 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.812 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.812 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.812 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.812 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.812 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.812 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12187 2024-09-17T21:30:49.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.812 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:49.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:49.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:49.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:30:49.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139563+0000", 2024-09-17T21:30:49.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.624e-06 2024-09-17T21:30:49.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139569+0000", 2024-09-17T21:30:49.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1090000000000003e-06 2024-09-17T21:30:49.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139573+0000", 2024-09-17T21:30:49.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4429999999999996e-06 2024-09-17T21:30:49.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178206+0000", 2024-09-17T21:30:49.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038632376000000003 2024-09-17T21:30:49.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178282+0000", 2024-09-17T21:30:49.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6076999999999997e-05 2024-09-17T21:30:49.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178387+0000", 2024-09-17T21:30:49.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104682 2024-09-17T21:30:49.815 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.815 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.816 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.816 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12188 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.816 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:30:49.816 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.660717243000001, 2024-09-17T21:30:49.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039053826999999999, 2024-09-17T21:30:49.816 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.816 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.816 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.816 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.816 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.816 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.816 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12188 2024-09-17T21:30:49.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.817 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:30:49.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:30:49.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139746+0000", 2024-09-17T21:30:49.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000167281 2024-09-17T21:30:49.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139749+0000", 2024-09-17T21:30:49.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9660000000000001e-06 2024-09-17T21:30:49.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139757+0000", 2024-09-17T21:30:49.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8029999999999997e-06 2024-09-17T21:30:49.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139771+0000", 2024-09-17T21:30:49.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3541e-05 2024-09-17T21:30:49.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178415+0000", 2024-09-17T21:30:49.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038644803999999998 2024-09-17T21:30:49.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178517+0000", 2024-09-17T21:30:49.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101298 2024-09-17T21:30:49.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178633+0000", 2024-09-17T21:30:49.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116134 2024-09-17T21:30:49.820 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.820 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.820 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.820 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12189 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.820 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:30:49.820 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.660516766000001, 2024-09-17T21:30:49.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039135991000000002, 2024-09-17T21:30:49.820 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.821 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.821 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.821 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.821 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.821 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.821 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12189 2024-09-17T21:30:49.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.821 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:30:49.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:30:49.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139780+0000", 2024-09-17T21:30:49.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:30:49.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139782+0000", 2024-09-17T21:30:49.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1950000000000002e-06 2024-09-17T21:30:49.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139788+0000", 2024-09-17T21:30:49.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0639999999999999e-06 2024-09-17T21:30:49.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139792+0000", 2024-09-17T21:30:49.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1960000000000001e-06 2024-09-17T21:30:49.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178671+0000", 2024-09-17T21:30:49.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038878176 2024-09-17T21:30:49.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178778+0000", 2024-09-17T21:30:49.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107278 2024-09-17T21:30:49.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178915+0000", 2024-09-17T21:30:49.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013757900000000001 2024-09-17T21:30:49.825 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.825 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.825 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.825 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12190 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.825 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:49.825 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.660497583000001, 2024-09-17T21:30:49.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039343781000000001, 2024-09-17T21:30:49.825 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.826 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.826 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.826 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.826 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.826 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.826 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12190 2024-09-17T21:30:49.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.826 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:49.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:49.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:49.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:49.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139801+0000", 2024-09-17T21:30:49.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.916e-06 2024-09-17T21:30:49.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139807+0000", 2024-09-17T21:30:49.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4540000000000004e-06 2024-09-17T21:30:49.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139812+0000", 2024-09-17T21:30:49.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9729999999999997e-06 2024-09-17T21:30:49.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178960+0000", 2024-09-17T21:30:49.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039147567000000001 2024-09-17T21:30:49.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179051+0000", 2024-09-17T21:30:49.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0754000000000006e-05 2024-09-17T21:30:49.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179142+0000", 2024-09-17T21:30:49.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1712999999999994e-05 2024-09-17T21:30:49.830 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.830 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.830 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.831 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12191 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.831 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:49.831 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.660477725, 2024-09-17T21:30:49.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039464336000000003, 2024-09-17T21:30:49.831 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.831 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.831 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.831 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12191 2024-09-17T21:30:49.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.832 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:49.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:49.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:49.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:30:49.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139821+0000", 2024-09-17T21:30:49.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1260000000000001e-06 2024-09-17T21:30:49.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139827+0000", 2024-09-17T21:30:49.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T21:30:49.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139840+0000", 2024-09-17T21:30:49.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2607e-05 2024-09-17T21:30:49.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179162+0000", 2024-09-17T21:30:49.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039322863999999999 2024-09-17T21:30:49.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179204+0000", 2024-09-17T21:30:49.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1938999999999999e-05 2024-09-17T21:30:49.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179283+0000", 2024-09-17T21:30:49.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8547000000000005e-05 2024-09-17T21:30:49.836 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.836 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.836 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.836 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12192 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.836 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:30:49.836 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.660450013000002, 2024-09-17T21:30:49.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039652318999999998, 2024-09-17T21:30:49.836 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.836 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.837 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.837 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.837 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.837 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.837 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12192 2024-09-17T21:30:49.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.837 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:30:49.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:30:49.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141458+0000", 2024-09-17T21:30:49.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001612098 2024-09-17T21:30:49.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141469+0000", 2024-09-17T21:30:49.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-17T21:30:49.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141478+0000", 2024-09-17T21:30:49.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T21:30:49.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141485+0000", 2024-09-17T21:30:49.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4760000000000003e-06 2024-09-17T21:30:49.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179317+0000", 2024-09-17T21:30:49.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037831832000000003 2024-09-17T21:30:49.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179395+0000", 2024-09-17T21:30:49.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8349000000000005e-05 2024-09-17T21:30:49.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179499+0000", 2024-09-17T21:30:49.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103628 2024-09-17T21:30:49.840 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.840 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.840 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.841 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12257 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.841 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:49.841 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.658261205000002, 2024-09-17T21:30:49.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038862751000000001, 2024-09-17T21:30:49.841 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.841 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.841 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.841 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12257 2024-09-17T21:30:49.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.842 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:49.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:49.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:49.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T21:30:49.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142037+0000", 2024-09-17T21:30:49.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.03e-06 2024-09-17T21:30:49.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142043+0000", 2024-09-17T21:30:49.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6319999999999996e-06 2024-09-17T21:30:49.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142052+0000", 2024-09-17T21:30:49.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2709999999999998e-06 2024-09-17T21:30:49.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180794+0000", 2024-09-17T21:30:49.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038741758000000001 2024-09-17T21:30:49.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180843+0000", 2024-09-17T21:30:49.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9021999999999999e-05 2024-09-17T21:30:49.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180898+0000", 2024-09-17T21:30:49.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673000000000003e-05 2024-09-17T21:30:49.845 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.845 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.845 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.845 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12258 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.845 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:49.845 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.658237688, 2024-09-17T21:30:49.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038950717000000003, 2024-09-17T21:30:49.845 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.845 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.845 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.846 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.846 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.846 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12258 2024-09-17T21:30:49.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.846 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:49.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:49.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:49.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199999999999999e-07 2024-09-17T21:30:49.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142061+0000", 2024-09-17T21:30:49.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8279999999999999e-06 2024-09-17T21:30:49.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142067+0000", 2024-09-17T21:30:49.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5600000000000001e-06 2024-09-17T21:30:49.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142071+0000", 2024-09-17T21:30:49.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6240000000000001e-06 2024-09-17T21:30:49.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180921+0000", 2024-09-17T21:30:49.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038849366000000003 2024-09-17T21:30:49.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180961+0000", 2024-09-17T21:30:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0219000000000002e-05 2024-09-17T21:30:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181009+0000", 2024-09-17T21:30:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8437999999999999e-05 2024-09-17T21:30:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12259 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.658210441000001, 2024-09-17T21:30:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039116832999999997, 2024-09-17T21:30:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12259 2024-09-17T21:30:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2200000000000004e-07 2024-09-17T21:30:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142088+0000", 2024-09-17T21:30:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0329999999999998e-06 2024-09-17T21:30:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142095+0000", 2024-09-17T21:30:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4320000000000004e-06 2024-09-17T21:30:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142100+0000", 2024-09-17T21:30:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:30:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181037+0000", 2024-09-17T21:30:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038937601000000002 2024-09-17T21:30:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181114+0000", 2024-09-17T21:30:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6857e-05 2024-09-17T21:30:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181203+0000", 2024-09-17T21:30:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8437999999999996e-05 2024-09-17T21:30:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12260 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:30:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.658190154000003, 2024-09-17T21:30:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039204184000000003, 2024-09-17T21:30:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12260 2024-09-17T21:30:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:30:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:30:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142107+0000", 2024-09-17T21:30:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:30:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142108+0000", 2024-09-17T21:30:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.753e-06 2024-09-17T21:30:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142114+0000", 2024-09-17T21:30:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7660000000000004e-06 2024-09-17T21:30:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142119+0000", 2024-09-17T21:30:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:30:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181216+0000", 2024-09-17T21:30:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039097191000000003 2024-09-17T21:30:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181260+0000", 2024-09-17T21:30:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4731000000000003e-05 2024-09-17T21:30:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181310+0000", 2024-09-17T21:30:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9818000000000003e-05 2024-09-17T21:30:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12261 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:30:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.658171830000001, 2024-09-17T21:30:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039292200999999999, 2024-09-17T21:30:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12261 2024-09-17T21:30:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:30:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:30:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142125+0000", 2024-09-17T21:30:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:30:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142127+0000", 2024-09-17T21:30:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.792e-06 2024-09-17T21:30:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142132+0000", 2024-09-17T21:30:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3979999999999998e-06 2024-09-17T21:30:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142136+0000", 2024-09-17T21:30:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.211e-06 2024-09-17T21:30:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181324+0000", 2024-09-17T21:30:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039187560000000003 2024-09-17T21:30:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181363+0000", 2024-09-17T21:30:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9375000000000002e-05 2024-09-17T21:30:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181417+0000", 2024-09-17T21:30:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3372000000000003e-05 2024-09-17T21:30:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12262 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:30:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.658154037000003, 2024-09-17T21:30:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039384875, 2024-09-17T21:30:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12262 2024-09-17T21:30:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:30:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:30:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142143+0000", 2024-09-17T21:30:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:30:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142148+0000", 2024-09-17T21:30:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4360000000000001e-06 2024-09-17T21:30:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142154+0000", 2024-09-17T21:30:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8719999999999999e-06 2024-09-17T21:30:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142158+0000", 2024-09-17T21:30:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.476e-06 2024-09-17T21:30:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181436+0000", 2024-09-17T21:30:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039277919000000001 2024-09-17T21:30:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181480+0000", 2024-09-17T21:30:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3649999999999997e-05 2024-09-17T21:30:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181527+0000", 2024-09-17T21:30:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7131e-05 2024-09-17T21:30:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12263 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1079296~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.658131601999997, 2024-09-17T21:30:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039473339000000003, 2024-09-17T21:30:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12263 2024-09-17T21:30:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:30:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142167+0000", 2024-09-17T21:30:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.81e-06 2024-09-17T21:30:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142172+0000", 2024-09-17T21:30:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:30:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142177+0000", 2024-09-17T21:30:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T21:30:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181553+0000", 2024-09-17T21:30:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039376130000000002 2024-09-17T21:30:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181592+0000", 2024-09-17T21:30:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9153999999999998e-05 2024-09-17T21:30:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181638+0000", 2024-09-17T21:30:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5864e-05 2024-09-17T21:30:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12278 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1095680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.658113626999999, 2024-09-17T21:30:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039454222999999997, 2024-09-17T21:30:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12278 2024-09-17T21:30:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:30:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142185+0000", 2024-09-17T21:30:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.697e-06 2024-09-17T21:30:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142191+0000", 2024-09-17T21:30:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:30:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142196+0000", 2024-09-17T21:30:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4829999999999999e-06 2024-09-17T21:30:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181548+0000", 2024-09-17T21:30:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039351200000000003 2024-09-17T21:30:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181591+0000", 2024-09-17T21:30:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3492000000000003e-05 2024-09-17T21:30:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181637+0000", 2024-09-17T21:30:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5847000000000003e-05 2024-09-17T21:30:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12279 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.658090338000001, 2024-09-17T21:30:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039539801999999999, 2024-09-17T21:30:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12279 2024-09-17T21:30:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.89e-07 2024-09-17T21:30:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142208+0000", 2024-09-17T21:30:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7579999999999999e-06 2024-09-17T21:30:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142214+0000", 2024-09-17T21:30:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:30:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142218+0000", 2024-09-17T21:30:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3429999999999998e-06 2024-09-17T21:30:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181653+0000", 2024-09-17T21:30:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039435322000000002 2024-09-17T21:30:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181698+0000", 2024-09-17T21:30:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4978999999999997e-05 2024-09-17T21:30:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181746+0000", 2024-09-17T21:30:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571999999999998e-05 2024-09-17T21:30:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12280 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1124352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:30:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.658072818999997, 2024-09-17T21:30:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039704358000000002, 2024-09-17T21:30:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12280 2024-09-17T21:30:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:30:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:30:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142224+0000", 2024-09-17T21:30:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:30:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142226+0000", 2024-09-17T21:30:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:30:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142231+0000", 2024-09-17T21:30:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:30:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142240+0000", 2024-09-17T21:30:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4590000000000004e-06 2024-09-17T21:30:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181774+0000", 2024-09-17T21:30:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039533833999999997 2024-09-17T21:30:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181859+0000", 2024-09-17T21:30:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4328999999999998e-05 2024-09-17T21:30:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181928+0000", 2024-09-17T21:30:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9139000000000003e-05 2024-09-17T21:30:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12281 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1136640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:30:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.658049912999999, 2024-09-17T21:30:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039787799999999998, 2024-09-17T21:30:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12281 2024-09-17T21:30:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:30:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:30:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142309+0000", 2024-09-17T21:30:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2323999999999995e-05 2024-09-17T21:30:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142317+0000", 2024-09-17T21:30:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1859999999999992e-06 2024-09-17T21:30:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142324+0000", 2024-09-17T21:30:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4699999999999996e-06 2024-09-17T21:30:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142330+0000", 2024-09-17T21:30:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4650000000000002e-06 2024-09-17T21:30:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181948+0000", 2024-09-17T21:30:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039618056999999998 2024-09-17T21:30:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181986+0000", 2024-09-17T21:30:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8442999999999998e-05 2024-09-17T21:30:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182034+0000", 2024-09-17T21:30:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7855000000000002e-05 2024-09-17T21:30:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12282 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1150976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.657957684000003, 2024-09-17T21:30:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039871919999999998, 2024-09-17T21:30:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12282 2024-09-17T21:30:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:30:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142341+0000", 2024-09-17T21:30:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.232e-06 2024-09-17T21:30:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142348+0000", 2024-09-17T21:30:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7320000000000001e-06 2024-09-17T21:30:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142353+0000", 2024-09-17T21:30:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:30:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182062+0000", 2024-09-17T21:30:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039709739000000001 2024-09-17T21:30:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182128+0000", 2024-09-17T21:30:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5739000000000002e-05 2024-09-17T21:30:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182210+0000", 2024-09-17T21:30:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:30:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12283 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:30:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.657937273000002, 2024-09-17T21:30:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039896434000000001, 2024-09-17T21:30:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:30:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12283 2024-09-17T21:30:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:30:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:30:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144852+0000", 2024-09-17T21:30:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024934079999999999 2024-09-17T21:30:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144856+0000", 2024-09-17T21:30:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:30:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144864+0000", 2024-09-17T21:30:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0160000000000008e-06 2024-09-17T21:30:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144869+0000", 2024-09-17T21:30:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:30:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182246+0000", 2024-09-17T21:30:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037377207000000003 2024-09-17T21:30:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182255+0000", 2024-09-17T21:30:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1840000000000005e-06 2024-09-17T21:30:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:49.899 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:49.901 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.7 to 1 (10000) 2024-09-17T21:30:49.901 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is unset 2024-09-17T21:30:49.902 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:30:49.903 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:49.898+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:49.903 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T21:30:49.903 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:30:50.002 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:49.999+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:50.046 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:50.046 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:50.047 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:50.047 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:50.055 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:30:50.090 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-17T21:30:50.103 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:50.100+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:50.204 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:50.201+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:50.204 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:50.204 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:50.204 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:50.204 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:50.204 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:50.213 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:30:50.266 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:30:50.267 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-17T21:30:50.282 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-17T21:30:49.869876+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1296199289,"num_objects":2096,"num_object_clones":521,"num_object_copies":4192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2096,"num_whiteouts":1017,"num_read":131294,"num_read_kb":763687,"num_write":16531,"num_write_kb":378041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":47148489,"num_keys_recovered":137,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13092,"ondisk_log_size":13092,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":310740,"kb_used_data":68636,"kb_used_omap":29,"kb_used_meta":241954,"kb_avail":660292140,"statfs":{"total":676457349120,"available":676139151360,"internally_reserved":0,"allocated":70283264,"data_stored":606127189,"data_compressed":15285245,"data_compressed_allocated":47529984,"data_compressed_original":587415569,"omap_allocated":30485,"internal_metadata":247761131},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70222106,"num_objects":335,"num_object_clones":129,"num_object_copies":670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":149,"num_read":12694,"num_read_kb":70935,"num_write":1747,"num_write_kb":30916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.033990"},"pg_stats":[{"pgid":"3.1f","version":"79'570","reported_seq":3659,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T21:30:48.867619+0000","last_change":"2024-09-17T21:29:59.559639+0000","last_active":"2024-09-17T21:30:48.867619+0000","last_peered":"2024-09-17T21:30:48.867619+0000","last_clean":"2024-09-17T21:30:48.867619+0000","last_became_active":"2024-09-17T21:29:59.541221+0000","last_became_peered":"2024-09-17T21:29:59.541221+0000","last_unstale":"2024-09-17T21:30:48.867619+0000","last_undegraded":"2024-09-17T21:30:48.867619+0000","last_fullsized":"2024-09-17T21:30:48.867619+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:59:52.686527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011306689999999999,"stat_sum":{"num_bytes":56214113,"num_objects":78,"num_object_clones":22,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":39,"num_read":4766,"num_read_kb":61233,"num_write":961,"num_write_kb":32360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":67819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"76'374","reported_seq":3170,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.837716+0000","last_change":"2024-09-17T21:29:43.473228+0000","last_active":"2024-09-17T21:30:45.837716+0000","last_peered":"2024-09-17T21:30:45.837716+0000","last_clean":"2024-09-17T21:30:45.837716+0000","last_became_active":"2024-09-17T21:29:41.472161+0000","last_became_peered":"2024-09-17T21:29:41.472161+0000","last_unstale":"2024-09-17T21:30:45.837716+0000","last_undegraded":"2024-09-17T21:30:45.837716+0000","last_fullsized":"2024-09-17T21:30:45.837716+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:08:49.455683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092644099999999996,"stat_sum":{"num_bytes":53059342,"num_objects":57,"num_object_clones":11,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":27,"num_read":3114,"num_read_kb":22225,"num_write":376,"num_write_kb":9068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"77'491","reported_seq":4170,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.865209+0000","last_change":"2024-09-17T21:30:02.564806+0000","last_active":"2024-09-17T21:30:45.865209+0000","last_peered":"2024-09-17T21:30:45.865209+0000","last_clean":"2024-09-17T21:30:45.865209+0000","last_became_active":"2024-09-17T21:30:02.564199+0000","last_became_peered":"2024-09-17T21:30:02.564199+0000","last_unstale":"2024-09-17T21:30:45.865209+0000","last_undegraded":"2024-09-17T21:30:45.865209+0000","last_fullsized":"2024-09-17T21:30:45.865209+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:38:51.442823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063012799999999998,"stat_sum":{"num_bytes":79682618,"num_objects":66,"num_object_clones":16,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":27,"num_read":5177,"num_read_kb":44968,"num_write":589,"num_write_kb":16774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":573415,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"77'268","reported_seq":1507,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.876613+0000","last_change":"2024-09-17T21:30:28.802285+0000","last_active":"2024-09-17T21:30:45.876613+0000","last_peered":"2024-09-17T21:30:45.876613+0000","last_clean":"2024-09-17T21:30:45.876613+0000","last_became_active":"2024-09-17T21:30:28.747704+0000","last_became_peered":"2024-09-17T21:30:28.747704+0000","last_unstale":"2024-09-17T21:30:45.876613+0000","last_undegraded":"2024-09-17T21:30:45.876613+0000","last_fullsized":"2024-09-17T21:30:45.876613+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:04:21.389918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047754999999999998,"stat_sum":{"num_bytes":29484789,"num_objects":62,"num_object_clones":12,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":37,"num_read":1804,"num_read_kb":7115,"num_write":257,"num_write_kb":3676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":259277,"num_keys_recovered":68,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"79'438","reported_seq":7809,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T21:30:48.867469+0000","last_change":"2024-09-17T21:29:43.486642+0000","last_active":"2024-09-17T21:30:48.867469+0000","last_peered":"2024-09-17T21:30:48.867469+0000","last_clean":"2024-09-17T21:30:48.867469+0000","last_became_active":"2024-09-17T21:29:41.476041+0000","last_became_peered":"2024-09-17T21:29:41.476041+0000","last_unstale":"2024-09-17T21:30:48.867469+0000","last_undegraded":"2024-09-17T21:30:48.867469+0000","last_fullsized":"2024-09-17T21:30:48.867469+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:29:21.410196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086460200000000001,"stat_sum":{"num_bytes":62753802,"num_objects":81,"num_object_clones":25,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":34,"num_read":8368,"num_read_kb":52559,"num_write":480,"num_write_kb":20845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"77'415","reported_seq":4858,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.911004+0000","last_change":"2024-09-17T21:30:28.776386+0000","last_active":"2024-09-17T21:30:45.911004+0000","last_peered":"2024-09-17T21:30:45.911004+0000","last_clean":"2024-09-17T21:30:45.911004+0000","last_became_active":"2024-09-17T21:30:28.747031+0000","last_became_peered":"2024-09-17T21:30:28.747031+0000","last_unstale":"2024-09-17T21:30:45.911004+0000","last_undegraded":"2024-09-17T21:30:45.911004+0000","last_fullsized":"2024-09-17T21:30:45.911004+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:31:25.029342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075875499999999995,"stat_sum":{"num_bytes":36387548,"num_objects":68,"num_object_clones":14,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":33,"num_read":5126,"num_read_kb":18395,"num_write":508,"num_write_kb":5200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":235969,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"76'305","reported_seq":4218,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.842930+0000","last_change":"2024-09-17T21:30:28.887353+0000","last_active":"2024-09-17T21:30:45.842930+0000","last_peered":"2024-09-17T21:30:45.842930+0000","last_clean":"2024-09-17T21:30:45.842930+0000","last_became_active":"2024-09-17T21:30:28.745634+0000","last_became_peered":"2024-09-17T21:30:28.745634+0000","last_unstale":"2024-09-17T21:30:45.842930+0000","last_undegraded":"2024-09-17T21:30:45.842930+0000","last_fullsized":"2024-09-17T21:30:45.842930+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:54:18.766742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048345199999999999,"stat_sum":{"num_bytes":85276056,"num_objects":78,"num_object_clones":19,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":36,"num_read":4467,"num_read_kb":56590,"num_write":348,"num_write_kb":40933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":36017677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"76'351","reported_seq":3664,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.837182+0000","last_change":"2024-09-17T21:29:43.472344+0000","last_active":"2024-09-17T21:30:45.837182+0000","last_peered":"2024-09-17T21:30:45.837182+0000","last_clean":"2024-09-17T21:30:45.837182+0000","last_became_active":"2024-09-17T21:29:41.472468+0000","last_became_peered":"2024-09-17T21:29:41.472468+0000","last_unstale":"2024-09-17T21:30:45.837182+0000","last_undegraded":"2024-09-17T21:30:45.837182+0000","last_fullsized":"2024-09-17T21:30:45.837182+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:26:28.798879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061906600000000004,"stat_sum":{"num_bytes":38493140,"num_objects":72,"num_object_clones":20,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":25,"num_read":4251,"num_read_kb":22281,"num_write":391,"num_write_kb":11619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"79'505","reported_seq":2142,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T21:30:48.867810+0000","last_change":"2024-09-17T21:30:02.561153+0000","last_active":"2024-09-17T21:30:48.867810+0000","last_peered":"2024-09-17T21:30:48.867810+0000","last_clean":"2024-09-17T21:30:48.867810+0000","last_became_active":"2024-09-17T21:30:02.560707+0000","last_became_peered":"2024-09-17T21:30:02.560707+0000","last_unstale":"2024-09-17T21:30:48.867810+0000","last_undegraded":"2024-09-17T21:30:48.867810+0000","last_fullsized":"2024-09-17T21:30:48.867810+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:23:34.347190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173896,"stat_sum":{"num_bytes":54004323,"num_objects":73,"num_object_clones":18,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":32,"num_read":3607,"num_read_kb":21563,"num_write":574,"num_write_kb":29616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":602801,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"75'936","reported_seq":2208,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.834533+0000","last_change":"2024-09-17T21:30:31.755892+0000","last_active":"2024-09-17T21:30:45.834533+0000","last_peered":"2024-09-17T21:30:45.834533+0000","last_clean":"2024-09-17T21:30:45.834533+0000","last_became_active":"2024-09-17T21:30:31.754854+0000","last_became_peered":"2024-09-17T21:30:31.754854+0000","last_unstale":"2024-09-17T21:30:45.834533+0000","last_undegraded":"2024-09-17T21:30:45.834533+0000","last_fullsized":"2024-09-17T21:30:45.834533+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:00:03.734610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072238100000000002,"stat_sum":{"num_bytes":35657208,"num_objects":56,"num_object_clones":10,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":29,"num_read":5144,"num_read_kb":10502,"num_write":1517,"num_write_kb":7154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":1297843,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"76'333","reported_seq":2198,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.872521+0000","last_change":"2024-09-17T21:29:43.564521+0000","last_active":"2024-09-17T21:30:45.872521+0000","last_peered":"2024-09-17T21:30:45.872521+0000","last_clean":"2024-09-17T21:30:45.872521+0000","last_became_active":"2024-09-17T21:29:41.478346+0000","last_became_peered":"2024-09-17T21:29:41.478346+0000","last_unstale":"2024-09-17T21:30:45.872521+0000","last_undegraded":"2024-09-17T21:30:45.872521+0000","last_fullsized":"2024-09-17T21:30:45.872521+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:18:35.993942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069290300000000003,"stat_sum":{"num_bytes":34435185,"num_objects":66,"num_object_clones":14,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":40,"num_read":2733,"num_read_kb":4338,"num_write":306,"num_write_kb":3439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"76'212","reported_seq":2181,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.840128+0000","last_change":"2024-09-17T21:29:43.477145+0000","last_active":"2024-09-17T21:30:45.840128+0000","last_peered":"2024-09-17T21:30:45.840128+0000","last_clean":"2024-09-17T21:30:45.840128+0000","last_became_active":"2024-09-17T21:29:41.477611+0000","last_became_peered":"2024-09-17T21:29:41.477611+0000","last_unstale":"2024-09-17T21:30:45.840128+0000","last_undegraded":"2024-09-17T21:30:45.840128+0000","last_fullsized":"2024-09-17T21:30:45.840128+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:29:58.429084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073991200000000001,"stat_sum":{"num_bytes":27813108,"num_objects":64,"num_object_clones":20,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":35,"num_read":1903,"num_read_kb":9197,"num_write":296,"num_write_kb":11138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":128,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.882316+0000","last_change":"2024-09-17T21:29:35.002656+0000","last_active":"2024-09-17T21:30:45.882316+0000","last_peered":"2024-09-17T21:30:45.882316+0000","last_clean":"2024-09-17T21:30:45.882316+0000","last_became_active":"2024-09-17T21:29:32.971677+0000","last_became_peered":"2024-09-17T21:29:32.971677+0000","last_unstale":"2024-09-17T21:30:45.882316+0000","last_undegraded":"2024-09-17T21:30:45.882316+0000","last_fullsized":"2024-09-17T21:30:45.882316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:53.781657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041121300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"79'603","reported_seq":3310,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T21:30:48.867270+0000","last_change":"2024-09-17T21:30:31.755784+0000","last_active":"2024-09-17T21:30:48.867270+0000","last_peered":"2024-09-17T21:30:48.867270+0000","last_clean":"2024-09-17T21:30:48.867270+0000","last_became_active":"2024-09-17T21:30:31.755039+0000","last_became_peered":"2024-09-17T21:30:31.755039+0000","last_unstale":"2024-09-17T21:30:48.867270+0000","last_undegraded":"2024-09-17T21:30:48.867270+0000","last_fullsized":"2024-09-17T21:30:48.867270+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:14:17.603360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010838759999999999,"stat_sum":{"num_bytes":26661270,"num_objects":57,"num_object_clones":16,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":27,"num_read":5718,"num_read_kb":15265,"num_write":816,"num_write_kb":8008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":448871,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.841894+0000","last_change":"2024-09-17T21:30:28.745759+0000","last_active":"2024-09-17T21:30:45.841894+0000","last_peered":"2024-09-17T21:30:45.841894+0000","last_clean":"2024-09-17T21:30:45.841894+0000","last_became_active":"2024-09-17T21:30:28.744742+0000","last_became_peered":"2024-09-17T21:30:28.744742+0000","last_unstale":"2024-09-17T21:30:45.841894+0000","last_undegraded":"2024-09-17T21:30:45.841894+0000","last_fullsized":"2024-09-17T21:30:45.841894+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:37:02.014566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"76'434","reported_seq":1968,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.837344+0000","last_change":"2024-09-17T21:29:43.473294+0000","last_active":"2024-09-17T21:30:45.837344+0000","last_peered":"2024-09-17T21:30:45.837344+0000","last_clean":"2024-09-17T21:30:45.837344+0000","last_became_active":"2024-09-17T21:29:41.472095+0000","last_became_peered":"2024-09-17T21:29:41.472095+0000","last_unstale":"2024-09-17T21:30:45.837344+0000","last_undegraded":"2024-09-17T21:30:45.837344+0000","last_fullsized":"2024-09-17T21:30:45.837344+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:27:04.301133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074063099999999995,"stat_sum":{"num_bytes":14765697,"num_objects":70,"num_object_clones":19,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":33,"num_read":3228,"num_read_kb":8423,"num_write":603,"num_write_kb":4107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.834644+0000","last_change":"2024-09-17T21:29:35.078854+0000","last_active":"2024-09-17T21:30:45.834644+0000","last_peered":"2024-09-17T21:30:45.834644+0000","last_clean":"2024-09-17T21:30:45.834644+0000","last_became_active":"2024-09-17T21:29:33.071665+0000","last_became_peered":"2024-09-17T21:29:33.071665+0000","last_unstale":"2024-09-17T21:30:45.834644+0000","last_undegraded":"2024-09-17T21:30:45.834644+0000","last_fullsized":"2024-09-17T21:30:45.834644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.415687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047969899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"77'647","reported_seq":4071,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:46.518995+0000","last_change":"2024-09-17T21:30:31.754078+0000","last_active":"2024-09-17T21:30:46.518995+0000","last_peered":"2024-09-17T21:30:46.518995+0000","last_clean":"2024-09-17T21:30:46.518995+0000","last_became_active":"2024-09-17T21:30:31.752750+0000","last_became_peered":"2024-09-17T21:30:31.752750+0000","last_unstale":"2024-09-17T21:30:46.518995+0000","last_undegraded":"2024-09-17T21:30:46.518995+0000","last_fullsized":"2024-09-17T21:30:46.518995+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:02:44.288559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045510199999999998,"stat_sum":{"num_bytes":28784662,"num_objects":73,"num_object_clones":17,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":38,"num_read":5752,"num_read_kb":25799,"num_write":930,"num_write_kb":12035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3231045,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.842066+0000","last_change":"2024-09-17T21:30:28.748319+0000","last_active":"2024-09-17T21:30:45.842066+0000","last_peered":"2024-09-17T21:30:45.842066+0000","last_clean":"2024-09-17T21:30:45.842066+0000","last_became_active":"2024-09-17T21:30:28.747728+0000","last_became_peered":"2024-09-17T21:30:28.747728+0000","last_unstale":"2024-09-17T21:30:45.842066+0000","last_undegraded":"2024-09-17T21:30:45.842066+0000","last_fullsized":"2024-09-17T21:30:45.842066+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:58:18.954005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"76'210","reported_seq":3834,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.837719+0000","last_change":"2024-09-17T21:29:59.583758+0000","last_active":"2024-09-17T21:30:45.837719+0000","last_peered":"2024-09-17T21:30:45.837719+0000","last_clean":"2024-09-17T21:30:45.837719+0000","last_became_active":"2024-09-17T21:29:59.542323+0000","last_became_peered":"2024-09-17T21:29:59.542323+0000","last_unstale":"2024-09-17T21:30:45.837719+0000","last_undegraded":"2024-09-17T21:30:45.837719+0000","last_fullsized":"2024-09-17T21:30:45.837719+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:19:35.767020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074772900000000004,"stat_sum":{"num_bytes":28247651,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":29,"num_read":3747,"num_read_kb":26934,"num_write":202,"num_write_kb":6968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":541352,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"76'486","reported_seq":3588,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.840411+0000","last_change":"2024-09-17T21:29:43.477334+0000","last_active":"2024-09-17T21:30:45.840411+0000","last_peered":"2024-09-17T21:30:45.840411+0000","last_clean":"2024-09-17T21:30:45.840411+0000","last_became_active":"2024-09-17T21:29:41.475273+0000","last_became_peered":"2024-09-17T21:29:41.475273+0000","last_unstale":"2024-09-17T21:30:45.840411+0000","last_undegraded":"2024-09-17T21:30:45.840411+0000","last_fullsized":"2024-09-17T21:30:45.840411+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:27:18.423795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083339600000000003,"stat_sum":{"num_bytes":48620953,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":23,"num_read":4949,"num_read_kb":12938,"num_write":666,"num_write_kb":4910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":72,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.841873+0000","last_change":"2024-09-17T21:30:28.744890+0000","last_active":"2024-09-17T21:30:45.841873+0000","last_peered":"2024-09-17T21:30:45.841873+0000","last_clean":"2024-09-17T21:30:45.841873+0000","last_became_active":"2024-09-17T21:30:28.744567+0000","last_became_peered":"2024-09-17T21:30:28.744567+0000","last_unstale":"2024-09-17T21:30:45.841873+0000","last_undegraded":"2024-09-17T21:30:45.841873+0000","last_fullsized":"2024-09-17T21:30:45.841873+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:52:21.176000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"77'298","reported_seq":2370,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:46.111237+0000","last_change":"2024-09-17T21:30:28.895927+0000","last_active":"2024-09-17T21:30:46.111237+0000","last_peered":"2024-09-17T21:30:46.111237+0000","last_clean":"2024-09-17T21:30:46.111237+0000","last_became_active":"2024-09-17T21:30:28.746690+0000","last_became_peered":"2024-09-17T21:30:28.746690+0000","last_unstale":"2024-09-17T21:30:46.111237+0000","last_undegraded":"2024-09-17T21:30:46.111237+0000","last_fullsized":"2024-09-17T21:30:46.111237+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:40:11.028322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060580400000000004,"stat_sum":{"num_bytes":18342406,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":29,"num_read":3104,"num_read_kb":11229,"num_write":312,"num_write_kb":5281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1513,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":128,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.837468+0000","last_change":"2024-09-17T21:29:34.981930+0000","last_active":"2024-09-17T21:30:45.837468+0000","last_peered":"2024-09-17T21:30:45.837468+0000","last_clean":"2024-09-17T21:30:45.837468+0000","last_became_active":"2024-09-17T21:29:33.071313+0000","last_became_peered":"2024-09-17T21:29:33.071313+0000","last_unstale":"2024-09-17T21:30:45.837468+0000","last_undegraded":"2024-09-17T21:30:45.837468+0000","last_fullsized":"2024-09-17T21:30:45.837468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:11:51.460374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000527295,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"78'304","reported_seq":3069,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T21:30:48.867506+0000","last_change":"2024-09-17T21:29:43.486638+0000","last_active":"2024-09-17T21:30:48.867506+0000","last_peered":"2024-09-17T21:30:48.867506+0000","last_clean":"2024-09-17T21:30:48.867506+0000","last_became_active":"2024-09-17T21:29:41.473964+0000","last_became_peered":"2024-09-17T21:29:41.473964+0000","last_unstale":"2024-09-17T21:30:48.867506+0000","last_undegraded":"2024-09-17T21:30:48.867506+0000","last_fullsized":"2024-09-17T21:30:48.867506+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:05:03.507218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087607199999999998,"stat_sum":{"num_bytes":68038985,"num_objects":70,"num_object_clones":17,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":36,"num_read":2936,"num_read_kb":57400,"num_write":493,"num_write_kb":32986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":137,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.834356+0000","last_change":"2024-09-17T21:30:28.746557+0000","last_active":"2024-09-17T21:30:45.834356+0000","last_peered":"2024-09-17T21:30:45.834356+0000","last_clean":"2024-09-17T21:30:45.834356+0000","last_became_active":"2024-09-17T21:30:28.746261+0000","last_became_peered":"2024-09-17T21:30:28.746261+0000","last_unstale":"2024-09-17T21:30:45.834356+0000","last_undegraded":"2024-09-17T21:30:45.834356+0000","last_fullsized":"2024-09-17T21:30:45.834356+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:42.591689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044115199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"76'206","reported_seq":3034,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.837623+0000","last_change":"2024-09-17T21:29:59.600040+0000","last_active":"2024-09-17T21:30:45.837623+0000","last_peered":"2024-09-17T21:30:45.837623+0000","last_clean":"2024-09-17T21:30:45.837623+0000","last_became_active":"2024-09-17T21:29:59.544470+0000","last_became_peered":"2024-09-17T21:29:59.544470+0000","last_unstale":"2024-09-17T21:30:45.837623+0000","last_undegraded":"2024-09-17T21:30:45.837623+0000","last_fullsized":"2024-09-17T21:30:45.837623+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:55.998336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062140600000000002,"stat_sum":{"num_bytes":24701271,"num_objects":63,"num_object_clones":17,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":31,"num_read":3287,"num_read_kb":67859,"num_write":221,"num_write_kb":13907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":504918,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":101,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.837212+0000","last_change":"2024-09-17T21:29:59.544306+0000","last_active":"2024-09-17T21:30:45.837212+0000","last_peered":"2024-09-17T21:30:45.837212+0000","last_clean":"2024-09-17T21:30:45.837212+0000","last_became_active":"2024-09-17T21:29:59.540943+0000","last_became_peered":"2024-09-17T21:29:59.540943+0000","last_unstale":"2024-09-17T21:30:45.837212+0000","last_undegraded":"2024-09-17T21:30:45.837212+0000","last_fullsized":"2024-09-17T21:30:45.837212+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:17.491167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":219,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.834360+0000","last_change":"2024-09-17T21:29:28.554672+0000","last_active":"2024-09-17T21:30:45.834360+0000","last_peered":"2024-09-17T21:30:45.834360+0000","last_clean":"2024-09-17T21:30:45.834360+0000","last_became_active":"2024-09-17T21:29:28.553733+0000","last_became_peered":"2024-09-17T21:29:28.553733+0000","last_unstale":"2024-09-17T21:30:45.834360+0000","last_undegraded":"2024-09-17T21:30:45.834360+0000","last_fullsized":"2024-09-17T21:30:45.834360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_clean_scrub_stamp":"2024-09-17T21:29:26.925225+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:36:42.585477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"76'453","reported_seq":3520,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.837395+0000","last_change":"2024-09-17T21:30:31.754504+0000","last_active":"2024-09-17T21:30:45.837395+0000","last_peered":"2024-09-17T21:30:45.837395+0000","last_clean":"2024-09-17T21:30:45.837395+0000","last_became_active":"2024-09-17T21:30:31.753839+0000","last_became_peered":"2024-09-17T21:30:31.753839+0000","last_unstale":"2024-09-17T21:30:45.837395+0000","last_undegraded":"2024-09-17T21:30:45.837395+0000","last_fullsized":"2024-09-17T21:30:45.837395+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:28.113702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082324299999999998,"stat_sum":{"num_bytes":33512806,"num_objects":50,"num_object_clones":12,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":19,"num_read":3890,"num_read_kb":28997,"num_write":608,"num_write_kb":13958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2410412,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"77'303","reported_seq":2844,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.893569+0000","last_change":"2024-09-17T21:29:43.476642+0000","last_active":"2024-09-17T21:30:45.893569+0000","last_peered":"2024-09-17T21:30:45.893569+0000","last_clean":"2024-09-17T21:30:45.893569+0000","last_became_active":"2024-09-17T21:29:41.477965+0000","last_became_peered":"2024-09-17T21:29:41.477965+0000","last_unstale":"2024-09-17T21:30:45.893569+0000","last_undegraded":"2024-09-17T21:30:45.893569+0000","last_fullsized":"2024-09-17T21:30:45.893569+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:20:10.956214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035185700000000002,"stat_sum":{"num_bytes":27227825,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":24,"num_read":3412,"num_read_kb":24580,"num_write":354,"num_write_kb":10797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"77'391","reported_seq":3019,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.882393+0000","last_change":"2024-09-17T21:29:43.563885+0000","last_active":"2024-09-17T21:30:45.882393+0000","last_peered":"2024-09-17T21:30:45.882393+0000","last_clean":"2024-09-17T21:30:45.882393+0000","last_became_active":"2024-09-17T21:29:41.473420+0000","last_became_peered":"2024-09-17T21:29:41.473420+0000","last_unstale":"2024-09-17T21:30:45.882393+0000","last_undegraded":"2024-09-17T21:30:45.882393+0000","last_fullsized":"2024-09-17T21:30:45.882393+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:13:57.298607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046561200000000001,"stat_sum":{"num_bytes":64828292,"num_objects":68,"num_object_clones":16,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":35,"num_read":3407,"num_read_kb":14340,"num_write":454,"num_write_kb":5588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"77'311","reported_seq":6180,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.858860+0000","last_change":"2024-09-17T21:29:43.472555+0000","last_active":"2024-09-17T21:30:45.858860+0000","last_peered":"2024-09-17T21:30:45.858860+0000","last_clean":"2024-09-17T21:30:45.858860+0000","last_became_active":"2024-09-17T21:29:41.472354+0000","last_became_peered":"2024-09-17T21:29:41.472354+0000","last_unstale":"2024-09-17T21:30:45.858860+0000","last_undegraded":"2024-09-17T21:30:45.858860+0000","last_fullsized":"2024-09-17T21:30:45.858860+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:01.108318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073818799999999997,"stat_sum":{"num_bytes":35857624,"num_objects":76,"num_object_clones":21,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":33,"num_read":6622,"num_read_kb":26942,"num_write":303,"num_write_kb":2341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"76'486","reported_seq":3800,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.837617+0000","last_change":"2024-09-17T21:30:02.564126+0000","last_active":"2024-09-17T21:30:45.837617+0000","last_peered":"2024-09-17T21:30:45.837617+0000","last_clean":"2024-09-17T21:30:45.837617+0000","last_became_active":"2024-09-17T21:30:02.563793+0000","last_became_peered":"2024-09-17T21:30:02.563793+0000","last_unstale":"2024-09-17T21:30:45.837617+0000","last_undegraded":"2024-09-17T21:30:45.837617+0000","last_fullsized":"2024-09-17T21:30:45.837617+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:16:05.284029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047685499999999999,"stat_sum":{"num_bytes":80207318,"num_objects":58,"num_object_clones":13,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":28,"num_read":6166,"num_read_kb":13463,"num_write":450,"num_write_kb":4205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":285875,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"76'425","reported_seq":3183,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.880662+0000","last_change":"2024-09-17T21:29:59.632891+0000","last_active":"2024-09-17T21:30:45.880662+0000","last_peered":"2024-09-17T21:30:45.880662+0000","last_clean":"2024-09-17T21:30:45.880662+0000","last_became_active":"2024-09-17T21:29:59.544069+0000","last_became_peered":"2024-09-17T21:29:59.544069+0000","last_unstale":"2024-09-17T21:30:45.880662+0000","last_undegraded":"2024-09-17T21:30:45.880662+0000","last_fullsized":"2024-09-17T21:30:45.880662+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:53.621796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065633800000000002,"stat_sum":{"num_bytes":18774010,"num_objects":65,"num_object_clones":18,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":32,"num_read":3826,"num_read_kb":11100,"num_write":562,"num_write_kb":4043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":669702,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"77'376","reported_seq":3376,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.845702+0000","last_change":"2024-09-17T21:29:43.563860+0000","last_active":"2024-09-17T21:30:45.845702+0000","last_peered":"2024-09-17T21:30:45.845702+0000","last_clean":"2024-09-17T21:30:45.845702+0000","last_became_active":"2024-09-17T21:29:41.473736+0000","last_became_peered":"2024-09-17T21:29:41.473736+0000","last_unstale":"2024-09-17T21:30:45.845702+0000","last_undegraded":"2024-09-17T21:30:45.845702+0000","last_fullsized":"2024-09-17T21:30:45.845702+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:48:11.353179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039426499999999998,"stat_sum":{"num_bytes":19179589,"num_objects":60,"num_object_clones":16,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":33,"num_read":3752,"num_read_kb":15158,"num_write":532,"num_write_kb":4215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"77'675","reported_seq":3796,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.834767+0000","last_change":"2024-09-17T21:29:43.564285+0000","last_active":"2024-09-17T21:30:45.834767+0000","last_peered":"2024-09-17T21:30:45.834767+0000","last_clean":"2024-09-17T21:30:45.834767+0000","last_became_active":"2024-09-17T21:29:41.473394+0000","last_became_peered":"2024-09-17T21:29:41.473394+0000","last_unstale":"2024-09-17T21:30:45.834767+0000","last_undegraded":"2024-09-17T21:30:45.834767+0000","last_fullsized":"2024-09-17T21:30:45.834767+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:45:59.441052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073387300000000001,"stat_sum":{"num_bytes":53576131,"num_objects":71,"num_object_clones":16,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":40,"num_read":5278,"num_read_kb":6466,"num_write":958,"num_write_kb":20625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"76'339","reported_seq":1850,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.882345+0000","last_change":"2024-09-17T21:29:43.564107+0000","last_active":"2024-09-17T21:30:45.882345+0000","last_peered":"2024-09-17T21:30:45.882345+0000","last_clean":"2024-09-17T21:30:45.882345+0000","last_became_active":"2024-09-17T21:29:41.471342+0000","last_became_peered":"2024-09-17T21:29:41.471342+0000","last_unstale":"2024-09-17T21:30:45.882345+0000","last_undegraded":"2024-09-17T21:30:45.882345+0000","last_fullsized":"2024-09-17T21:30:45.882345+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:11:27.078309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062934700000000004,"stat_sum":{"num_bytes":19112990,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":26,"num_read":2199,"num_read_kb":12602,"num_write":466,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"76'297","reported_seq":4124,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.834432+0000","last_change":"2024-09-17T21:29:43.563795+0000","last_active":"2024-09-17T21:30:45.834432+0000","last_peered":"2024-09-17T21:30:45.834432+0000","last_clean":"2024-09-17T21:30:45.834432+0000","last_became_active":"2024-09-17T21:29:41.473252+0000","last_became_peered":"2024-09-17T21:29:41.473252+0000","last_unstale":"2024-09-17T21:30:45.834432+0000","last_undegraded":"2024-09-17T21:30:45.834432+0000","last_fullsized":"2024-09-17T21:30:45.834432+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:25.375201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044419899999999999,"stat_sum":{"num_bytes":42827888,"num_objects":73,"num_object_clones":18,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":34,"num_read":4426,"num_read_kb":32051,"num_write":301,"num_write_kb":14263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"79'288","reported_seq":1784,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T21:30:48.867304+0000","last_change":"2024-09-17T21:29:43.486918+0000","last_active":"2024-09-17T21:30:48.867304+0000","last_peered":"2024-09-17T21:30:48.867304+0000","last_clean":"2024-09-17T21:30:48.867304+0000","last_became_active":"2024-09-17T21:29:41.477019+0000","last_became_peered":"2024-09-17T21:29:41.477019+0000","last_unstale":"2024-09-17T21:30:48.867304+0000","last_undegraded":"2024-09-17T21:30:48.867304+0000","last_fullsized":"2024-09-17T21:30:48.867304+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:03.568450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010796479999999999,"stat_sum":{"num_bytes":20815849,"num_objects":59,"num_object_clones":16,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":32,"num_read":1802,"num_read_kb":4575,"num_write":300,"num_write_kb":2124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"76'287","reported_seq":3277,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.834620+0000","last_change":"2024-09-17T21:29:43.564292+0000","last_active":"2024-09-17T21:30:45.834620+0000","last_peered":"2024-09-17T21:30:45.834620+0000","last_clean":"2024-09-17T21:30:45.834620+0000","last_became_active":"2024-09-17T21:29:41.473182+0000","last_became_peered":"2024-09-17T21:29:41.473182+0000","last_unstale":"2024-09-17T21:30:45.834620+0000","last_undegraded":"2024-09-17T21:30:45.834620+0000","last_fullsized":"2024-09-17T21:30:45.834620+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:27:31.519211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052844700000000003,"stat_sum":{"num_bytes":28264453,"num_objects":83,"num_object_clones":25,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":41,"num_read":3267,"num_read_kb":16545,"num_write":294,"num_write_kb":6612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1295608902,"num_objects":2092,"num_object_clones":521,"num_object_copies":4184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2092,"num_whiteouts":1017,"num_read":131228,"num_read_kb":763632,"num_write":16428,"num_write_kb":376411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":47148489,"num_keys_recovered":137,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":64880640,"data_stored":610434627,"data_compressed":15567105,"data_compressed_allocated":48222208,"data_compressed_original":594471412,"omap_allocated":10476,"internal_metadata":0},"log_size":13017,"ondisk_log_size":13017,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53376,"kb_used_data":8304,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94318464,"statfs":{"total":96636764160,"available":96582107136,"internally_reserved":0,"allocated":8503296,"data_stored":46849655,"data_compressed":1867786,"data_compressed_allocated":5001216,"data_compressed_original":43925786,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42028,"kb_used_data":6684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94329812,"statfs":{"total":96636764160,"available":96593727488,"internally_reserved":0,"allocated":6844416,"data_stored":52852837,"data_compressed":1839683,"data_compressed_allocated":4513792,"data_compressed_original":51169001,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":0,"seq":64424509460,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40684,"kb_used_data":13660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331156,"statfs":{"total":96636764160,"available":96595103744,"internally_reserved":0,"allocated":13987840,"data_stored":116713209,"data_compressed":3201163,"data_compressed_allocated":9641984,"data_compressed_original":112963073,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51060,"kb_used_data":11552,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94320780,"statfs":{"total":96636764160,"available":96584478720,"internally_reserved":0,"allocated":11829248,"data_stored":101616869,"data_compressed":2722439,"data_compressed_allocated":8294400,"data_compressed_original":98921333,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41424,"kb_used_data":6080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330416,"statfs":{"total":96636764160,"available":96594345984,"internally_reserved":0,"allocated":6225920,"data_stored":61915783,"data_compressed":989670,"data_compressed_allocated":3969024,"data_compressed_original":59942839,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":63,"seq":270582939653,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42316,"kb_used_data":9528,"kb_used_omap":20,"kb_used_meta":32747,"kb_avail":94329524,"statfs":{"total":96636764160,"available":96593432576,"internally_reserved":0,"allocated":9756672,"data_stored":88236957,"data_compressed":1600819,"data_compressed_allocated":6918144,"data_compressed_original":86157367,"omap_allocated":20925,"internal_metadata":33533507},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39852,"kb_used_data":12828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331988,"statfs":{"total":96636764160,"available":96595955712,"internally_reserved":0,"allocated":13135872,"data_stored":137941879,"data_compressed":3063685,"data_compressed_allocated":9191424,"data_compressed_original":134336170,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12156928,"data_stored":136931570,"data_compressed":3058716,"data_compressed_allocated":9154560,"data_compressed_original":133762730,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8658944,"data_stored":87613317,"data_compressed":1475547,"data_compressed_allocated":6782976,"data_compressed_original":85977143,"omap_allocated":10476,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5464064,"data_stored":62831876,"data_compressed":1015439,"data_compressed_allocated":4075520,"data_compressed_original":61253559,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11386880,"data_stored":105685966,"data_compressed":2882339,"data_compressed_allocated":8716288,"data_compressed_original":103384408,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":13082624,"data_stored":116276709,"data_compressed":3180708,"data_compressed_allocated":9621504,"data_compressed_original":112930305,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":1737628,"data_compressed":248875,"data_compressed_allocated":327680,"data_compressed_original":1593344,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6021120,"data_stored":53499810,"data_compressed":1842664,"data_compressed_allocated":4579328,"data_compressed_original":52217577,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7565312,"data_stored":45857751,"data_compressed":1862817,"data_compressed_allocated":4964352,"data_compressed_original":43352346,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:30:50.285 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:30:50.304 INFO:teuthology.orchestra.run.smithi129.stdout:5282 trunc from 0xe742bdf to 0x70a4612 2024-09-17T21:30:50.304 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:50.302+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:50.323 INFO:teuthology.orchestra.run.smithi129.stdout:5283 read 0x647ebf5 thru 0x6486b58 (0x7f64 bytes) 2024-09-17T21:30:50.323 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:50.323 INFO:teuthology.orchestra.run.smithi129.stdout:5285 writesame 0x138d599 thru 0x1398711 data_size 0x1(0xb179 bytes) 2024-09-17T21:30:50.357 INFO:teuthology.orchestra.run.smithi129.stdout:5286 write 0xe0cf7b9 thru 0xe0d7bd2 (0x841a bytes) 2024-09-17T21:30:50.362 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:50.362 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:50.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:50.362 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:50.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.362 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:36935 3.a 3:54c8759b:::rbd_data.1120e62eea89.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-17T21:30:50.362 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:50.362 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.399677345000001, 2024-09-17T21:30:50.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020540151999999999, 2024-09-17T21:30:50.362 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.362 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.363 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:50.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.363 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 36935 2024-09-17T21:30:50.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.363 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:50.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:50.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958910+0000", 2024-09-17T21:30:50.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:30:50.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958918+0000", 2024-09-17T21:30:50.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6469999999999998e-06 2024-09-17T21:30:50.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958933+0000", 2024-09-17T21:30:50.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4791e-05 2024-09-17T21:30:50.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958940+0000", 2024-09-17T21:30:50.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-17T21:30:50.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958990+0000", 2024-09-17T21:30:50.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9759000000000003e-05 2024-09-17T21:30:50.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.959039+0000", 2024-09-17T21:30:50.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9487000000000002e-05 2024-09-17T21:30:50.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:50.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.962098+0000", 2024-09-17T21:30:50.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003059258 2024-09-17T21:30:50.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:50.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979325+0000", 2024-09-17T21:30:50.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017226841 2024-09-17T21:30:50.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:50.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979343+0000", 2024-09-17T21:30:50.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7486000000000002e-05 2024-09-17T21:30:50.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:50.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979401+0000", 2024-09-17T21:30:50.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7980999999999999e-05 2024-09-17T21:30:50.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979449+0000", 2024-09-17T21:30:50.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8639e-05 2024-09-17T21:30:50.368 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.368 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.368 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.369 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39447 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 909312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.369 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:50.369 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.390739445000001, 2024-09-17T21:30:50.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017419153999999999, 2024-09-17T21:30:50.369 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.369 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.369 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.369 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.370 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.370 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.370 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39447 2024-09-17T21:30:50.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.370 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:50.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:50.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:50.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:30:50.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967849+0000", 2024-09-17T21:30:50.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4339999999999999e-06 2024-09-17T21:30:50.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967853+0000", 2024-09-17T21:30:50.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:30:50.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967857+0000", 2024-09-17T21:30:50.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:30:50.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985193+0000", 2024-09-17T21:30:50.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017335973000000001 2024-09-17T21:30:50.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985228+0000", 2024-09-17T21:30:50.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5154999999999998e-05 2024-09-17T21:30:50.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985266+0000", 2024-09-17T21:30:50.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8448000000000002e-05 2024-09-17T21:30:50.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.374 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.374 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39448 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 974848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.374 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:50.374 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.390726629, 2024-09-17T21:30:50.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017517066000000001, 2024-09-17T21:30:50.374 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.374 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.375 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.375 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39448 2024-09-17T21:30:50.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.375 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:50.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:50.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:50.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.72e-07 2024-09-17T21:30:50.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967861+0000", 2024-09-17T21:30:50.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.096e-06 2024-09-17T21:30:50.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967865+0000", 2024-09-17T21:30:50.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9339999999999999e-06 2024-09-17T21:30:50.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967868+0000", 2024-09-17T21:30:50.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.508e-06 2024-09-17T21:30:50.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985285+0000", 2024-09-17T21:30:50.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017417208 2024-09-17T21:30:50.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985339+0000", 2024-09-17T21:30:50.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3931000000000003e-05 2024-09-17T21:30:50.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985377+0000", 2024-09-17T21:30:50.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8016999999999999e-05 2024-09-17T21:30:50.380 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.380 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.380 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.380 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39449 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.380 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:50.380 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.390714518999999, 2024-09-17T21:30:50.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017595137, 2024-09-17T21:30:50.380 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.380 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.381 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.381 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.381 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.381 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.381 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39449 2024-09-17T21:30:50.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.381 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:50.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:50.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:50.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2700000000000001e-07 2024-09-17T21:30:50.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967873+0000", 2024-09-17T21:30:50.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.164e-06 2024-09-17T21:30:50.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967877+0000", 2024-09-17T21:30:50.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.23e-06 2024-09-17T21:30:50.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967880+0000", 2024-09-17T21:30:50.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.286e-06 2024-09-17T21:30:50.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985397+0000", 2024-09-17T21:30:50.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017516982 2024-09-17T21:30:50.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985431+0000", 2024-09-17T21:30:50.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3899999999999997e-05 2024-09-17T21:30:50.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985467+0000", 2024-09-17T21:30:50.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6347999999999998e-05 2024-09-17T21:30:50.385 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.385 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.385 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.385 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39450 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1105920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.385 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:50.386 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.390694261999997, 2024-09-17T21:30:50.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017677824000000002, 2024-09-17T21:30:50.386 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.386 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.386 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.386 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.386 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.386 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.386 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39450 2024-09-17T21:30:50.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.386 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:50.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:50.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967893+0000", 2024-09-17T21:30:50.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9100000000000004e-07 2024-09-17T21:30:50.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967895+0000", 2024-09-17T21:30:50.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:30:50.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967901+0000", 2024-09-17T21:30:50.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.083e-06 2024-09-17T21:30:50.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967906+0000", 2024-09-17T21:30:50.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:30:50.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985493+0000", 2024-09-17T21:30:50.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017587565999999999 2024-09-17T21:30:50.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985531+0000", 2024-09-17T21:30:50.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7490000000000002e-05 2024-09-17T21:30:50.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985570+0000", 2024-09-17T21:30:50.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9205000000000002e-05 2024-09-17T21:30:50.390 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.390 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.390 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.390 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39451 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1171456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.390 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:50.390 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.390673882000002, 2024-09-17T21:30:50.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017752337999999999, 2024-09-17T21:30:50.391 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.391 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.391 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.391 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39451 2024-09-17T21:30:50.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.391 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:50.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:50.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:50.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:30:50.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967915+0000", 2024-09-17T21:30:50.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780000000000001e-06 2024-09-17T21:30:50.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967922+0000", 2024-09-17T21:30:50.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7020000000000004e-06 2024-09-17T21:30:50.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967926+0000", 2024-09-17T21:30:50.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.809e-06 2024-09-17T21:30:50.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985590+0000", 2024-09-17T21:30:50.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017663913999999999 2024-09-17T21:30:50.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985627+0000", 2024-09-17T21:30:50.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6896e-05 2024-09-17T21:30:50.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985665+0000", 2024-09-17T21:30:50.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7793999999999998e-05 2024-09-17T21:30:50.395 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.395 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.395 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.395 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39452 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1236992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.395 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:50.395 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.39065394, 2024-09-17T21:30:50.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017829602999999999, 2024-09-17T21:30:50.395 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.395 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.395 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.395 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.396 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.396 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.396 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39452 2024-09-17T21:30:50.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.396 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:50.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:50.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:50.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:30:50.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967947+0000", 2024-09-17T21:30:50.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4280999999999999e-05 2024-09-17T21:30:50.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967954+0000", 2024-09-17T21:30:50.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3249999999999996e-06 2024-09-17T21:30:50.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967959+0000", 2024-09-17T21:30:50.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2299999999999999e-06 2024-09-17T21:30:50.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985687+0000", 2024-09-17T21:30:50.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017727844999999999 2024-09-17T21:30:50.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985725+0000", 2024-09-17T21:30:50.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8243e-05 2024-09-17T21:30:50.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985762+0000", 2024-09-17T21:30:50.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7023000000000001e-05 2024-09-17T21:30:50.399 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.400 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.400 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.400 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39453 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.400 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:50.400 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.390617452000001, 2024-09-17T21:30:50.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017893793000000002, 2024-09-17T21:30:50.400 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.400 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.400 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.400 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.400 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.401 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39453 2024-09-17T21:30:50.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.401 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:50.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:50.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967970+0000", 2024-09-17T21:30:50.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:30:50.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967972+0000", 2024-09-17T21:30:50.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0229999999999999e-06 2024-09-17T21:30:50.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967978+0000", 2024-09-17T21:30:50.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0430000000000004e-06 2024-09-17T21:30:50.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967982+0000", 2024-09-17T21:30:50.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2570000000000001e-06 2024-09-17T21:30:50.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985782+0000", 2024-09-17T21:30:50.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017799683 2024-09-17T21:30:50.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985818+0000", 2024-09-17T21:30:50.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6304999999999999e-05 2024-09-17T21:30:50.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985863+0000", 2024-09-17T21:30:50.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4953999999999997e-05 2024-09-17T21:30:50.404 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.404 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.404 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.405 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39454 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1368064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.405 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:50.405 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.390598124, 2024-09-17T21:30:50.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017968886, 2024-09-17T21:30:50.405 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.405 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.405 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.405 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.405 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.405 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.405 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39454 2024-09-17T21:30:50.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.406 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:50.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:50.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967989+0000", 2024-09-17T21:30:50.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.15e-07 2024-09-17T21:30:50.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967991+0000", 2024-09-17T21:30:50.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.945e-06 2024-09-17T21:30:50.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967997+0000", 2024-09-17T21:30:50.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0190000000000003e-06 2024-09-17T21:30:50.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968001+0000", 2024-09-17T21:30:50.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6650000000000002e-06 2024-09-17T21:30:50.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985883+0000", 2024-09-17T21:30:50.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017881971999999999 2024-09-17T21:30:50.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985920+0000", 2024-09-17T21:30:50.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-05 2024-09-17T21:30:50.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985957+0000", 2024-09-17T21:30:50.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7370000000000003e-05 2024-09-17T21:30:50.409 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.409 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.409 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.409 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39455 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.409 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:50.409 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.390578949999998, 2024-09-17T21:30:50.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018044437999999999, 2024-09-17T21:30:50.410 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.410 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.410 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.410 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.410 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.410 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.410 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39455 2024-09-17T21:30:50.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.410 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:50.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:50.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:50.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:30:50.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968010+0000", 2024-09-17T21:30:50.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.886e-06 2024-09-17T21:30:50.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968017+0000", 2024-09-17T21:30:50.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:30:50.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968021+0000", 2024-09-17T21:30:50.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850000000000004e-06 2024-09-17T21:30:50.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985977+0000", 2024-09-17T21:30:50.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017956368 2024-09-17T21:30:50.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986015+0000", 2024-09-17T21:30:50.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7610000000000001e-05 2024-09-17T21:30:50.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986052+0000", 2024-09-17T21:30:50.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7109e-05 2024-09-17T21:30:50.414 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.414 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.414 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.414 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39456 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1499136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.414 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:50.414 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.390558650000003, 2024-09-17T21:30:50.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018114514000000002, 2024-09-17T21:30:50.414 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.414 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.414 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.414 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.414 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.414 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.415 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39456 2024-09-17T21:30:50.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.415 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:50.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:50.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:50.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-17T21:30:50.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968030+0000", 2024-09-17T21:30:50.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0049999999999999e-06 2024-09-17T21:30:50.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968036+0000", 2024-09-17T21:30:50.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0129999999999999e-06 2024-09-17T21:30:50.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968041+0000", 2024-09-17T21:30:50.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8300000000000003e-06 2024-09-17T21:30:50.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986071+0000", 2024-09-17T21:30:50.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018029620999999999 2024-09-17T21:30:50.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986106+0000", 2024-09-17T21:30:50.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4867999999999999e-05 2024-09-17T21:30:50.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986142+0000", 2024-09-17T21:30:50.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6566000000000002e-05 2024-09-17T21:30:50.418 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.418 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.418 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.418 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39457 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1564672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.418 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:50.418 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.390538833999997, 2024-09-17T21:30:50.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018188644, 2024-09-17T21:30:50.419 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.419 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.419 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.419 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.419 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.419 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.419 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39457 2024-09-17T21:30:50.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.419 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:50.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:50.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:50.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7199999999999999e-07 2024-09-17T21:30:50.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968050+0000", 2024-09-17T21:30:50.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1110000000000002e-06 2024-09-17T21:30:50.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968057+0000", 2024-09-17T21:30:50.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6170000000000003e-06 2024-09-17T21:30:50.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968062+0000", 2024-09-17T21:30:50.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:30:50.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986163+0000", 2024-09-17T21:30:50.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018101435999999999 2024-09-17T21:30:50.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986199+0000", 2024-09-17T21:30:50.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.591e-05 2024-09-17T21:30:50.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986236+0000", 2024-09-17T21:30:50.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7416999999999998e-05 2024-09-17T21:30:50.423 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.423 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.423 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.423 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39458 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1630208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.423 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:50.423 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.390511893999999, 2024-09-17T21:30:50.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018255856000000001, 2024-09-17T21:30:50.423 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.423 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.423 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.423 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.424 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.424 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.424 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39458 2024-09-17T21:30:50.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.424 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:50.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:50.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:50.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:30:50.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968077+0000", 2024-09-17T21:30:50.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.779e-06 2024-09-17T21:30:50.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968081+0000", 2024-09-17T21:30:50.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.532e-06 2024-09-17T21:30:50.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968085+0000", 2024-09-17T21:30:50.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999999e-06 2024-09-17T21:30:50.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986256+0000", 2024-09-17T21:30:50.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018171273000000002 2024-09-17T21:30:50.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986293+0000", 2024-09-17T21:30:50.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6534e-05 2024-09-17T21:30:50.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986330+0000", 2024-09-17T21:30:50.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7536000000000001e-05 2024-09-17T21:30:50.427 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.427 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.427 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.428 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39459 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1695744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.428 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:50.428 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.390497316000001, 2024-09-17T21:30:50.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334493, 2024-09-17T21:30:50.428 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.428 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.428 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.428 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.428 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.428 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.428 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39459 2024-09-17T21:30:50.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.428 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:50.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:50.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968090+0000", 2024-09-17T21:30:50.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:30:50.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968091+0000", 2024-09-17T21:30:50.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3319999999999999e-06 2024-09-17T21:30:50.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968094+0000", 2024-09-17T21:30:50.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2590000000000001e-06 2024-09-17T21:30:50.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968097+0000", 2024-09-17T21:30:50.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:30:50.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986350+0000", 2024-09-17T21:30:50.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018253364000000001 2024-09-17T21:30:50.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986387+0000", 2024-09-17T21:30:50.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6350999999999998e-05 2024-09-17T21:30:50.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986424+0000", 2024-09-17T21:30:50.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6897999999999998e-05 2024-09-17T21:30:50.432 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.432 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.432 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.432 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39460 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1761280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.432 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:50.432 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.390486180000003, 2024-09-17T21:30:50.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018415918999999999, 2024-09-17T21:30:50.432 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.433 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.433 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.433 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.433 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.433 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.433 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39460 2024-09-17T21:30:50.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.433 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:50.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:50.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968101+0000", 2024-09-17T21:30:50.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T21:30:50.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968102+0000", 2024-09-17T21:30:50.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1689999999999999e-06 2024-09-17T21:30:50.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968105+0000", 2024-09-17T21:30:50.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:30:50.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968108+0000", 2024-09-17T21:30:50.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6630000000000002e-06 2024-09-17T21:30:50.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986442+0000", 2024-09-17T21:30:50.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334487 2024-09-17T21:30:50.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986479+0000", 2024-09-17T21:30:50.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6557000000000003e-05 2024-09-17T21:30:50.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986516+0000", 2024-09-17T21:30:50.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7648999999999998e-05 2024-09-17T21:30:50.436 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.436 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.436 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.437 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39461 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1826816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.437 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:50.437 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.390475420000001, 2024-09-17T21:30:50.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018499768, 2024-09-17T21:30:50.437 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.437 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.437 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.437 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.437 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.437 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.437 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39461 2024-09-17T21:30:50.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.437 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:50.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:50.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:50.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4699999999999998e-07 2024-09-17T21:30:50.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968112+0000", 2024-09-17T21:30:50.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-06 2024-09-17T21:30:50.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968116+0000", 2024-09-17T21:30:50.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1080000000000001e-06 2024-09-17T21:30:50.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968122+0000", 2024-09-17T21:30:50.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1070000000000001e-06 2024-09-17T21:30:50.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986537+0000", 2024-09-17T21:30:50.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018414834000000001 2024-09-17T21:30:50.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986572+0000", 2024-09-17T21:30:50.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5726000000000003e-05 2024-09-17T21:30:50.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986611+0000", 2024-09-17T21:30:50.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8584000000000002e-05 2024-09-17T21:30:50.441 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.441 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.441 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.441 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39462 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1892352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.441 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:50.442 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.390461055999999, 2024-09-17T21:30:50.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018583252000000001, 2024-09-17T21:30:50.442 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.442 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.442 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.442 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39462 2024-09-17T21:30:50.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.442 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:50.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:50.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968126+0000", 2024-09-17T21:30:50.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:50.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968127+0000", 2024-09-17T21:30:50.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.229e-06 2024-09-17T21:30:50.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968131+0000", 2024-09-17T21:30:50.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8010000000000001e-06 2024-09-17T21:30:50.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968134+0000", 2024-09-17T21:30:50.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8880000000000001e-06 2024-09-17T21:30:50.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986634+0000", 2024-09-17T21:30:50.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018500501999999999 2024-09-17T21:30:50.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986671+0000", 2024-09-17T21:30:50.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6881000000000001e-05 2024-09-17T21:30:50.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986709+0000", 2024-09-17T21:30:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7546999999999998e-05 2024-09-17T21:30:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39463 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1957888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.390449441000001, 2024-09-17T21:30:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018667186999999998, 2024-09-17T21:30:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.446 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39463 2024-09-17T21:30:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:30:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968138+0000", 2024-09-17T21:30:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:30:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968142+0000", 2024-09-17T21:30:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.145e-06 2024-09-17T21:30:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968144+0000", 2024-09-17T21:30:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6869999999999999e-06 2024-09-17T21:30:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986728+0000", 2024-09-17T21:30:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018584163000000001 2024-09-17T21:30:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986766+0000", 2024-09-17T21:30:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7987000000000001e-05 2024-09-17T21:30:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986804+0000", 2024-09-17T21:30:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7892000000000002e-05 2024-09-17T21:30:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39464 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.450 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.390439131999997, 2024-09-17T21:30:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018753723999999999, 2024-09-17T21:30:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39464 2024-09-17T21:30:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968148+0000", 2024-09-17T21:30:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:30:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968152+0000", 2024-09-17T21:30:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3749999999999996e-06 2024-09-17T21:30:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968155+0000", 2024-09-17T21:30:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.371e-06 2024-09-17T21:30:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968158+0000", 2024-09-17T21:30:50.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T21:30:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986824+0000", 2024-09-17T21:30:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018665779 2024-09-17T21:30:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986863+0000", 2024-09-17T21:30:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9273000000000002e-05 2024-09-17T21:30:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986901+0000", 2024-09-17T21:30:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7978000000000001e-05 2024-09-17T21:30:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39465 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.390423732999999, 2024-09-17T21:30:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018825341999999998, 2024-09-17T21:30:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:50.455 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39465 2024-09-17T21:30:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:50.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-17T21:30:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968164+0000", 2024-09-17T21:30:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1289999999999999e-06 2024-09-17T21:30:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:50.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968167+0000", 2024-09-17T21:30:50.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1379999999999999e-06 2024-09-17T21:30:50.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:50.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968170+0000", 2024-09-17T21:30:50.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:30:50.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:50.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986914+0000", 2024-09-17T21:30:50.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018743829 2024-09-17T21:30:50.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:50.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986950+0000", 2024-09-17T21:30:50.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5819999999999999e-05 2024-09-17T21:30:50.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:50.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:50.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:50.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986988+0000", 2024-09-17T21:30:50.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8362000000000003e-05 2024-09-17T21:30:50.459 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.459 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.459 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.459 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:50.459 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:50.459 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:50.460 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:30:50.462 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:30:50.463 INFO:teuthology.orchestra.run.smithi129.stdout:5288 write 0xb58efe4 thru 0xb599a61 (0xaa7e bytes) 2024-09-17T21:30:50.463 INFO:teuthology.orchestra.run.smithi129.stdout:5289 writesame 0x98b1e40 thru 0x98bf9c6 data_size 0xd(0xdb87 bytes) 2024-09-17T21:30:50.463 INFO:teuthology.orchestra.run.smithi129.stdout:5290 read 0x1ae468f thru 0x1ae65c4 (0x1f36 bytes) 2024-09-17T21:30:50.463 INFO:teuthology.orchestra.run.smithi129.stdout:5291 read 0xdee1567 thru 0xdee9904 (0x839e bytes) 2024-09-17T21:30:50.463 INFO:teuthology.orchestra.run.smithi129.stdout:5293 read 0x4fafb66 thru 0x4fb7a5c (0x7ef7 bytes) 2024-09-17T21:30:50.463 INFO:teuthology.orchestra.run.smithi129.stdout:5294 read 0x71b383 thru 0x729f63 (0xebe1 bytes) 2024-09-17T21:30:50.463 INFO:teuthology.orchestra.run.smithi129.stdout:5295 read 0x644c48b thru 0x64502d3 (0x3e49 bytes) 2024-09-17T21:30:50.463 INFO:teuthology.orchestra.run.smithi129.stdout:5296 read 0x83c0ddc thru 0x83c8cde (0x7f03 bytes) 2024-09-17T21:30:50.464 INFO:teuthology.orchestra.run.smithi129.stdout:5297 trunc from 0xe0d7bd3 to 0x212419b 2024-09-17T21:30:50.464 INFO:teuthology.orchestra.run.smithi129.stdout:5298 punch from 0x11cf83b to 0x11d3e91, (0x4656 bytes) 2024-09-17T21:30:50.464 INFO:teuthology.orchestra.run.smithi129.stdout:5299 read 0x1f78ea9 thru 0x1f814a4 (0x85fc bytes) 2024-09-17T21:30:50.464 INFO:teuthology.orchestra.run.smithi129.stdout:5300 clone 41 order 22 su 16384 sc 3 2024-09-17T21:30:50.464 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:50.458+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:50.562 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:50.560+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:50.586 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:50.586 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:50.602 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:30:50.603 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:30:50.607 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:50.607 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:50.607 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:50.608 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:50.614 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:30:50.614 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-17T21:30:50.616 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:30:50.619 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:50.620 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:50.626 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-17T21:30:49.869876+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1296199289,"num_objects":2096,"num_object_clones":521,"num_object_copies":4192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2096,"num_whiteouts":1017,"num_read":131294,"num_read_kb":763687,"num_write":16531,"num_write_kb":378041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":47148489,"num_keys_recovered":137,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13092,"ondisk_log_size":13092,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":310740,"kb_used_data":68636,"kb_used_omap":29,"kb_used_meta":241954,"kb_avail":660292140,"statfs":{"total":676457349120,"available":676139151360,"internally_reserved":0,"allocated":70283264,"data_stored":606127189,"data_compressed":15285245,"data_compressed_allocated":47529984,"data_compressed_original":587415569,"omap_allocated":30485,"internal_metadata":247761131},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70222106,"num_objects":335,"num_object_clones":129,"num_object_copies":670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":149,"num_read":12694,"num_read_kb":70935,"num_write":1747,"num_write_kb":30916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.033990"},"pg_stats":[{"pgid":"3.1f","version":"79'570","reported_seq":3659,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T21:30:48.867619+0000","last_change":"2024-09-17T21:29:59.559639+0000","last_active":"2024-09-17T21:30:48.867619+0000","last_peered":"2024-09-17T21:30:48.867619+0000","last_clean":"2024-09-17T21:30:48.867619+0000","last_became_active":"2024-09-17T21:29:59.541221+0000","last_became_peered":"2024-09-17T21:29:59.541221+0000","last_unstale":"2024-09-17T21:30:48.867619+0000","last_undegraded":"2024-09-17T21:30:48.867619+0000","last_fullsized":"2024-09-17T21:30:48.867619+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:59:52.686527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011306689999999999,"stat_sum":{"num_bytes":56214113,"num_objects":78,"num_object_clones":22,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":39,"num_read":4766,"num_read_kb":61233,"num_write":961,"num_write_kb":32360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":67819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"76'374","reported_seq":3170,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.837716+0000","last_change":"2024-09-17T21:29:43.473228+0000","last_active":"2024-09-17T21:30:45.837716+0000","last_peered":"2024-09-17T21:30:45.837716+0000","last_clean":"2024-09-17T21:30:45.837716+0000","last_became_active":"2024-09-17T21:29:41.472161+0000","last_became_peered":"2024-09-17T21:29:41.472161+0000","last_unstale":"2024-09-17T21:30:45.837716+0000","last_undegraded":"2024-09-17T21:30:45.837716+0000","last_fullsized":"2024-09-17T21:30:45.837716+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:08:49.455683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092644099999999996,"stat_sum":{"num_bytes":53059342,"num_objects":57,"num_object_clones":11,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":27,"num_read":3114,"num_read_kb":22225,"num_write":376,"num_write_kb":9068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"77'491","reported_seq":4170,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.865209+0000","last_change":"2024-09-17T21:30:02.564806+0000","last_active":"2024-09-17T21:30:45.865209+0000","last_peered":"2024-09-17T21:30:45.865209+0000","last_clean":"2024-09-17T21:30:45.865209+0000","last_became_active":"2024-09-17T21:30:02.564199+0000","last_became_peered":"2024-09-17T21:30:02.564199+0000","last_unstale":"2024-09-17T21:30:45.865209+0000","last_undegraded":"2024-09-17T21:30:45.865209+0000","last_fullsized":"2024-09-17T21:30:45.865209+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:38:51.442823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063012799999999998,"stat_sum":{"num_bytes":79682618,"num_objects":66,"num_object_clones":16,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":27,"num_read":5177,"num_read_kb":44968,"num_write":589,"num_write_kb":16774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":573415,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"77'268","reported_seq":1507,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.876613+0000","last_change":"2024-09-17T21:30:28.802285+0000","last_active":"2024-09-17T21:30:45.876613+0000","last_peered":"2024-09-17T21:30:45.876613+0000","last_clean":"2024-09-17T21:30:45.876613+0000","last_became_active":"2024-09-17T21:30:28.747704+0000","last_became_peered":"2024-09-17T21:30:28.747704+0000","last_unstale":"2024-09-17T21:30:45.876613+0000","last_undegraded":"2024-09-17T21:30:45.876613+0000","last_fullsized":"2024-09-17T21:30:45.876613+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:04:21.389918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047754999999999998,"stat_sum":{"num_bytes":29484789,"num_objects":62,"num_object_clones":12,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":37,"num_read":1804,"num_read_kb":7115,"num_write":257,"num_write_kb":3676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":259277,"num_keys_recovered":68,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"79'438","reported_seq":7809,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T21:30:48.867469+0000","last_change":"2024-09-17T21:29:43.486642+0000","last_active":"2024-09-17T21:30:48.867469+0000","last_peered":"2024-09-17T21:30:48.867469+0000","last_clean":"2024-09-17T21:30:48.867469+0000","last_became_active":"2024-09-17T21:29:41.476041+0000","last_became_peered":"2024-09-17T21:29:41.476041+0000","last_unstale":"2024-09-17T21:30:48.867469+0000","last_undegraded":"2024-09-17T21:30:48.867469+0000","last_fullsized":"2024-09-17T21:30:48.867469+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:29:21.410196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086460200000000001,"stat_sum":{"num_bytes":62753802,"num_objects":81,"num_object_clones":25,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":34,"num_read":8368,"num_read_kb":52559,"num_write":480,"num_write_kb":20845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"77'415","reported_seq":4858,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.911004+0000","last_change":"2024-09-17T21:30:28.776386+0000","last_active":"2024-09-17T21:30:45.911004+0000","last_peered":"2024-09-17T21:30:45.911004+0000","last_clean":"2024-09-17T21:30:45.911004+0000","last_became_active":"2024-09-17T21:30:28.747031+0000","last_became_peered":"2024-09-17T21:30:28.747031+0000","last_unstale":"2024-09-17T21:30:45.911004+0000","last_undegraded":"2024-09-17T21:30:45.911004+0000","last_fullsized":"2024-09-17T21:30:45.911004+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:31:25.029342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075875499999999995,"stat_sum":{"num_bytes":36387548,"num_objects":68,"num_object_clones":14,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":33,"num_read":5126,"num_read_kb":18395,"num_write":508,"num_write_kb":5200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":235969,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"76'305","reported_seq":4218,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.842930+0000","last_change":"2024-09-17T21:30:28.887353+0000","last_active":"2024-09-17T21:30:45.842930+0000","last_peered":"2024-09-17T21:30:45.842930+0000","last_clean":"2024-09-17T21:30:45.842930+0000","last_became_active":"2024-09-17T21:30:28.745634+0000","last_became_peered":"2024-09-17T21:30:28.745634+0000","last_unstale":"2024-09-17T21:30:45.842930+0000","last_undegraded":"2024-09-17T21:30:45.842930+0000","last_fullsized":"2024-09-17T21:30:45.842930+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:54:18.766742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048345199999999999,"stat_sum":{"num_bytes":85276056,"num_objects":78,"num_object_clones":19,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":36,"num_read":4467,"num_read_kb":56590,"num_write":348,"num_write_kb":40933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":36017677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"76'351","reported_seq":3664,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.837182+0000","last_change":"2024-09-17T21:29:43.472344+0000","last_active":"2024-09-17T21:30:45.837182+0000","last_peered":"2024-09-17T21:30:45.837182+0000","last_clean":"2024-09-17T21:30:45.837182+0000","last_became_active":"2024-09-17T21:29:41.472468+0000","last_became_peered":"2024-09-17T21:29:41.472468+0000","last_unstale":"2024-09-17T21:30:45.837182+0000","last_undegraded":"2024-09-17T21:30:45.837182+0000","last_fullsized":"2024-09-17T21:30:45.837182+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:26:28.798879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061906600000000004,"stat_sum":{"num_bytes":38493140,"num_objects":72,"num_object_clones":20,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":25,"num_read":4251,"num_read_kb":22281,"num_write":391,"num_write_kb":11619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"79'505","reported_seq":2142,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T21:30:48.867810+0000","last_change":"2024-09-17T21:30:02.561153+0000","last_active":"2024-09-17T21:30:48.867810+0000","last_peered":"2024-09-17T21:30:48.867810+0000","last_clean":"2024-09-17T21:30:48.867810+0000","last_became_active":"2024-09-17T21:30:02.560707+0000","last_became_peered":"2024-09-17T21:30:02.560707+0000","last_unstale":"2024-09-17T21:30:48.867810+0000","last_undegraded":"2024-09-17T21:30:48.867810+0000","last_fullsized":"2024-09-17T21:30:48.867810+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:23:34.347190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173896,"stat_sum":{"num_bytes":54004323,"num_objects":73,"num_object_clones":18,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":32,"num_read":3607,"num_read_kb":21563,"num_write":574,"num_write_kb":29616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":602801,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"75'936","reported_seq":2208,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.834533+0000","last_change":"2024-09-17T21:30:31.755892+0000","last_active":"2024-09-17T21:30:45.834533+0000","last_peered":"2024-09-17T21:30:45.834533+0000","last_clean":"2024-09-17T21:30:45.834533+0000","last_became_active":"2024-09-17T21:30:31.754854+0000","last_became_peered":"2024-09-17T21:30:31.754854+0000","last_unstale":"2024-09-17T21:30:45.834533+0000","last_undegraded":"2024-09-17T21:30:45.834533+0000","last_fullsized":"2024-09-17T21:30:45.834533+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:00:03.734610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072238100000000002,"stat_sum":{"num_bytes":35657208,"num_objects":56,"num_object_clones":10,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":29,"num_read":5144,"num_read_kb":10502,"num_write":1517,"num_write_kb":7154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":1297843,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"76'333","reported_seq":2198,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.872521+0000","last_change":"2024-09-17T21:29:43.564521+0000","last_active":"2024-09-17T21:30:45.872521+0000","last_peered":"2024-09-17T21:30:45.872521+0000","last_clean":"2024-09-17T21:30:45.872521+0000","last_became_active":"2024-09-17T21:29:41.478346+0000","last_became_peered":"2024-09-17T21:29:41.478346+0000","last_unstale":"2024-09-17T21:30:45.872521+0000","last_undegraded":"2024-09-17T21:30:45.872521+0000","last_fullsized":"2024-09-17T21:30:45.872521+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:18:35.993942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069290300000000003,"stat_sum":{"num_bytes":34435185,"num_objects":66,"num_object_clones":14,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":40,"num_read":2733,"num_read_kb":4338,"num_write":306,"num_write_kb":3439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"76'212","reported_seq":2181,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.840128+0000","last_change":"2024-09-17T21:29:43.477145+0000","last_active":"2024-09-17T21:30:45.840128+0000","last_peered":"2024-09-17T21:30:45.840128+0000","last_clean":"2024-09-17T21:30:45.840128+0000","last_became_active":"2024-09-17T21:29:41.477611+0000","last_became_peered":"2024-09-17T21:29:41.477611+0000","last_unstale":"2024-09-17T21:30:45.840128+0000","last_undegraded":"2024-09-17T21:30:45.840128+0000","last_fullsized":"2024-09-17T21:30:45.840128+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:29:58.429084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073991200000000001,"stat_sum":{"num_bytes":27813108,"num_objects":64,"num_object_clones":20,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":35,"num_read":1903,"num_read_kb":9197,"num_write":296,"num_write_kb":11138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":128,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.882316+0000","last_change":"2024-09-17T21:29:35.002656+0000","last_active":"2024-09-17T21:30:45.882316+0000","last_peered":"2024-09-17T21:30:45.882316+0000","last_clean":"2024-09-17T21:30:45.882316+0000","last_became_active":"2024-09-17T21:29:32.971677+0000","last_became_peered":"2024-09-17T21:29:32.971677+0000","last_unstale":"2024-09-17T21:30:45.882316+0000","last_undegraded":"2024-09-17T21:30:45.882316+0000","last_fullsized":"2024-09-17T21:30:45.882316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:53.781657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041121300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"79'603","reported_seq":3310,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T21:30:48.867270+0000","last_change":"2024-09-17T21:30:31.755784+0000","last_active":"2024-09-17T21:30:48.867270+0000","last_peered":"2024-09-17T21:30:48.867270+0000","last_clean":"2024-09-17T21:30:48.867270+0000","last_became_active":"2024-09-17T21:30:31.755039+0000","last_became_peered":"2024-09-17T21:30:31.755039+0000","last_unstale":"2024-09-17T21:30:48.867270+0000","last_undegraded":"2024-09-17T21:30:48.867270+0000","last_fullsized":"2024-09-17T21:30:48.867270+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:14:17.603360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010838759999999999,"stat_sum":{"num_bytes":26661270,"num_objects":57,"num_object_clones":16,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":27,"num_read":5718,"num_read_kb":15265,"num_write":816,"num_write_kb":8008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":448871,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.841894+0000","last_change":"2024-09-17T21:30:28.745759+0000","last_active":"2024-09-17T21:30:45.841894+0000","last_peered":"2024-09-17T21:30:45.841894+0000","last_clean":"2024-09-17T21:30:45.841894+0000","last_became_active":"2024-09-17T21:30:28.744742+0000","last_became_peered":"2024-09-17T21:30:28.744742+0000","last_unstale":"2024-09-17T21:30:45.841894+0000","last_undegraded":"2024-09-17T21:30:45.841894+0000","last_fullsized":"2024-09-17T21:30:45.841894+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:37:02.014566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"76'434","reported_seq":1968,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.837344+0000","last_change":"2024-09-17T21:29:43.473294+0000","last_active":"2024-09-17T21:30:45.837344+0000","last_peered":"2024-09-17T21:30:45.837344+0000","last_clean":"2024-09-17T21:30:45.837344+0000","last_became_active":"2024-09-17T21:29:41.472095+0000","last_became_peered":"2024-09-17T21:29:41.472095+0000","last_unstale":"2024-09-17T21:30:45.837344+0000","last_undegraded":"2024-09-17T21:30:45.837344+0000","last_fullsized":"2024-09-17T21:30:45.837344+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:27:04.301133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074063099999999995,"stat_sum":{"num_bytes":14765697,"num_objects":70,"num_object_clones":19,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":33,"num_read":3228,"num_read_kb":8423,"num_write":603,"num_write_kb":4107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.834644+0000","last_change":"2024-09-17T21:29:35.078854+0000","last_active":"2024-09-17T21:30:45.834644+0000","last_peered":"2024-09-17T21:30:45.834644+0000","last_clean":"2024-09-17T21:30:45.834644+0000","last_became_active":"2024-09-17T21:29:33.071665+0000","last_became_peered":"2024-09-17T21:29:33.071665+0000","last_unstale":"2024-09-17T21:30:45.834644+0000","last_undegraded":"2024-09-17T21:30:45.834644+0000","last_fullsized":"2024-09-17T21:30:45.834644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.415687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047969899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"77'647","reported_seq":4071,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:46.518995+0000","last_change":"2024-09-17T21:30:31.754078+0000","last_active":"2024-09-17T21:30:46.518995+0000","last_peered":"2024-09-17T21:30:46.518995+0000","last_clean":"2024-09-17T21:30:46.518995+0000","last_became_active":"2024-09-17T21:30:31.752750+0000","last_became_peered":"2024-09-17T21:30:31.752750+0000","last_unstale":"2024-09-17T21:30:46.518995+0000","last_undegraded":"2024-09-17T21:30:46.518995+0000","last_fullsized":"2024-09-17T21:30:46.518995+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:02:44.288559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045510199999999998,"stat_sum":{"num_bytes":28784662,"num_objects":73,"num_object_clones":17,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":38,"num_read":5752,"num_read_kb":25799,"num_write":930,"num_write_kb":12035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3231045,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.842066+0000","last_change":"2024-09-17T21:30:28.748319+0000","last_active":"2024-09-17T21:30:45.842066+0000","last_peered":"2024-09-17T21:30:45.842066+0000","last_clean":"2024-09-17T21:30:45.842066+0000","last_became_active":"2024-09-17T21:30:28.747728+0000","last_became_peered":"2024-09-17T21:30:28.747728+0000","last_unstale":"2024-09-17T21:30:45.842066+0000","last_undegraded":"2024-09-17T21:30:45.842066+0000","last_fullsized":"2024-09-17T21:30:45.842066+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:58:18.954005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"76'210","reported_seq":3834,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.837719+0000","last_change":"2024-09-17T21:29:59.583758+0000","last_active":"2024-09-17T21:30:45.837719+0000","last_peered":"2024-09-17T21:30:45.837719+0000","last_clean":"2024-09-17T21:30:45.837719+0000","last_became_active":"2024-09-17T21:29:59.542323+0000","last_became_peered":"2024-09-17T21:29:59.542323+0000","last_unstale":"2024-09-17T21:30:45.837719+0000","last_undegraded":"2024-09-17T21:30:45.837719+0000","last_fullsized":"2024-09-17T21:30:45.837719+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:19:35.767020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074772900000000004,"stat_sum":{"num_bytes":28247651,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":29,"num_read":3747,"num_read_kb":26934,"num_write":202,"num_write_kb":6968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":541352,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"76'486","reported_seq":3588,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.840411+0000","last_change":"2024-09-17T21:29:43.477334+0000","last_active":"2024-09-17T21:30:45.840411+0000","last_peered":"2024-09-17T21:30:45.840411+0000","last_clean":"2024-09-17T21:30:45.840411+0000","last_became_active":"2024-09-17T21:29:41.475273+0000","last_became_peered":"2024-09-17T21:29:41.475273+0000","last_unstale":"2024-09-17T21:30:45.840411+0000","last_undegraded":"2024-09-17T21:30:45.840411+0000","last_fullsized":"2024-09-17T21:30:45.840411+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:27:18.423795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083339600000000003,"stat_sum":{"num_bytes":48620953,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":23,"num_read":4949,"num_read_kb":12938,"num_write":666,"num_write_kb":4910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":72,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.841873+0000","last_change":"2024-09-17T21:30:28.744890+0000","last_active":"2024-09-17T21:30:45.841873+0000","last_peered":"2024-09-17T21:30:45.841873+0000","last_clean":"2024-09-17T21:30:45.841873+0000","last_became_active":"2024-09-17T21:30:28.744567+0000","last_became_peered":"2024-09-17T21:30:28.744567+0000","last_unstale":"2024-09-17T21:30:45.841873+0000","last_undegraded":"2024-09-17T21:30:45.841873+0000","last_fullsized":"2024-09-17T21:30:45.841873+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:52:21.176000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"77'298","reported_seq":2370,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:46.111237+0000","last_change":"2024-09-17T21:30:28.895927+0000","last_active":"2024-09-17T21:30:46.111237+0000","last_peered":"2024-09-17T21:30:46.111237+0000","last_clean":"2024-09-17T21:30:46.111237+0000","last_became_active":"2024-09-17T21:30:28.746690+0000","last_became_peered":"2024-09-17T21:30:28.746690+0000","last_unstale":"2024-09-17T21:30:46.111237+0000","last_undegraded":"2024-09-17T21:30:46.111237+0000","last_fullsized":"2024-09-17T21:30:46.111237+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:40:11.028322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060580400000000004,"stat_sum":{"num_bytes":18342406,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":29,"num_read":3104,"num_read_kb":11229,"num_write":312,"num_write_kb":5281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1513,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":128,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.837468+0000","last_change":"2024-09-17T21:29:34.981930+0000","last_active":"2024-09-17T21:30:45.837468+0000","last_peered":"2024-09-17T21:30:45.837468+0000","last_clean":"2024-09-17T21:30:45.837468+0000","last_became_active":"2024-09-17T21:29:33.071313+0000","last_became_peered":"2024-09-17T21:29:33.071313+0000","last_unstale":"2024-09-17T21:30:45.837468+0000","last_undegraded":"2024-09-17T21:30:45.837468+0000","last_fullsized":"2024-09-17T21:30:45.837468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:11:51.460374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000527295,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"78'304","reported_seq":3069,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T21:30:48.867506+0000","last_change":"2024-09-17T21:29:43.486638+0000","last_active":"2024-09-17T21:30:48.867506+0000","last_peered":"2024-09-17T21:30:48.867506+0000","last_clean":"2024-09-17T21:30:48.867506+0000","last_became_active":"2024-09-17T21:29:41.473964+0000","last_became_peered":"2024-09-17T21:29:41.473964+0000","last_unstale":"2024-09-17T21:30:48.867506+0000","last_undegraded":"2024-09-17T21:30:48.867506+0000","last_fullsized":"2024-09-17T21:30:48.867506+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:05:03.507218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087607199999999998,"stat_sum":{"num_bytes":68038985,"num_objects":70,"num_object_clones":17,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":36,"num_read":2936,"num_read_kb":57400,"num_write":493,"num_write_kb":32986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":137,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.834356+0000","last_change":"2024-09-17T21:30:28.746557+0000","last_active":"2024-09-17T21:30:45.834356+0000","last_peered":"2024-09-17T21:30:45.834356+0000","last_clean":"2024-09-17T21:30:45.834356+0000","last_became_active":"2024-09-17T21:30:28.746261+0000","last_became_peered":"2024-09-17T21:30:28.746261+0000","last_unstale":"2024-09-17T21:30:45.834356+0000","last_undegraded":"2024-09-17T21:30:45.834356+0000","last_fullsized":"2024-09-17T21:30:45.834356+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:42.591689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044115199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"76'206","reported_seq":3034,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.837623+0000","last_change":"2024-09-17T21:29:59.600040+0000","last_active":"2024-09-17T21:30:45.837623+0000","last_peered":"2024-09-17T21:30:45.837623+0000","last_clean":"2024-09-17T21:30:45.837623+0000","last_became_active":"2024-09-17T21:29:59.544470+0000","last_became_peered":"2024-09-17T21:29:59.544470+0000","last_unstale":"2024-09-17T21:30:45.837623+0000","last_undegraded":"2024-09-17T21:30:45.837623+0000","last_fullsized":"2024-09-17T21:30:45.837623+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:55.998336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062140600000000002,"stat_sum":{"num_bytes":24701271,"num_objects":63,"num_object_clones":17,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":31,"num_read":3287,"num_read_kb":67859,"num_write":221,"num_write_kb":13907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":504918,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":101,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.837212+0000","last_change":"2024-09-17T21:29:59.544306+0000","last_active":"2024-09-17T21:30:45.837212+0000","last_peered":"2024-09-17T21:30:45.837212+0000","last_clean":"2024-09-17T21:30:45.837212+0000","last_became_active":"2024-09-17T21:29:59.540943+0000","last_became_peered":"2024-09-17T21:29:59.540943+0000","last_unstale":"2024-09-17T21:30:45.837212+0000","last_undegraded":"2024-09-17T21:30:45.837212+0000","last_fullsized":"2024-09-17T21:30:45.837212+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:17.491167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":219,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.834360+0000","last_change":"2024-09-17T21:29:28.554672+0000","last_active":"2024-09-17T21:30:45.834360+0000","last_peered":"2024-09-17T21:30:45.834360+0000","last_clean":"2024-09-17T21:30:45.834360+0000","last_became_active":"2024-09-17T21:29:28.553733+0000","last_became_peered":"2024-09-17T21:29:28.553733+0000","last_unstale":"2024-09-17T21:30:45.834360+0000","last_undegraded":"2024-09-17T21:30:45.834360+0000","last_fullsized":"2024-09-17T21:30:45.834360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_clean_scrub_stamp":"2024-09-17T21:29:26.925225+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:36:42.585477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"76'453","reported_seq":3520,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.837395+0000","last_change":"2024-09-17T21:30:31.754504+0000","last_active":"2024-09-17T21:30:45.837395+0000","last_peered":"2024-09-17T21:30:45.837395+0000","last_clean":"2024-09-17T21:30:45.837395+0000","last_became_active":"2024-09-17T21:30:31.753839+0000","last_became_peered":"2024-09-17T21:30:31.753839+0000","last_unstale":"2024-09-17T21:30:45.837395+0000","last_undegraded":"2024-09-17T21:30:45.837395+0000","last_fullsized":"2024-09-17T21:30:45.837395+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:28.113702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082324299999999998,"stat_sum":{"num_bytes":33512806,"num_objects":50,"num_object_clones":12,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":19,"num_read":3890,"num_read_kb":28997,"num_write":608,"num_write_kb":13958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2410412,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"77'303","reported_seq":2844,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.893569+0000","last_change":"2024-09-17T21:29:43.476642+0000","last_active":"2024-09-17T21:30:45.893569+0000","last_peered":"2024-09-17T21:30:45.893569+0000","last_clean":"2024-09-17T21:30:45.893569+0000","last_became_active":"2024-09-17T21:29:41.477965+0000","last_became_peered":"2024-09-17T21:29:41.477965+0000","last_unstale":"2024-09-17T21:30:45.893569+0000","last_undegraded":"2024-09-17T21:30:45.893569+0000","last_fullsized":"2024-09-17T21:30:45.893569+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:20:10.956214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035185700000000002,"stat_sum":{"num_bytes":27227825,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":24,"num_read":3412,"num_read_kb":24580,"num_write":354,"num_write_kb":10797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"77'391","reported_seq":3019,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.882393+0000","last_change":"2024-09-17T21:29:43.563885+0000","last_active":"2024-09-17T21:30:45.882393+0000","last_peered":"2024-09-17T21:30:45.882393+0000","last_clean":"2024-09-17T21:30:45.882393+0000","last_became_active":"2024-09-17T21:29:41.473420+0000","last_became_peered":"2024-09-17T21:29:41.473420+0000","last_unstale":"2024-09-17T21:30:45.882393+0000","last_undegraded":"2024-09-17T21:30:45.882393+0000","last_fullsized":"2024-09-17T21:30:45.882393+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:13:57.298607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046561200000000001,"stat_sum":{"num_bytes":64828292,"num_objects":68,"num_object_clones":16,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":35,"num_read":3407,"num_read_kb":14340,"num_write":454,"num_write_kb":5588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"77'311","reported_seq":6180,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.858860+0000","last_change":"2024-09-17T21:29:43.472555+0000","last_active":"2024-09-17T21:30:45.858860+0000","last_peered":"2024-09-17T21:30:45.858860+0000","last_clean":"2024-09-17T21:30:45.858860+0000","last_became_active":"2024-09-17T21:29:41.472354+0000","last_became_peered":"2024-09-17T21:29:41.472354+0000","last_unstale":"2024-09-17T21:30:45.858860+0000","last_undegraded":"2024-09-17T21:30:45.858860+0000","last_fullsized":"2024-09-17T21:30:45.858860+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:01.108318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073818799999999997,"stat_sum":{"num_bytes":35857624,"num_objects":76,"num_object_clones":21,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":33,"num_read":6622,"num_read_kb":26942,"num_write":303,"num_write_kb":2341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"76'486","reported_seq":3800,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.837617+0000","last_change":"2024-09-17T21:30:02.564126+0000","last_active":"2024-09-17T21:30:45.837617+0000","last_peered":"2024-09-17T21:30:45.837617+0000","last_clean":"2024-09-17T21:30:45.837617+0000","last_became_active":"2024-09-17T21:30:02.563793+0000","last_became_peered":"2024-09-17T21:30:02.563793+0000","last_unstale":"2024-09-17T21:30:45.837617+0000","last_undegraded":"2024-09-17T21:30:45.837617+0000","last_fullsized":"2024-09-17T21:30:45.837617+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:16:05.284029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047685499999999999,"stat_sum":{"num_bytes":80207318,"num_objects":58,"num_object_clones":13,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":28,"num_read":6166,"num_read_kb":13463,"num_write":450,"num_write_kb":4205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":285875,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"76'425","reported_seq":3183,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.880662+0000","last_change":"2024-09-17T21:29:59.632891+0000","last_active":"2024-09-17T21:30:45.880662+0000","last_peered":"2024-09-17T21:30:45.880662+0000","last_clean":"2024-09-17T21:30:45.880662+0000","last_became_active":"2024-09-17T21:29:59.544069+0000","last_became_peered":"2024-09-17T21:29:59.544069+0000","last_unstale":"2024-09-17T21:30:45.880662+0000","last_undegraded":"2024-09-17T21:30:45.880662+0000","last_fullsized":"2024-09-17T21:30:45.880662+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:53.621796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065633800000000002,"stat_sum":{"num_bytes":18774010,"num_objects":65,"num_object_clones":18,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":32,"num_read":3826,"num_read_kb":11100,"num_write":562,"num_write_kb":4043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":669702,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"77'376","reported_seq":3376,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.845702+0000","last_change":"2024-09-17T21:29:43.563860+0000","last_active":"2024-09-17T21:30:45.845702+0000","last_peered":"2024-09-17T21:30:45.845702+0000","last_clean":"2024-09-17T21:30:45.845702+0000","last_became_active":"2024-09-17T21:29:41.473736+0000","last_became_peered":"2024-09-17T21:29:41.473736+0000","last_unstale":"2024-09-17T21:30:45.845702+0000","last_undegraded":"2024-09-17T21:30:45.845702+0000","last_fullsized":"2024-09-17T21:30:45.845702+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:48:11.353179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039426499999999998,"stat_sum":{"num_bytes":19179589,"num_objects":60,"num_object_clones":16,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":33,"num_read":3752,"num_read_kb":15158,"num_write":532,"num_write_kb":4215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"77'675","reported_seq":3796,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.834767+0000","last_change":"2024-09-17T21:29:43.564285+0000","last_active":"2024-09-17T21:30:45.834767+0000","last_peered":"2024-09-17T21:30:45.834767+0000","last_clean":"2024-09-17T21:30:45.834767+0000","last_became_active":"2024-09-17T21:29:41.473394+0000","last_became_peered":"2024-09-17T21:29:41.473394+0000","last_unstale":"2024-09-17T21:30:45.834767+0000","last_undegraded":"2024-09-17T21:30:45.834767+0000","last_fullsized":"2024-09-17T21:30:45.834767+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:45:59.441052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073387300000000001,"stat_sum":{"num_bytes":53576131,"num_objects":71,"num_object_clones":16,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":40,"num_read":5278,"num_read_kb":6466,"num_write":958,"num_write_kb":20625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"76'339","reported_seq":1850,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.882345+0000","last_change":"2024-09-17T21:29:43.564107+0000","last_active":"2024-09-17T21:30:45.882345+0000","last_peered":"2024-09-17T21:30:45.882345+0000","last_clean":"2024-09-17T21:30:45.882345+0000","last_became_active":"2024-09-17T21:29:41.471342+0000","last_became_peered":"2024-09-17T21:29:41.471342+0000","last_unstale":"2024-09-17T21:30:45.882345+0000","last_undegraded":"2024-09-17T21:30:45.882345+0000","last_fullsized":"2024-09-17T21:30:45.882345+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:11:27.078309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062934700000000004,"stat_sum":{"num_bytes":19112990,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":26,"num_read":2199,"num_read_kb":12602,"num_write":466,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"76'297","reported_seq":4124,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.834432+0000","last_change":"2024-09-17T21:29:43.563795+0000","last_active":"2024-09-17T21:30:45.834432+0000","last_peered":"2024-09-17T21:30:45.834432+0000","last_clean":"2024-09-17T21:30:45.834432+0000","last_became_active":"2024-09-17T21:29:41.473252+0000","last_became_peered":"2024-09-17T21:29:41.473252+0000","last_unstale":"2024-09-17T21:30:45.834432+0000","last_undegraded":"2024-09-17T21:30:45.834432+0000","last_fullsized":"2024-09-17T21:30:45.834432+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:25.375201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044419899999999999,"stat_sum":{"num_bytes":42827888,"num_objects":73,"num_object_clones":18,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":34,"num_read":4426,"num_read_kb":32051,"num_write":301,"num_write_kb":14263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"79'288","reported_seq":1784,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T21:30:48.867304+0000","last_change":"2024-09-17T21:29:43.486918+0000","last_active":"2024-09-17T21:30:48.867304+0000","last_peered":"2024-09-17T21:30:48.867304+0000","last_clean":"2024-09-17T21:30:48.867304+0000","last_became_active":"2024-09-17T21:29:41.477019+0000","last_became_peered":"2024-09-17T21:29:41.477019+0000","last_unstale":"2024-09-17T21:30:48.867304+0000","last_undegraded":"2024-09-17T21:30:48.867304+0000","last_fullsized":"2024-09-17T21:30:48.867304+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:03.568450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010796479999999999,"stat_sum":{"num_bytes":20815849,"num_objects":59,"num_object_clones":16,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":32,"num_read":1802,"num_read_kb":4575,"num_write":300,"num_write_kb":2124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"76'287","reported_seq":3277,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T21:30:45.834620+0000","last_change":"2024-09-17T21:29:43.564292+0000","last_active":"2024-09-17T21:30:45.834620+0000","last_peered":"2024-09-17T21:30:45.834620+0000","last_clean":"2024-09-17T21:30:45.834620+0000","last_became_active":"2024-09-17T21:29:41.473182+0000","last_became_peered":"2024-09-17T21:29:41.473182+0000","last_unstale":"2024-09-17T21:30:45.834620+0000","last_undegraded":"2024-09-17T21:30:45.834620+0000","last_fullsized":"2024-09-17T21:30:45.834620+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:27:31.519211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052844700000000003,"stat_sum":{"num_bytes":28264453,"num_objects":83,"num_object_clones":25,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":41,"num_read":3267,"num_read_kb":16545,"num_write":294,"num_write_kb":6612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1295608902,"num_objects":2092,"num_object_clones":521,"num_object_copies":4184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2092,"num_whiteouts":1017,"num_read":131228,"num_read_kb":763632,"num_write":16428,"num_write_kb":376411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":47148489,"num_keys_recovered":137,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":64880640,"data_stored":610434627,"data_compressed":15567105,"data_compressed_allocated":48222208,"data_compressed_original":594471412,"omap_allocated":10476,"internal_metadata":0},"log_size":13017,"ondisk_log_size":13017,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53376,"kb_used_data":8304,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94318464,"statfs":{"total":96636764160,"available":96582107136,"internally_reserved":0,"allocated":8503296,"data_stored":46849655,"data_compressed":1867786,"data_compressed_allocated":5001216,"data_compressed_original":43925786,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42028,"kb_used_data":6684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94329812,"statfs":{"total":96636764160,"available":96593727488,"internally_reserved":0,"allocated":6844416,"data_stored":52852837,"data_compressed":1839683,"data_compressed_allocated":4513792,"data_compressed_original":51169001,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":0,"seq":64424509460,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40684,"kb_used_data":13660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331156,"statfs":{"total":96636764160,"available":96595103744,"internally_reserved":0,"allocated":13987840,"data_stored":116713209,"data_compressed":3201163,"data_compressed_allocated":9641984,"data_compressed_original":112963073,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51060,"kb_used_data":11552,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94320780,"statfs":{"total":96636764160,"available":96584478720,"internally_reserved":0,"allocated":11829248,"data_stored":101616869,"data_compressed":2722439,"data_compressed_allocated":8294400,"data_compressed_original":98921333,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41424,"kb_used_data":6080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330416,"statfs":{"total":96636764160,"available":96594345984,"internally_reserved":0,"allocated":6225920,"data_stored":61915783,"data_compressed":989670,"data_compressed_allocated":3969024,"data_compressed_original":59942839,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":63,"seq":270582939653,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42316,"kb_used_data":9528,"kb_used_omap":20,"kb_used_meta":32747,"kb_avail":94329524,"statfs":{"total":96636764160,"available":96593432576,"internally_reserved":0,"allocated":9756672,"data_stored":88236957,"data_compressed":1600819,"data_compressed_allocated":6918144,"data_compressed_original":86157367,"omap_allocated":20925,"internal_metadata":33533507},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39852,"kb_used_data":12828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331988,"statfs":{"total":96636764160,"available":96595955712,"internally_reserved":0,"allocated":13135872,"data_stored":137941879,"data_compressed":3063685,"data_compressed_allocated":9191424,"data_compressed_original":134336170,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12156928,"data_stored":136931570,"data_compressed":3058716,"data_compressed_allocated":9154560,"data_compressed_original":133762730,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8658944,"data_stored":87613317,"data_compressed":1475547,"data_compressed_allocated":6782976,"data_compressed_original":85977143,"omap_allocated":10476,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5464064,"data_stored":62831876,"data_compressed":1015439,"data_compressed_allocated":4075520,"data_compressed_original":61253559,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11386880,"data_stored":105685966,"data_compressed":2882339,"data_compressed_allocated":8716288,"data_compressed_original":103384408,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":13082624,"data_stored":116276709,"data_compressed":3180708,"data_compressed_allocated":9621504,"data_compressed_original":112930305,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":1737628,"data_compressed":248875,"data_compressed_allocated":327680,"data_compressed_original":1593344,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6021120,"data_stored":53499810,"data_compressed":1842664,"data_compressed_allocated":4579328,"data_compressed_original":52217577,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7565312,"data_stored":45857751,"data_compressed":1862817,"data_compressed_allocated":4964352,"data_compressed_original":43352346,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:30:50.628 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T21:30:50.628 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:30:50.629 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T21:30:50.629 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 0.989723 2024-09-17T21:30:50.629 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 0.9897233866907813 2024-09-17T21:30:50.637 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:50.637 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:50.654 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:50.654 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:50.671 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:50.671 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:50.700 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:50.698+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:50.702 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:50.702 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:50.733 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:50.733 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:50.761 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:50.761 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:50.770 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:30:50.785 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:50.783+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:50.864 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:50.862+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:50.918 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:50.918 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:50.927 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:30:50.941 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone40 from 0x212419b (overlap 0x212419b) to 0x10920cd 2024-09-17T21:30:50.965 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:50.963+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:51.008 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-17T21:30:51.045 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:51.045 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:51.057 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:30:51.099 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:51.096+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:51.168 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:51.165+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:51.207 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:51.207 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:51.217 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:30:51.271 INFO:teuthology.orchestra.run.smithi129.stdout:5301 punch from 0x678d03 to 0x683553, (0xa850 bytes) 2024-09-17T21:30:51.271 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:51.269+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:51.276 INFO:teuthology.orchestra.run.smithi129.stdout:5304 read 0x1471895 thru 0x147b916 (0xa082 bytes) 2024-09-17T21:30:51.295 INFO:teuthology.orchestra.run.smithi129.stdout:5305 write 0xdafce27 thru 0xdb0a8ec (0xdac6 bytes) 2024-09-17T21:30:51.305 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:51.305 INFO:teuthology.orchestra.run.smithi129.stdout:5307 write 0x124c448 thru 0x125689e (0xa457 bytes) 2024-09-17T21:30:51.336 INFO:teuthology.orchestra.run.smithi129.stdout:5308 read 0xc33846b thru 0xc34015f (0x7cf5 bytes) 2024-09-17T21:30:51.336 INFO:teuthology.orchestra.run.smithi129.stdout:5310 write 0x6d8d344 thru 0x6d969d5 (0x9692 bytes) 2024-09-17T21:30:51.343 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:51.343 INFO:teuthology.orchestra.run.smithi129.stdout:5312 read 0xd6d9a19 thru 0xd6e694b (0xcf33 bytes) 2024-09-17T21:30:51.343 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:51.343 INFO:teuthology.orchestra.run.smithi129.stdout:5315 read 0xaa36f72 thru 0xaa405de (0x966d bytes) 2024-09-17T21:30:51.343 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:51.344 INFO:teuthology.orchestra.run.smithi129.stdout:5318 write 0x6505a6f thru 0x650716e (0x1700 bytes) 2024-09-17T21:30:51.347 INFO:teuthology.orchestra.run.smithi129.stdout:5319 write 0x9f624db thru 0x9f6dfcf (0xbaf5 bytes) 2024-09-17T21:30:51.353 INFO:teuthology.orchestra.run.smithi129.stdout:5321 writesame 0x68bebfb thru 0x68c48a0 data_size 0x76(0x5ca6 bytes) 2024-09-17T21:30:51.363 INFO:teuthology.orchestra.run.smithi129.stdout:5322 trunc from 0xdb0a8ed to 0xb24eee9 2024-09-17T21:30:51.369 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:51.366+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:51.376 INFO:teuthology.orchestra.run.smithi129.stdout:5323 write 0x29c6572 thru 0x29ca9ac (0x443b bytes) 2024-09-17T21:30:51.380 INFO:teuthology.orchestra.run.smithi129.stdout:5324 write 0x933015b thru 0x93351ca (0x5070 bytes) 2024-09-17T21:30:51.385 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:51.385 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:51.391 INFO:teuthology.orchestra.run.smithi129.stdout:5325 write 0xd3a3184 thru 0xd3a6b27 (0x39a4 bytes) 2024-09-17T21:30:51.397 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:30:51.397 INFO:teuthology.orchestra.run.smithi129.stdout:5326 read 0x9c56647 thru 0x9c65555 (0xef0f bytes) 2024-09-17T21:30:51.397 INFO:teuthology.orchestra.run.smithi129.stdout:5327 write 0x387d88b thru 0x388d4cb (0xfc41 bytes) 2024-09-17T21:30:51.401 INFO:teuthology.orchestra.run.smithi129.stdout:5328 write 0x5ba5ae1 thru 0x5ba8228 (0x2748 bytes) 2024-09-17T21:30:51.404 INFO:teuthology.orchestra.run.smithi129.stdout:5332 writesame 0x2cdfae1 thru 0x2ce7ccb data_size 0x1(0x81eb bytes) 2024-09-17T21:30:51.421 INFO:teuthology.orchestra.run.smithi129.stdout:5333 read 0x35d2289 thru 0x35dd353 (0xb0cb bytes) 2024-09-17T21:30:51.422 INFO:teuthology.orchestra.run.smithi129.stdout:5334 writesame 0x21f3c02 thru 0x21f4ead data_size 0xa(0x12ac bytes) 2024-09-17T21:30:51.433 INFO:teuthology.orchestra.run.smithi129.stdout:5335 read 0x4d4bea7 thru 0x4d51c0b (0x5d65 bytes) 2024-09-17T21:30:51.436 INFO:teuthology.orchestra.run.smithi129.stdout:5336 trunc from 0xd3a6b28 to 0xac15575 2024-09-17T21:30:51.447 INFO:teuthology.orchestra.run.smithi129.stdout:5337 read 0x9b1ecab thru 0x9b262ae (0x7604 bytes) 2024-09-17T21:30:51.453 INFO:teuthology.orchestra.run.smithi129.stdout:5338 trunc from 0xac15575 to 0xede0968 2024-09-17T21:30:51.458 INFO:teuthology.orchestra.run.smithi129.stdout:5339 writesame 0x2e7eb8d thru 0x2e81cd3 data_size 0xf(0x3147 bytes) 2024-09-17T21:30:51.459 INFO:teuthology.orchestra.run.smithi129.stdout:5340 write 0xc59b6a6 thru 0xc5a37b0 (0x810b bytes) 2024-09-17T21:30:51.470 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:51.468+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:51.470 INFO:teuthology.orchestra.run.smithi129.stdout:5341 read 0xdf86bba thru 0xdf87e23 (0x126a bytes) 2024-09-17T21:30:51.471 INFO:teuthology.orchestra.run.smithi129.stdout:5342 writesame 0x9df3009 thru 0x9df83e8 data_size 0x16(0x53e0 bytes) 2024-09-17T21:30:51.472 INFO:teuthology.orchestra.run.smithi129.stdout:5344 punch from 0x1a3c058 to 0x1a4b275, (0xf21d bytes) 2024-09-17T21:30:51.472 INFO:teuthology.orchestra.run.smithi129.stdout:5346 write 0xecd4dca thru 0xece0365 (0xb59c bytes) 2024-09-17T21:30:51.477 INFO:teuthology.orchestra.run.smithi129.stdout:5347 write 0xc1c6836 thru 0xc1c70be (0x889 bytes) 2024-09-17T21:30:51.479 INFO:teuthology.orchestra.run.smithi129.stdout:5348 write 0x49cf202 thru 0x49d11ac (0x1fab bytes) 2024-09-17T21:30:51.484 INFO:teuthology.orchestra.run.smithi129.stdout:5349 trunc from 0xede0968 to 0xde23b74 2024-09-17T21:30:51.500 INFO:teuthology.orchestra.run.smithi129.stdout:5351 trunc from 0xde23b74 to 0x8e9a0a3 2024-09-17T21:30:51.515 INFO:teuthology.orchestra.run.smithi129.stdout:5352 punch from 0x2ca65e5 to 0x2cb38bf, (0xd2da bytes) 2024-09-17T21:30:51.515 INFO:teuthology.orchestra.run.smithi129.stdout:5353 punch from 0x7f9c1cb to 0x7fa784b, (0xb680 bytes) 2024-09-17T21:30:51.515 INFO:teuthology.orchestra.run.smithi129.stdout:5354 read 0x5c74fbe thru 0x5c7ef0e (0x9f51 bytes) 2024-09-17T21:30:51.521 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:51.521 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:51.524 INFO:teuthology.orchestra.run.smithi129.stdout:5355 write 0xbcd0f48 thru 0xbcdda23 (0xcadc bytes) 2024-09-17T21:30:51.532 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:30:51.532 INFO:teuthology.orchestra.run.smithi129.stdout:5356 write 0x8ac15c2 thru 0x8ac5793 (0x41d2 bytes) 2024-09-17T21:30:51.537 INFO:teuthology.orchestra.run.smithi129.stdout:5358 read 0x632c5cf thru 0x6330b34 (0x4566 bytes) 2024-09-17T21:30:51.538 INFO:teuthology.orchestra.run.smithi129.stdout:5359 read 0x194409a thru 0x195044f (0xc3b6 bytes) 2024-09-17T21:30:51.538 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:51.538 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:51.539 INFO:teuthology.orchestra.run.smithi129.stdout:5362 trunc from 0xbcdda24 to 0xc12b94f 2024-09-17T21:30:51.544 INFO:teuthology.orchestra.run.smithi129.stdout:5364 read 0xae34a68 thru 0xae3822b (0x37c4 bytes) 2024-09-17T21:30:51.544 INFO:teuthology.orchestra.run.smithi129.stdout:5365 write 0x9b88baa thru 0x9b91202 (0x8659 bytes) 2024-09-17T21:30:51.555 INFO:teuthology.orchestra.run.smithi129.stdout:5366 punch from 0x1222ada to 0x122e003, (0xb529 bytes) 2024-09-17T21:30:51.555 INFO:teuthology.orchestra.run.smithi129.stdout:5367 punch from 0xadabcbd to 0xadaea9c, (0x2ddf bytes) 2024-09-17T21:30:51.555 INFO:teuthology.orchestra.run.smithi129.stdout:5368 read 0x5fe37b3 thru 0x5fe55f2 (0x1e40 bytes) 2024-09-17T21:30:51.555 INFO:teuthology.orchestra.run.smithi129.stdout:5369 punch from 0x8363a6a to 0x836a6a1, (0x6c37 bytes) 2024-09-17T21:30:51.563 INFO:teuthology.orchestra.run.smithi129.stdout:5371 writesame 0xde3ad24 thru 0xde468f2 data_size 0x1(0xbbcf bytes) 2024-09-17T21:30:51.563 INFO:teuthology.orchestra.run.smithi129.stdout:5372 write 0xc6b43 thru 0xcddef (0x72ad bytes) 2024-09-17T21:30:51.570 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:51.568+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:51.608 INFO:teuthology.orchestra.run.smithi129.stdout:5374 read 0x9ba410c thru 0x9bace5b (0x8d50 bytes) 2024-09-17T21:30:51.609 INFO:teuthology.orchestra.run.smithi129.stdout:5375 punch from 0xc70ce9c to 0xc71b06d, (0xe1d1 bytes) 2024-09-17T21:30:51.609 INFO:teuthology.orchestra.run.smithi129.stdout:5377 write 0xc3041de thru 0xc30ccfc (0x8b1f bytes) 2024-09-17T21:30:51.632 INFO:teuthology.orchestra.run.smithi129.stdout:5378 trunc from 0xde468f3 to 0x16d0564 2024-09-17T21:30:51.658 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:51.658 INFO:teuthology.orchestra.run.smithi129.stdout:5380 read 0x67061c thru 0x6799a7 (0x938c bytes) 2024-09-17T21:30:51.659 INFO:teuthology.orchestra.run.smithi129.stdout:5383 read 0xbbe87a thru 0xbcaf9c (0xc723 bytes) 2024-09-17T21:30:51.666 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:51.666 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:51.671 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:51.669+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:51.673 INFO:teuthology.orchestra.run.smithi129.stdout:5384 write 0x8baa98d thru 0x8babad6 (0x114a bytes) 2024-09-17T21:30:51.677 INFO:teuthology.orchestra.run.smithi129.stdout:5385 read 0x2966e90 thru 0x29714c7 (0xa638 bytes) 2024-09-17T21:30:51.678 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:30:51.686 INFO:teuthology.orchestra.run.smithi129.stdout:5386 trunc from 0x8babad7 to 0xa8d9eb 2024-09-17T21:30:51.720 INFO:teuthology.orchestra.run.smithi129.stdout:5387 write 0xa00fd15 thru 0xa01482c (0x4b18 bytes) 2024-09-17T21:30:51.724 INFO:teuthology.orchestra.run.smithi129.stdout:5388 write 0x94a95ae thru 0x94b4638 (0xb08b bytes) 2024-09-17T21:30:51.729 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:51.729 INFO:teuthology.orchestra.run.smithi129.stdout:5391 punch from 0x984d29 to 0x98753a, (0x2811 bytes) 2024-09-17T21:30:51.729 INFO:teuthology.orchestra.run.smithi129.stdout:5392 writesame 0x55eb704 thru 0x55ecbac data_size 0x29(0x14a9 bytes) 2024-09-17T21:30:51.738 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:51.739 INFO:teuthology.orchestra.run.smithi129.stdout:5395 read 0x1811393 thru 0x1819228 (0x7e96 bytes) 2024-09-17T21:30:51.747 INFO:teuthology.orchestra.run.smithi129.stdout:5398 write 0xc4d5ab4 thru 0xc4df417 (0x9964 bytes) 2024-09-17T21:30:51.754 INFO:teuthology.orchestra.run.smithi129.stdout:5399 writesame 0xb750d8e thru 0xb754030 data_size 0x1d(0x32a3 bytes) 2024-09-17T21:30:51.762 INFO:teuthology.orchestra.run.smithi129.stdout:5400 read 0x45ddf thru 0x4bd14 (0x5f36 bytes) 2024-09-17T21:30:51.763 INFO:teuthology.orchestra.run.smithi129.stdout:5401 write 0x9b26eb1 thru 0x9b2c6e5 (0x5835 bytes) 2024-09-17T21:30:51.765 INFO:teuthology.orchestra.run.smithi129.stdout:5402 read 0xc1ece1b thru 0xc1f9362 (0xc548 bytes) 2024-09-17T21:30:51.765 INFO:teuthology.orchestra.run.smithi129.stdout:5403 writesame 0x90ced81 thru 0x90d89a3 data_size 0x1(0x9c23 bytes) 2024-09-17T21:30:51.772 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:51.770+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:51.780 INFO:teuthology.orchestra.run.smithi129.stdout:5404 trunc from 0xc4df418 to 0xd97ae91 2024-09-17T21:30:51.785 INFO:teuthology.orchestra.run.smithi129.stdout:5405 write 0x7bc9c8c thru 0x7bd2fe5 (0x935a bytes) 2024-09-17T21:30:51.791 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:51.791 INFO:teuthology.orchestra.run.smithi129.stdout:5409 write 0x2764b0a thru 0x276b082 (0x6579 bytes) 2024-09-17T21:30:51.795 INFO:teuthology.orchestra.run.smithi129.stdout:5410 write 0x7b40428 thru 0x7b4efde (0xebb7 bytes) 2024-09-17T21:30:51.797 INFO:teuthology.orchestra.run.smithi129.stdout:5411 read 0x882b391 thru 0x882ce86 (0x1af6 bytes) 2024-09-17T21:30:51.797 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:51.798 INFO:teuthology.orchestra.run.smithi129.stdout:5413 punch from 0xa64b76e to 0xa64da80, (0x2312 bytes) 2024-09-17T21:30:51.798 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:51.798 INFO:teuthology.orchestra.run.smithi129.stdout:5415 writesame 0x268e234 thru 0x269436b data_size 0x33(0x6138 bytes) 2024-09-17T21:30:51.804 INFO:teuthology.orchestra.run.smithi129.stdout:5416 trunc from 0xd97ae91 to 0xa2dc26d 2024-09-17T21:30:51.814 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:51.814 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:51.825 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:30:51.826 INFO:teuthology.orchestra.run.smithi129.stdout:5419 writesame 0xb37100e thru 0xb380bdd data_size 0x1(0xfbd0 bytes) 2024-09-17T21:30:51.826 INFO:teuthology.orchestra.run.smithi129.stdout:5420 write 0x3f605fd thru 0x3f60af6 (0x4fa bytes) 2024-09-17T21:30:51.828 INFO:teuthology.orchestra.run.smithi129.stdout:5422 write 0x8244e08 thru 0x8251874 (0xca6d bytes) 2024-09-17T21:30:51.830 INFO:teuthology.orchestra.run.smithi129.stdout:5423 writesame 0x8a26d2c thru 0x8a2c342 data_size 0x1(0x5617 bytes) 2024-09-17T21:30:51.851 INFO:teuthology.orchestra.run.smithi129.stdout:5424 trunc from 0xb380bde to 0x24695c7 2024-09-17T21:30:51.866 INFO:teuthology.orchestra.run.smithi129.stdout:5425 punch from 0x104ef3d to 0x10579d7, (0x8a9a bytes) 2024-09-17T21:30:51.866 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:51.873 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:51.871+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:51.887 INFO:teuthology.orchestra.run.smithi129.stderr:set osd.3 primary-affinity to 0.989723 (8648622) 2024-09-17T21:30:51.898 INFO:teuthology.orchestra.run.smithi129.stdout:5429 writesame 0xeadac56 thru 0xeae043f data_size 0x42(0x57ea bytes) 2024-09-17T21:30:51.905 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:30:51.915 INFO:teuthology.orchestra.run.smithi129.stdout:5430 trunc from 0xeae0440 to 0x64f8cf 2024-09-17T21:30:51.929 INFO:teuthology.orchestra.run.smithi129.stdout:5431 read 0x4c123 thru 0x50ec3 (0x4da1 bytes) 2024-09-17T21:30:51.935 INFO:teuthology.orchestra.run.smithi129.stdout:5433 trunc from 0x64f8cf to 0x4275097 2024-09-17T21:30:51.940 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:51.941 INFO:teuthology.orchestra.run.smithi129.stdout:5435 read 0x2a8190f thru 0x2a8355a (0x1c4c bytes) 2024-09-17T21:30:51.943 INFO:teuthology.orchestra.run.smithi129.stdout:5436 trunc from 0x4275097 to 0x29dc97a 2024-09-17T21:30:51.948 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:51.958 INFO:teuthology.orchestra.run.smithi129.stdout:5441 write 0x63abaf9 thru 0x63b7b4c (0xc054 bytes) 2024-09-17T21:30:51.965 INFO:teuthology.orchestra.run.smithi129.stdout:5443 write 0x12596 thru 0x1269f (0x10a bytes) 2024-09-17T21:30:51.973 INFO:teuthology.orchestra.run.smithi129.stdout:5444 writesame 0x7176e54 thru 0x7186d5e data_size 0x1(0xff0b bytes) 2024-09-17T21:30:51.974 INFO:teuthology.orchestra.run.smithi129.stdout:5445 read 0x2cd3653 thru 0x2cddf61 (0xa90f bytes) 2024-09-17T21:30:51.974 INFO:teuthology.orchestra.run.smithi129.stdout:5446 read 0x1a736b6 thru 0x1a73eb3 (0x7fe bytes) 2024-09-17T21:30:51.974 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:51.974 INFO:teuthology.orchestra.run.smithi129.stdout:5449 punch from 0x5892b9b to 0x589b011, (0x8476 bytes) 2024-09-17T21:30:51.974 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:51.971+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:51.974 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:51.974 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:51.988 INFO:teuthology.orchestra.run.smithi129.stdout:5450 write 0xd93f73a thru 0xd94917f (0x9a46 bytes) 2024-09-17T21:30:51.989 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:30:51.993 INFO:teuthology.orchestra.run.smithi129.stdout:5451 write 0xa76467 thru 0xa76e38 (0x9d2 bytes) 2024-09-17T21:30:51.995 INFO:teuthology.orchestra.run.smithi129.stdout:5452 write 0x7971d9c thru 0x7976602 (0x4867 bytes) 2024-09-17T21:30:51.999 INFO:teuthology.orchestra.run.smithi129.stdout:5453 write 0x53f8407 thru 0x53fa752 (0x234c bytes) 2024-09-17T21:30:52.003 INFO:teuthology.orchestra.run.smithi129.stdout:5454 punch from 0x5c72ca8 to 0x5c7cbcf, (0x9f27 bytes) 2024-09-17T21:30:52.003 INFO:teuthology.orchestra.run.smithi129.stdout:5456 read 0x8047951 thru 0x8048c64 (0x1314 bytes) 2024-09-17T21:30:52.003 INFO:teuthology.orchestra.run.smithi129.stdout:5459 read 0xb1a96f3 thru 0xb1b57aa (0xc0b8 bytes) 2024-09-17T21:30:52.007 INFO:teuthology.orchestra.run.smithi129.stdout:5460 trunc from 0xd949180 to 0xad30884 2024-09-17T21:30:52.018 INFO:teuthology.orchestra.run.smithi129.stdout:5461 punch from 0x5e67d28 to 0x5e685d9, (0x8b1 bytes) 2024-09-17T21:30:52.018 INFO:teuthology.orchestra.run.smithi129.stdout:5463 writesame 0x2364a77 thru 0x237460c data_size 0x1(0xfb96 bytes) 2024-09-17T21:30:52.039 INFO:teuthology.orchestra.run.smithi129.stdout:5465 punch from 0x5d044b9 to 0x5d0fca9, (0xb7f0 bytes) 2024-09-17T21:30:52.040 INFO:teuthology.orchestra.run.smithi129.stdout:5466 trunc from 0xad30884 to 0xa552d84 2024-09-17T21:30:52.046 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:52.054 INFO:teuthology.orchestra.run.smithi129.stdout:5468 trunc from 0xa552d84 to 0x4476f46 2024-09-17T21:30:52.074 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:52.072+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:52.084 INFO:teuthology.orchestra.run.smithi129.stdout:5469 writesame 0xea50dcc thru 0xea5ea1e data_size 0x9(0xdc53 bytes) 2024-09-17T21:30:52.099 INFO:teuthology.orchestra.run.smithi129.stdout:5470 trunc from 0xea5ea1f to 0x22144c8 2024-09-17T21:30:52.107 INFO:teuthology.orchestra.run.smithi129.stdout:5471 punch from 0x17eba03 to 0x17facf8, (0xf2f5 bytes) 2024-09-17T21:30:52.116 INFO:teuthology.orchestra.run.smithi129.stdout:5472 trunc from 0x22144c8 to 0x98cb21d 2024-09-17T21:30:52.122 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:52.122 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:52.123 INFO:teuthology.orchestra.run.smithi129.stdout:5473 write 0x243b3c1 thru 0x2448c0a (0xd84a bytes) 2024-09-17T21:30:52.130 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:52.134 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:30:52.135 INFO:teuthology.orchestra.run.smithi129.stdout:5475 trunc from 0x98cb21d to 0xd93e604 2024-09-17T21:30:52.143 INFO:teuthology.orchestra.run.smithi129.stdout:5476 punch from 0x79d3836 to 0x79df865, (0xc02f bytes) 2024-09-17T21:30:52.143 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:52.151 INFO:teuthology.orchestra.run.smithi129.stdout:5479 write 0xdc204b1 thru 0xdc2b869 (0xb3b9 bytes) 2024-09-17T21:30:52.162 INFO:teuthology.orchestra.run.smithi129.stdout:5480 punch from 0x4248d8d to 0x4249fad, (0x1220 bytes) 2024-09-17T21:30:52.162 INFO:teuthology.orchestra.run.smithi129.stdout:5481 clone 42 order 25 su 16777216 sc 7 2024-09-17T21:30:52.175 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:52.172+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:52.240 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:52.240 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:52.249 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:30:52.290 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:52.288+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:52.376 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:52.374+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:52.394 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:52.394 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:52.403 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:30:52.477 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:52.475+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:52.505 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:52.506 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:52.514 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:30:52.578 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:52.576+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:52.658 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:52.658 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:52.667 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:30:52.711 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:52.709+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:52.745 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:30:52.773 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:52.773 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:52.781 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:52.778+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:52.782 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:30:52.864 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:52.864 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:52.881 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:30:52.881 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:30:52.881 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:52.879+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:52.897 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is set 2024-09-17T21:30:52.898 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:52.898 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:52.918 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:52.918 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:52.931 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:52.931 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:52.941 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:30:52.967 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone41 from 0xdc2b86a (overlap 0x64f8cf) to 0x327c67 2024-09-17T21:30:52.982 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:52.980+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:53.042 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-17T21:30:53.083 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:53.081+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:53.090 INFO:teuthology.orchestra.run.smithi129.stdout:5482 write 0x596f71e thru 0x59738a2 (0x4185 bytes) 2024-09-17T21:30:53.096 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:53.096 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:53.106 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:30:53.108 INFO:teuthology.orchestra.run.smithi129.stdout:5483 trunc from 0xdc2b86a to 0x586316b 2024-09-17T21:30:53.136 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:53.137 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:53.147 INFO:teuthology.orchestra.run.smithi129.stdout:5484 writesame 0x74a5a45 thru 0x74b3d94 data_size 0x8(0xe350 bytes) 2024-09-17T21:30:53.149 INFO:teuthology.orchestra.run.smithi129.stdout:5485 punch from 0x25d5a79 to 0x25e2a5b, (0xcfe2 bytes) 2024-09-17T21:30:53.149 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:53.149 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:53.149 INFO:teuthology.orchestra.run.smithi129.stdout:5488 write 0x4b48271 thru 0x4b500fd (0x7e8d bytes) 2024-09-17T21:30:53.150 INFO:teuthology.orchestra.run.smithi129.stdout:5489 write 0x3f23ec6 thru 0x3f2a1d8 (0x6313 bytes) 2024-09-17T21:30:53.156 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:53.156 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:53.159 INFO:teuthology.orchestra.run.smithi129.stdout:5491 write 0x88e5028 thru 0x88ee042 (0x901b bytes) 2024-09-17T21:30:53.170 INFO:teuthology.orchestra.run.smithi129.stdout:5494 writesame 0xc4661e3 thru 0xc46c6ea data_size 0x6a(0x6508 bytes) 2024-09-17T21:30:53.172 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:53.172 INFO:teuthology.orchestra.run.smithi129.stdout:5496 clone 43 order 22 su 131072 sc 11 2024-09-17T21:30:53.176 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:53.176 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:53.184 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:53.181+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:53.215 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:53.216 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:53.216 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:53.224 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:30:53.308 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:53.306+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:53.369 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:53.369 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:53.377 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:30:53.392 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:53.393 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:53.418 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:53.416+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:53.486 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:53.483+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:53.523 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10799 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:30:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.454442216000004, 2024-09-17T21:30:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039122745, 2024-09-17T21:30:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10799 2024-09-17T21:30:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:30:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:30:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065469+0000", 2024-09-17T21:30:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:30:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065477+0000", 2024-09-17T21:30:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1319999999999994e-06 2024-09-17T21:30:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065483+0000", 2024-09-17T21:30:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2040000000000002e-06 2024-09-17T21:30:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065487+0000", 2024-09-17T21:30:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.42e-06 2024-09-17T21:30:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099984+0000", 2024-09-17T21:30:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034496484000000001 2024-09-17T21:30:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100065+0000", 2024-09-17T21:30:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1200999999999997e-05 2024-09-17T21:30:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100422+0000", 2024-09-17T21:30:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035657700000000001 2024-09-17T21:30:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104214+0000", 2024-09-17T21:30:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003792045 2024-09-17T21:30:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104505+0000", 2024-09-17T21:30:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029141399999999999 2024-09-17T21:30:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104556+0000", 2024-09-17T21:30:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1388999999999999e-05 2024-09-17T21:30:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104591+0000", 2024-09-17T21:30:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4455999999999997e-05 2024-09-17T21:30:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12187 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.380349484999996, 2024-09-17T21:30:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038825750999999999, 2024-09-17T21:30:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12187 2024-09-17T21:30:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:30:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:30:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139563+0000", 2024-09-17T21:30:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.624e-06 2024-09-17T21:30:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139569+0000", 2024-09-17T21:30:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1090000000000003e-06 2024-09-17T21:30:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139573+0000", 2024-09-17T21:30:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4429999999999996e-06 2024-09-17T21:30:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178206+0000", 2024-09-17T21:30:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038632376000000003 2024-09-17T21:30:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178282+0000", 2024-09-17T21:30:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6076999999999997e-05 2024-09-17T21:30:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178387+0000", 2024-09-17T21:30:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104682 2024-09-17T21:30:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12188 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:30:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.380331303999995, 2024-09-17T21:30:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039053826999999999, 2024-09-17T21:30:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12188 2024-09-17T21:30:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:30:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:30:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139746+0000", 2024-09-17T21:30:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000167281 2024-09-17T21:30:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139749+0000", 2024-09-17T21:30:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9660000000000001e-06 2024-09-17T21:30:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139757+0000", 2024-09-17T21:30:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8029999999999997e-06 2024-09-17T21:30:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139771+0000", 2024-09-17T21:30:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3541e-05 2024-09-17T21:30:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178415+0000", 2024-09-17T21:30:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038644803999999998 2024-09-17T21:30:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178517+0000", 2024-09-17T21:30:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101298 2024-09-17T21:30:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178633+0000", 2024-09-17T21:30:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116134 2024-09-17T21:30:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12189 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:30:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.380130827000002, 2024-09-17T21:30:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039135991000000002, 2024-09-17T21:30:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12189 2024-09-17T21:30:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:30:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:30:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139780+0000", 2024-09-17T21:30:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:30:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139782+0000", 2024-09-17T21:30:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1950000000000002e-06 2024-09-17T21:30:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139788+0000", 2024-09-17T21:30:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0639999999999999e-06 2024-09-17T21:30:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139792+0000", 2024-09-17T21:30:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1960000000000001e-06 2024-09-17T21:30:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178671+0000", 2024-09-17T21:30:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038878176 2024-09-17T21:30:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178778+0000", 2024-09-17T21:30:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107278 2024-09-17T21:30:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178915+0000", 2024-09-17T21:30:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013757900000000001 2024-09-17T21:30:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12190 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.380111643999996, 2024-09-17T21:30:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039343781000000001, 2024-09-17T21:30:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12190 2024-09-17T21:30:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:30:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139801+0000", 2024-09-17T21:30:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.916e-06 2024-09-17T21:30:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139807+0000", 2024-09-17T21:30:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4540000000000004e-06 2024-09-17T21:30:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139812+0000", 2024-09-17T21:30:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9729999999999997e-06 2024-09-17T21:30:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178960+0000", 2024-09-17T21:30:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039147567000000001 2024-09-17T21:30:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179051+0000", 2024-09-17T21:30:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0754000000000006e-05 2024-09-17T21:30:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179142+0000", 2024-09-17T21:30:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1712999999999994e-05 2024-09-17T21:30:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12191 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.380091785999994, 2024-09-17T21:30:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039464336000000003, 2024-09-17T21:30:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12191 2024-09-17T21:30:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:53.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:53.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:30:53.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:30:53.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139821+0000", 2024-09-17T21:30:53.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1260000000000001e-06 2024-09-17T21:30:53.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139827+0000", 2024-09-17T21:30:53.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T21:30:53.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139840+0000", 2024-09-17T21:30:53.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2607e-05 2024-09-17T21:30:53.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179162+0000", 2024-09-17T21:30:53.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039322863999999999 2024-09-17T21:30:53.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179204+0000", 2024-09-17T21:30:53.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1938999999999999e-05 2024-09-17T21:30:53.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179283+0000", 2024-09-17T21:30:53.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8547000000000005e-05 2024-09-17T21:30:53.553 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.553 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.553 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.553 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12192 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.553 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:30:53.554 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.380064074000003, 2024-09-17T21:30:53.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039652318999999998, 2024-09-17T21:30:53.554 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.554 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.554 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:53.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.554 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12192 2024-09-17T21:30:53.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.554 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:30:53.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:30:53.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141458+0000", 2024-09-17T21:30:53.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001612098 2024-09-17T21:30:53.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141469+0000", 2024-09-17T21:30:53.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-17T21:30:53.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141478+0000", 2024-09-17T21:30:53.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T21:30:53.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141485+0000", 2024-09-17T21:30:53.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4760000000000003e-06 2024-09-17T21:30:53.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179317+0000", 2024-09-17T21:30:53.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037831832000000003 2024-09-17T21:30:53.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179395+0000", 2024-09-17T21:30:53.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8349000000000005e-05 2024-09-17T21:30:53.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179499+0000", 2024-09-17T21:30:53.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103628 2024-09-17T21:30:53.558 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.558 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.558 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.558 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12257 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.558 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:53.558 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.377875266000004, 2024-09-17T21:30:53.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038862751000000001, 2024-09-17T21:30:53.558 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.558 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.558 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:53.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.559 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.559 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.559 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12257 2024-09-17T21:30:53.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.559 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:53.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:53.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:30:53.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T21:30:53.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142037+0000", 2024-09-17T21:30:53.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.03e-06 2024-09-17T21:30:53.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142043+0000", 2024-09-17T21:30:53.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6319999999999996e-06 2024-09-17T21:30:53.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142052+0000", 2024-09-17T21:30:53.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2709999999999998e-06 2024-09-17T21:30:53.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180794+0000", 2024-09-17T21:30:53.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038741758000000001 2024-09-17T21:30:53.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180843+0000", 2024-09-17T21:30:53.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9021999999999999e-05 2024-09-17T21:30:53.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180898+0000", 2024-09-17T21:30:53.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673000000000003e-05 2024-09-17T21:30:53.562 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.562 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.562 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.562 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12258 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.563 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:53.563 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.377851749000001, 2024-09-17T21:30:53.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038950717000000003, 2024-09-17T21:30:53.563 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.563 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.563 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:53.563 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.563 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.563 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.563 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12258 2024-09-17T21:30:53.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.563 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:53.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:53.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:30:53.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199999999999999e-07 2024-09-17T21:30:53.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142061+0000", 2024-09-17T21:30:53.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8279999999999999e-06 2024-09-17T21:30:53.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142067+0000", 2024-09-17T21:30:53.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5600000000000001e-06 2024-09-17T21:30:53.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142071+0000", 2024-09-17T21:30:53.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6240000000000001e-06 2024-09-17T21:30:53.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180921+0000", 2024-09-17T21:30:53.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038849366000000003 2024-09-17T21:30:53.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180961+0000", 2024-09-17T21:30:53.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0219000000000002e-05 2024-09-17T21:30:53.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181009+0000", 2024-09-17T21:30:53.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8437999999999999e-05 2024-09-17T21:30:53.567 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.567 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.567 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.568 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12259 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.568 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:53.568 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.377824501999996, 2024-09-17T21:30:53.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039116832999999997, 2024-09-17T21:30:53.568 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.568 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.568 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:53.568 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.568 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.568 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.568 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12259 2024-09-17T21:30:53.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.568 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:53.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:53.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:30:53.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2200000000000004e-07 2024-09-17T21:30:53.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142088+0000", 2024-09-17T21:30:53.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0329999999999998e-06 2024-09-17T21:30:53.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142095+0000", 2024-09-17T21:30:53.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4320000000000004e-06 2024-09-17T21:30:53.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142100+0000", 2024-09-17T21:30:53.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:30:53.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181037+0000", 2024-09-17T21:30:53.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038937601000000002 2024-09-17T21:30:53.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181114+0000", 2024-09-17T21:30:53.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6857e-05 2024-09-17T21:30:53.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181203+0000", 2024-09-17T21:30:53.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8437999999999996e-05 2024-09-17T21:30:53.572 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.572 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.572 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.572 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12260 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.572 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:30:53.573 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.377804214999998, 2024-09-17T21:30:53.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039204184000000003, 2024-09-17T21:30:53.573 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.573 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.573 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:53.573 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.573 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.573 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.573 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12260 2024-09-17T21:30:53.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.573 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:30:53.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:30:53.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142107+0000", 2024-09-17T21:30:53.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:30:53.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142108+0000", 2024-09-17T21:30:53.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.753e-06 2024-09-17T21:30:53.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142114+0000", 2024-09-17T21:30:53.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7660000000000004e-06 2024-09-17T21:30:53.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142119+0000", 2024-09-17T21:30:53.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:30:53.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181216+0000", 2024-09-17T21:30:53.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039097191000000003 2024-09-17T21:30:53.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181260+0000", 2024-09-17T21:30:53.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4731000000000003e-05 2024-09-17T21:30:53.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181310+0000", 2024-09-17T21:30:53.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9818000000000003e-05 2024-09-17T21:30:53.577 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.577 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.577 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.577 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12261 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.577 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:30:53.577 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.377785891000002, 2024-09-17T21:30:53.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039292200999999999, 2024-09-17T21:30:53.578 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.578 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.578 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:53.578 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.578 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.578 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.578 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12261 2024-09-17T21:30:53.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.578 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:30:53.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:30:53.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142125+0000", 2024-09-17T21:30:53.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:30:53.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142127+0000", 2024-09-17T21:30:53.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.792e-06 2024-09-17T21:30:53.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142132+0000", 2024-09-17T21:30:53.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3979999999999998e-06 2024-09-17T21:30:53.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142136+0000", 2024-09-17T21:30:53.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.211e-06 2024-09-17T21:30:53.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181324+0000", 2024-09-17T21:30:53.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039187560000000003 2024-09-17T21:30:53.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181363+0000", 2024-09-17T21:30:53.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9375000000000002e-05 2024-09-17T21:30:53.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181417+0000", 2024-09-17T21:30:53.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3372000000000003e-05 2024-09-17T21:30:53.582 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.582 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.582 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.582 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12262 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.582 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:30:53.582 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.377768098000004, 2024-09-17T21:30:53.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039384875, 2024-09-17T21:30:53.582 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.582 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.582 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:53.582 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.582 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.583 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.583 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12262 2024-09-17T21:30:53.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.583 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:30:53.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:30:53.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142143+0000", 2024-09-17T21:30:53.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:30:53.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142148+0000", 2024-09-17T21:30:53.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4360000000000001e-06 2024-09-17T21:30:53.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142154+0000", 2024-09-17T21:30:53.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8719999999999999e-06 2024-09-17T21:30:53.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142158+0000", 2024-09-17T21:30:53.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.476e-06 2024-09-17T21:30:53.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181436+0000", 2024-09-17T21:30:53.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039277919000000001 2024-09-17T21:30:53.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181480+0000", 2024-09-17T21:30:53.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3649999999999997e-05 2024-09-17T21:30:53.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181527+0000", 2024-09-17T21:30:53.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7131e-05 2024-09-17T21:30:53.586 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.586 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.586 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.586 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12263 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1079296~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.586 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:53.587 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.377745662999999, 2024-09-17T21:30:53.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039473339000000003, 2024-09-17T21:30:53.587 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.587 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.587 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:53.587 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.587 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.587 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.587 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12263 2024-09-17T21:30:53.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.587 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:53.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:53.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:30:53.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:30:53.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142167+0000", 2024-09-17T21:30:53.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.81e-06 2024-09-17T21:30:53.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142172+0000", 2024-09-17T21:30:53.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:30:53.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142177+0000", 2024-09-17T21:30:53.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T21:30:53.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181553+0000", 2024-09-17T21:30:53.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039376130000000002 2024-09-17T21:30:53.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181592+0000", 2024-09-17T21:30:53.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9153999999999998e-05 2024-09-17T21:30:53.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181638+0000", 2024-09-17T21:30:53.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5864e-05 2024-09-17T21:30:53.591 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.591 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.591 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.591 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12278 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1095680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.591 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:53.591 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.377727687999993, 2024-09-17T21:30:53.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039454222999999997, 2024-09-17T21:30:53.591 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.591 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.591 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:53.591 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.592 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.592 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.592 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12278 2024-09-17T21:30:53.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.592 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:53.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:53.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:30:53.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:30:53.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142185+0000", 2024-09-17T21:30:53.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.697e-06 2024-09-17T21:30:53.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142191+0000", 2024-09-17T21:30:53.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:30:53.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142196+0000", 2024-09-17T21:30:53.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4829999999999999e-06 2024-09-17T21:30:53.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181548+0000", 2024-09-17T21:30:53.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039351200000000003 2024-09-17T21:30:53.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181591+0000", 2024-09-17T21:30:53.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3492000000000003e-05 2024-09-17T21:30:53.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181637+0000", 2024-09-17T21:30:53.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5847000000000003e-05 2024-09-17T21:30:53.595 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.595 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.595 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.596 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12279 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.596 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:53.596 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.377704398999995, 2024-09-17T21:30:53.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039539801999999999, 2024-09-17T21:30:53.596 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.596 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.596 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:53.596 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.596 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.596 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.596 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12279 2024-09-17T21:30:53.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.596 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:53.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:53.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:30:53.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.89e-07 2024-09-17T21:30:53.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142208+0000", 2024-09-17T21:30:53.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7579999999999999e-06 2024-09-17T21:30:53.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142214+0000", 2024-09-17T21:30:53.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:30:53.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142218+0000", 2024-09-17T21:30:53.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3429999999999998e-06 2024-09-17T21:30:53.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181653+0000", 2024-09-17T21:30:53.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039435322000000002 2024-09-17T21:30:53.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181698+0000", 2024-09-17T21:30:53.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4978999999999997e-05 2024-09-17T21:30:53.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181746+0000", 2024-09-17T21:30:53.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571999999999998e-05 2024-09-17T21:30:53.600 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.600 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.600 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.600 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12280 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1124352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.600 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:30:53.600 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.377686879999999, 2024-09-17T21:30:53.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039704358000000002, 2024-09-17T21:30:53.600 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.600 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.601 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:53.601 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.601 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.601 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.601 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12280 2024-09-17T21:30:53.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.601 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:30:53.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:30:53.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142224+0000", 2024-09-17T21:30:53.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:30:53.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142226+0000", 2024-09-17T21:30:53.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:30:53.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142231+0000", 2024-09-17T21:30:53.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:30:53.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142240+0000", 2024-09-17T21:30:53.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4590000000000004e-06 2024-09-17T21:30:53.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181774+0000", 2024-09-17T21:30:53.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039533833999999997 2024-09-17T21:30:53.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181859+0000", 2024-09-17T21:30:53.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4328999999999998e-05 2024-09-17T21:30:53.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181928+0000", 2024-09-17T21:30:53.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9139000000000003e-05 2024-09-17T21:30:53.604 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.604 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.604 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.605 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12281 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1136640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.605 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:30:53.605 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.377663974000001, 2024-09-17T21:30:53.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039787799999999998, 2024-09-17T21:30:53.605 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.605 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.605 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:53.605 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.605 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.605 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.605 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12281 2024-09-17T21:30:53.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.606 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:30:53.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:30:53.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142309+0000", 2024-09-17T21:30:53.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2323999999999995e-05 2024-09-17T21:30:53.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142317+0000", 2024-09-17T21:30:53.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1859999999999992e-06 2024-09-17T21:30:53.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142324+0000", 2024-09-17T21:30:53.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4699999999999996e-06 2024-09-17T21:30:53.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142330+0000", 2024-09-17T21:30:53.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4650000000000002e-06 2024-09-17T21:30:53.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181948+0000", 2024-09-17T21:30:53.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039618056999999998 2024-09-17T21:30:53.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181986+0000", 2024-09-17T21:30:53.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8442999999999998e-05 2024-09-17T21:30:53.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182034+0000", 2024-09-17T21:30:53.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7855000000000002e-05 2024-09-17T21:30:53.609 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.609 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.609 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.609 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12282 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1150976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.609 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:53.609 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.377571744999997, 2024-09-17T21:30:53.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039871919999999998, 2024-09-17T21:30:53.609 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.609 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.610 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:53.610 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.610 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.610 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.610 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12282 2024-09-17T21:30:53.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.610 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:53.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:53.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:30:53.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:30:53.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142341+0000", 2024-09-17T21:30:53.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.232e-06 2024-09-17T21:30:53.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142348+0000", 2024-09-17T21:30:53.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7320000000000001e-06 2024-09-17T21:30:53.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142353+0000", 2024-09-17T21:30:53.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:30:53.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182062+0000", 2024-09-17T21:30:53.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039709739000000001 2024-09-17T21:30:53.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:53.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182128+0000", 2024-09-17T21:30:53.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5739000000000002e-05 2024-09-17T21:30:53.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182210+0000", 2024-09-17T21:30:53.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:30:53.613 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.613 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.613 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.614 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12283 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:53.614 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:30:53.614 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.377551334000003, 2024-09-17T21:30:53.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039896434000000001, 2024-09-17T21:30:53.614 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.614 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.614 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:30:53.614 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.614 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.614 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.614 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12283 2024-09-17T21:30:53.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.615 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:30:53.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:30:53.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144852+0000", 2024-09-17T21:30:53.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024934079999999999 2024-09-17T21:30:53.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144856+0000", 2024-09-17T21:30:53.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:30:53.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144864+0000", 2024-09-17T21:30:53.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0160000000000008e-06 2024-09-17T21:30:53.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144869+0000", 2024-09-17T21:30:53.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:30:53.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:53.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182246+0000", 2024-09-17T21:30:53.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037377207000000003 2024-09-17T21:30:53.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:53.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182255+0000", 2024-09-17T21:30:53.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1840000000000005e-06 2024-09-17T21:30:53.618 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.618 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.618 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.618 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:53.618 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:53.618 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:53.619 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:30:53.620 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:53.617+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:53.721 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:53.719+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:53.723 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:53.723 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:53.723 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:53.723 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:53.732 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:30:53.821 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:53.819+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:53.834 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:53.834 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:53.834 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:53.834 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:53.834 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:53.843 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:30:53.927 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:53.925+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:53.979 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone42 from 0xc46c6eb (overlap 0x586316b) to 0x0 2024-09-17T21:30:53.995 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:53.995 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:53.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:53.996 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:53.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.996 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:36935 3.a 3:54c8759b:::rbd_data.1120e62eea89.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-17T21:30:53.996 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:53.996 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.033796381000002, 2024-09-17T21:30:53.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020540151999999999, 2024-09-17T21:30:53.996 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:53.996 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:53.996 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:53.996 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:53.996 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:53.997 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:53.997 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 36935 2024-09-17T21:30:53.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.997 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:53.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:53.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:53.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:53.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:30:53.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:53.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:53.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958910+0000", 2024-09-17T21:30:53.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:30:53.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:53.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958918+0000", 2024-09-17T21:30:53.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6469999999999998e-06 2024-09-17T21:30:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958933+0000", 2024-09-17T21:30:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4791e-05 2024-09-17T21:30:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958940+0000", 2024-09-17T21:30:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-17T21:30:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958990+0000", 2024-09-17T21:30:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9759000000000003e-05 2024-09-17T21:30:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.959039+0000", 2024-09-17T21:30:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9487000000000002e-05 2024-09-17T21:30:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.962098+0000", 2024-09-17T21:30:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003059258 2024-09-17T21:30:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979325+0000", 2024-09-17T21:30:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017226841 2024-09-17T21:30:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979343+0000", 2024-09-17T21:30:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7486000000000002e-05 2024-09-17T21:30:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979401+0000", 2024-09-17T21:30:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7980999999999999e-05 2024-09-17T21:30:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979449+0000", 2024-09-17T21:30:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8639e-05 2024-09-17T21:30:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39447 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 909312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024858481000003, 2024-09-17T21:30:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017419153999999999, 2024-09-17T21:30:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39447 2024-09-17T21:30:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:30:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:30:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967849+0000", 2024-09-17T21:30:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4339999999999999e-06 2024-09-17T21:30:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967853+0000", 2024-09-17T21:30:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:30:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967857+0000", 2024-09-17T21:30:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:30:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985193+0000", 2024-09-17T21:30:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017335973000000001 2024-09-17T21:30:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985228+0000", 2024-09-17T21:30:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5154999999999998e-05 2024-09-17T21:30:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985266+0000", 2024-09-17T21:30:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8448000000000002e-05 2024-09-17T21:30:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39448 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 974848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024845665000001, 2024-09-17T21:30:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017517066000000001, 2024-09-17T21:30:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39448 2024-09-17T21:30:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:30:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.72e-07 2024-09-17T21:30:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967861+0000", 2024-09-17T21:30:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.096e-06 2024-09-17T21:30:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967865+0000", 2024-09-17T21:30:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9339999999999999e-06 2024-09-17T21:30:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967868+0000", 2024-09-17T21:30:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.508e-06 2024-09-17T21:30:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985285+0000", 2024-09-17T21:30:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017417208 2024-09-17T21:30:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985339+0000", 2024-09-17T21:30:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3931000000000003e-05 2024-09-17T21:30:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985377+0000", 2024-09-17T21:30:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8016999999999999e-05 2024-09-17T21:30:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39449 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024833555000001, 2024-09-17T21:30:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017595137, 2024-09-17T21:30:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39449 2024-09-17T21:30:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:30:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2700000000000001e-07 2024-09-17T21:30:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967873+0000", 2024-09-17T21:30:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.164e-06 2024-09-17T21:30:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967877+0000", 2024-09-17T21:30:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.23e-06 2024-09-17T21:30:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967880+0000", 2024-09-17T21:30:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.286e-06 2024-09-17T21:30:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985397+0000", 2024-09-17T21:30:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017516982 2024-09-17T21:30:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985431+0000", 2024-09-17T21:30:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3899999999999997e-05 2024-09-17T21:30:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985467+0000", 2024-09-17T21:30:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6347999999999998e-05 2024-09-17T21:30:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.019 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.019 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.019 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39450 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1105920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.019 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:54.019 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024813297999998, 2024-09-17T21:30:54.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017677824000000002, 2024-09-17T21:30:54.019 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.019 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.019 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.020 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.020 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.020 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.020 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39450 2024-09-17T21:30:54.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.020 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:54.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:30:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967893+0000", 2024-09-17T21:30:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9100000000000004e-07 2024-09-17T21:30:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967895+0000", 2024-09-17T21:30:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:30:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967901+0000", 2024-09-17T21:30:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.083e-06 2024-09-17T21:30:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967906+0000", 2024-09-17T21:30:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:30:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985493+0000", 2024-09-17T21:30:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017587565999999999 2024-09-17T21:30:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985531+0000", 2024-09-17T21:30:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7490000000000002e-05 2024-09-17T21:30:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985570+0000", 2024-09-17T21:30:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9205000000000002e-05 2024-09-17T21:30:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39451 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1171456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024792918000003, 2024-09-17T21:30:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017752337999999999, 2024-09-17T21:30:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39451 2024-09-17T21:30:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:30:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:30:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967915+0000", 2024-09-17T21:30:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780000000000001e-06 2024-09-17T21:30:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967922+0000", 2024-09-17T21:30:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7020000000000004e-06 2024-09-17T21:30:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967926+0000", 2024-09-17T21:30:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.809e-06 2024-09-17T21:30:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985590+0000", 2024-09-17T21:30:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017663913999999999 2024-09-17T21:30:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985627+0000", 2024-09-17T21:30:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6896e-05 2024-09-17T21:30:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985665+0000", 2024-09-17T21:30:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7793999999999998e-05 2024-09-17T21:30:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39452 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1236992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024772976000001, 2024-09-17T21:30:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017829602999999999, 2024-09-17T21:30:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.031 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.031 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.031 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39452 2024-09-17T21:30:54.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.031 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:54.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:30:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:30:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967947+0000", 2024-09-17T21:30:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4280999999999999e-05 2024-09-17T21:30:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967954+0000", 2024-09-17T21:30:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3249999999999996e-06 2024-09-17T21:30:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967959+0000", 2024-09-17T21:30:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2299999999999999e-06 2024-09-17T21:30:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985687+0000", 2024-09-17T21:30:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017727844999999999 2024-09-17T21:30:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985725+0000", 2024-09-17T21:30:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8243e-05 2024-09-17T21:30:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985762+0000", 2024-09-17T21:30:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7023000000000001e-05 2024-09-17T21:30:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39453 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024736488000002, 2024-09-17T21:30:54.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017893793000000002, 2024-09-17T21:30:54.036 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.036 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.036 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.036 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.036 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.036 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.036 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39453 2024-09-17T21:30:54.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.036 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:30:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967970+0000", 2024-09-17T21:30:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:30:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967972+0000", 2024-09-17T21:30:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0229999999999999e-06 2024-09-17T21:30:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967978+0000", 2024-09-17T21:30:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0430000000000004e-06 2024-09-17T21:30:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967982+0000", 2024-09-17T21:30:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2570000000000001e-06 2024-09-17T21:30:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985782+0000", 2024-09-17T21:30:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017799683 2024-09-17T21:30:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985818+0000", 2024-09-17T21:30:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6304999999999999e-05 2024-09-17T21:30:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985863+0000", 2024-09-17T21:30:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4953999999999997e-05 2024-09-17T21:30:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39454 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1368064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024717160000002, 2024-09-17T21:30:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017968886, 2024-09-17T21:30:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39454 2024-09-17T21:30:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:30:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967989+0000", 2024-09-17T21:30:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.15e-07 2024-09-17T21:30:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967991+0000", 2024-09-17T21:30:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.945e-06 2024-09-17T21:30:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967997+0000", 2024-09-17T21:30:54.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0190000000000003e-06 2024-09-17T21:30:54.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968001+0000", 2024-09-17T21:30:54.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6650000000000002e-06 2024-09-17T21:30:54.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985883+0000", 2024-09-17T21:30:54.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017881971999999999 2024-09-17T21:30:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985920+0000", 2024-09-17T21:30:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-05 2024-09-17T21:30:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985957+0000", 2024-09-17T21:30:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7370000000000003e-05 2024-09-17T21:30:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39455 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024697986, 2024-09-17T21:30:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018044437999999999, 2024-09-17T21:30:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.047 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.047 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39455 2024-09-17T21:30:54.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.047 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:54.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:54.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:30:54.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:30:54.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968010+0000", 2024-09-17T21:30:54.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.886e-06 2024-09-17T21:30:54.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968017+0000", 2024-09-17T21:30:54.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:30:54.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968021+0000", 2024-09-17T21:30:54.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850000000000004e-06 2024-09-17T21:30:54.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985977+0000", 2024-09-17T21:30:54.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017956368 2024-09-17T21:30:54.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986015+0000", 2024-09-17T21:30:54.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7610000000000001e-05 2024-09-17T21:30:54.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986052+0000", 2024-09-17T21:30:54.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7109e-05 2024-09-17T21:30:54.051 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.051 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.051 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.051 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39456 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1499136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.051 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:54.052 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024677685999997, 2024-09-17T21:30:54.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018114514000000002, 2024-09-17T21:30:54.052 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.052 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.052 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.052 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.052 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.052 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.052 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39456 2024-09-17T21:30:54.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.052 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:54.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:54.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:30:54.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-17T21:30:54.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968030+0000", 2024-09-17T21:30:54.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0049999999999999e-06 2024-09-17T21:30:54.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968036+0000", 2024-09-17T21:30:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0129999999999999e-06 2024-09-17T21:30:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968041+0000", 2024-09-17T21:30:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8300000000000003e-06 2024-09-17T21:30:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986071+0000", 2024-09-17T21:30:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018029620999999999 2024-09-17T21:30:54.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986106+0000", 2024-09-17T21:30:54.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4867999999999999e-05 2024-09-17T21:30:54.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986142+0000", 2024-09-17T21:30:54.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6566000000000002e-05 2024-09-17T21:30:54.056 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.056 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.057 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.057 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39457 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1564672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.057 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:54.057 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024657869999999, 2024-09-17T21:30:54.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018188644, 2024-09-17T21:30:54.057 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.057 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.057 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.057 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39457 2024-09-17T21:30:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:30:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7199999999999999e-07 2024-09-17T21:30:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968050+0000", 2024-09-17T21:30:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1110000000000002e-06 2024-09-17T21:30:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968057+0000", 2024-09-17T21:30:54.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6170000000000003e-06 2024-09-17T21:30:54.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968062+0000", 2024-09-17T21:30:54.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:30:54.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986163+0000", 2024-09-17T21:30:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018101435999999999 2024-09-17T21:30:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986199+0000", 2024-09-17T21:30:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.591e-05 2024-09-17T21:30:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986236+0000", 2024-09-17T21:30:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7416999999999998e-05 2024-09-17T21:30:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39458 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1630208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024630930000001, 2024-09-17T21:30:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018255856000000001, 2024-09-17T21:30:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39458 2024-09-17T21:30:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:30:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:30:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968077+0000", 2024-09-17T21:30:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.779e-06 2024-09-17T21:30:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968081+0000", 2024-09-17T21:30:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.532e-06 2024-09-17T21:30:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968085+0000", 2024-09-17T21:30:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999999e-06 2024-09-17T21:30:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986256+0000", 2024-09-17T21:30:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018171273000000002 2024-09-17T21:30:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986293+0000", 2024-09-17T21:30:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6534e-05 2024-09-17T21:30:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986330+0000", 2024-09-17T21:30:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7536000000000001e-05 2024-09-17T21:30:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39459 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1695744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024616352000002, 2024-09-17T21:30:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334493, 2024-09-17T21:30:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39459 2024-09-17T21:30:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:30:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968090+0000", 2024-09-17T21:30:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:30:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968091+0000", 2024-09-17T21:30:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3319999999999999e-06 2024-09-17T21:30:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968094+0000", 2024-09-17T21:30:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2590000000000001e-06 2024-09-17T21:30:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968097+0000", 2024-09-17T21:30:54.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:30:54.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986350+0000", 2024-09-17T21:30:54.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018253364000000001 2024-09-17T21:30:54.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986387+0000", 2024-09-17T21:30:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6350999999999998e-05 2024-09-17T21:30:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986424+0000", 2024-09-17T21:30:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6897999999999998e-05 2024-09-17T21:30:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39460 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1761280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024605215999998, 2024-09-17T21:30:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018415918999999999, 2024-09-17T21:30:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39460 2024-09-17T21:30:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:54.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:30:54.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968101+0000", 2024-09-17T21:30:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T21:30:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968102+0000", 2024-09-17T21:30:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1689999999999999e-06 2024-09-17T21:30:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968105+0000", 2024-09-17T21:30:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:30:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968108+0000", 2024-09-17T21:30:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6630000000000002e-06 2024-09-17T21:30:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986442+0000", 2024-09-17T21:30:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334487 2024-09-17T21:30:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986479+0000", 2024-09-17T21:30:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6557000000000003e-05 2024-09-17T21:30:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986516+0000", 2024-09-17T21:30:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7648999999999998e-05 2024-09-17T21:30:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39461 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1826816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024594456000003, 2024-09-17T21:30:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018499768, 2024-09-17T21:30:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39461 2024-09-17T21:30:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:30:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4699999999999998e-07 2024-09-17T21:30:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968112+0000", 2024-09-17T21:30:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-06 2024-09-17T21:30:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968116+0000", 2024-09-17T21:30:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1080000000000001e-06 2024-09-17T21:30:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968122+0000", 2024-09-17T21:30:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1070000000000001e-06 2024-09-17T21:30:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986537+0000", 2024-09-17T21:30:54.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018414834000000001 2024-09-17T21:30:54.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986572+0000", 2024-09-17T21:30:54.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5726000000000003e-05 2024-09-17T21:30:54.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986611+0000", 2024-09-17T21:30:54.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8584000000000002e-05 2024-09-17T21:30:54.082 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39462 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1892352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024580092000001, 2024-09-17T21:30:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018583252000000001, 2024-09-17T21:30:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39462 2024-09-17T21:30:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:30:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968126+0000", 2024-09-17T21:30:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968127+0000", 2024-09-17T21:30:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.229e-06 2024-09-17T21:30:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968131+0000", 2024-09-17T21:30:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8010000000000001e-06 2024-09-17T21:30:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968134+0000", 2024-09-17T21:30:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8880000000000001e-06 2024-09-17T21:30:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986634+0000", 2024-09-17T21:30:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018500501999999999 2024-09-17T21:30:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986671+0000", 2024-09-17T21:30:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6881000000000001e-05 2024-09-17T21:30:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986709+0000", 2024-09-17T21:30:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7546999999999998e-05 2024-09-17T21:30:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39463 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1957888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024568477000003, 2024-09-17T21:30:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018667186999999998, 2024-09-17T21:30:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39463 2024-09-17T21:30:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:30:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:30:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968138+0000", 2024-09-17T21:30:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:30:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968142+0000", 2024-09-17T21:30:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.145e-06 2024-09-17T21:30:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968144+0000", 2024-09-17T21:30:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6869999999999999e-06 2024-09-17T21:30:54.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986728+0000", 2024-09-17T21:30:54.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018584163000000001 2024-09-17T21:30:54.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986766+0000", 2024-09-17T21:30:54.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7987000000000001e-05 2024-09-17T21:30:54.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986804+0000", 2024-09-17T21:30:54.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7892000000000002e-05 2024-09-17T21:30:54.093 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.093 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.093 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.093 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39464 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.093 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:54.093 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024558167999999, 2024-09-17T21:30:54.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018753723999999999, 2024-09-17T21:30:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39464 2024-09-17T21:30:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:30:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968148+0000", 2024-09-17T21:30:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:30:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968152+0000", 2024-09-17T21:30:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3749999999999996e-06 2024-09-17T21:30:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968155+0000", 2024-09-17T21:30:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.371e-06 2024-09-17T21:30:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968158+0000", 2024-09-17T21:30:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T21:30:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986824+0000", 2024-09-17T21:30:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018665779 2024-09-17T21:30:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986863+0000", 2024-09-17T21:30:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9273000000000002e-05 2024-09-17T21:30:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986901+0000", 2024-09-17T21:30:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7978000000000001e-05 2024-09-17T21:30:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39465 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:30:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.024542769, 2024-09-17T21:30:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018825341999999998, 2024-09-17T21:30:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39465 2024-09-17T21:30:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:30:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-17T21:30:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968164+0000", 2024-09-17T21:30:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1289999999999999e-06 2024-09-17T21:30:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968167+0000", 2024-09-17T21:30:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1379999999999999e-06 2024-09-17T21:30:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968170+0000", 2024-09-17T21:30:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:30:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986914+0000", 2024-09-17T21:30:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018743829 2024-09-17T21:30:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986950+0000", 2024-09-17T21:30:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5819999999999999e-05 2024-09-17T21:30:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986988+0000", 2024-09-17T21:30:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8362000000000003e-05 2024-09-17T21:30:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.104 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.104 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:54.106 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:30:54.106 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:54.103+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:54.106 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-17T21:30:54.107 INFO:teuthology.orchestra.run.smithi129.stdout:5497 read 0x70efbd4 thru 0x70fe2e0 (0xe70d bytes) 2024-09-17T21:30:54.107 INFO:teuthology.orchestra.run.smithi129.stdout:5498 writesame 0x2063064 thru 0x206f71f data_size 0x5c(0xc6bc bytes) 2024-09-17T21:30:54.107 INFO:teuthology.orchestra.run.smithi129.stdout:5500 read 0xc085a83 thru 0xc08f335 (0x98b3 bytes) 2024-09-17T21:30:54.107 INFO:teuthology.orchestra.run.smithi129.stdout:5502 writesame 0xe051047 thru 0xe054982 data_size 0x63(0x393c bytes) 2024-09-17T21:30:54.107 INFO:teuthology.orchestra.run.smithi129.stdout:5505 read 0x77d0f4d thru 0x77d9f86 (0x903a bytes) 2024-09-17T21:30:54.107 INFO:teuthology.orchestra.run.smithi129.stdout:5506 trunc from 0xe054983 to 0x14d1797 2024-09-17T21:30:54.171 INFO:teuthology.orchestra.run.smithi129.stdout:5508 writesame 0xc284f9c thru 0xc28c8a1 data_size 0x7(0x7906 bytes) 2024-09-17T21:30:54.171 INFO:teuthology.orchestra.run.smithi129.stdout:5510 read 0x39e5c54 thru 0x39f4f7a (0xf327 bytes) 2024-09-17T21:30:54.179 INFO:teuthology.orchestra.run.smithi129.stdout:5512 writesame 0xdcfad8f thru 0xdd08820 data_size 0x65(0xda92 bytes) 2024-09-17T21:30:54.186 INFO:teuthology.orchestra.run.smithi129.stdout:5513 trunc from 0xdd08821 to 0x7a085eb 2024-09-17T21:30:54.191 INFO:teuthology.orchestra.run.smithi129.stdout:5515 punch from 0x265833 to 0x273bbc, (0xe389 bytes) 2024-09-17T21:30:54.192 INFO:teuthology.orchestra.run.smithi129.stdout:5516 read 0x17c8df1 thru 0x17ce81b (0x5a2b bytes) 2024-09-17T21:30:54.197 INFO:teuthology.orchestra.run.smithi129.stdout:5517 trunc from 0x7a085eb to 0x2fc81a5 2024-09-17T21:30:54.203 INFO:teuthology.orchestra.run.smithi129.stdout:5518 read 0x23effb2 thru 0x23fdce1 (0xdd30 bytes) 2024-09-17T21:30:54.206 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:54.204+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:54.214 INFO:teuthology.orchestra.run.smithi129.stdout:5519 trunc from 0x2fc81a5 to 0xc23d1b5 2024-09-17T21:30:54.220 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:54.229 INFO:teuthology.orchestra.run.smithi129.stdout:5521 write 0xeb0bc57 thru 0xeb138a8 (0x7c52 bytes) 2024-09-17T21:30:54.232 INFO:teuthology.orchestra.run.smithi129.stdout:5522 write 0xa2a904 thru 0xa36efe (0xc5fb bytes) 2024-09-17T21:30:54.234 INFO:teuthology.orchestra.run.smithi129.stdout:5523 write 0xcfbd97c thru 0xcfc4a18 (0x709d bytes) 2024-09-17T21:30:54.244 INFO:teuthology.orchestra.run.smithi129.stdout:5524 write 0x7f6b94e thru 0x7f6c134 (0x7e7 bytes) 2024-09-17T21:30:54.252 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:54.253 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:54.253 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:30:54.253 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:54.258 INFO:teuthology.orchestra.run.smithi129.stdout:5525 trunc from 0xeb138a9 to 0x52a2144 2024-09-17T21:30:54.262 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:30:54.268 INFO:teuthology.orchestra.run.smithi129.stdout:5526 read 0x240256f thru 0x24065de (0x4070 bytes) 2024-09-17T21:30:54.284 INFO:teuthology.orchestra.run.smithi129.stdout:5527 writesame 0xdbd601c thru 0xdbd9b9b data_size 0x44(0x3b80 bytes) 2024-09-17T21:30:54.284 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:54.284 INFO:teuthology.orchestra.run.smithi129.stdout:5529 read 0x1912764 thru 0x1916008 (0x38a5 bytes) 2024-09-17T21:30:54.285 INFO:teuthology.orchestra.run.smithi129.stdout:5531 punch from 0xabbb93d to 0xabbba80, (0x143 bytes) 2024-09-17T21:30:54.292 INFO:teuthology.orchestra.run.smithi129.stdout:5532 trunc from 0xdbd9b9c to 0x6ed9cda 2024-09-17T21:30:54.306 INFO:teuthology.orchestra.run.smithi129.stdout:5533 write 0x9868594 thru 0x9868c7f (0x6ec bytes) 2024-09-17T21:30:54.307 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:54.304+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:54.312 INFO:teuthology.orchestra.run.smithi129.stdout:5534 read 0x5c42192 thru 0x5c42421 (0x290 bytes) 2024-09-17T21:30:54.312 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:54.312 INFO:teuthology.orchestra.run.smithi129.stdout:5537 punch from 0x18e233c to 0x18e52e8, (0x2fac bytes) 2024-09-17T21:30:54.312 INFO:teuthology.orchestra.run.smithi129.stdout:5538 punch from 0x53b84b1 to 0x53bff29, (0x7a78 bytes) 2024-09-17T21:30:54.312 INFO:teuthology.orchestra.run.smithi129.stdout:5539 write 0x5b5e25a thru 0x5b66ed4 (0x8c7b bytes) 2024-09-17T21:30:54.319 INFO:teuthology.orchestra.run.smithi129.stdout:5540 write 0x92955ee thru 0x929f689 (0xa09c bytes) 2024-09-17T21:30:54.322 INFO:teuthology.orchestra.run.smithi129.stdout:5541 punch from 0x7ce04f7 to 0x7ce3922, (0x342b bytes) 2024-09-17T21:30:54.322 INFO:teuthology.orchestra.run.smithi129.stdout:5542 write 0x33f93e8 thru 0x34007ea (0x7403 bytes) 2024-09-17T21:30:54.332 INFO:teuthology.orchestra.run.smithi129.stdout:5543 trunc from 0x9868c80 to 0xd32e2c9 2024-09-17T21:30:54.337 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:54.348 INFO:teuthology.orchestra.run.smithi129.stdout:5545 trunc from 0xd32e2c9 to 0x474c9fe 2024-09-17T21:30:54.368 INFO:teuthology.orchestra.run.smithi129.stdout:5546 writesame 0x7926676 thru 0x7934177 data_size 0x61(0xdb02 bytes) 2024-09-17T21:30:54.369 INFO:teuthology.orchestra.run.smithi129.stdout:5548 read 0x1de3462 thru 0x1df135d (0xdefc bytes) 2024-09-17T21:30:54.369 INFO:teuthology.orchestra.run.smithi129.stdout:5549 punch from 0x1f080ae to 0x1f11839, (0x978b bytes) 2024-09-17T21:30:54.369 INFO:teuthology.orchestra.run.smithi129.stdout:5550 read 0xff7ed6 thru 0x1003a55 (0xbb80 bytes) 2024-09-17T21:30:54.370 INFO:teuthology.orchestra.run.smithi129.stdout:5551 read 0x6493c9f thru 0x64a344b (0xf7ad bytes) 2024-09-17T21:30:54.370 INFO:teuthology.orchestra.run.smithi129.stdout:5552 write 0x43cdada thru 0x43d6cb8 (0x91df bytes) 2024-09-17T21:30:54.371 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:54.371 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:30:54.371 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:30:54.372 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:30:54.372 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:54.372 INFO:teuthology.orchestra.run.smithi129.stdout:5553 read 0x1304a87 thru 0x131222a (0xd7a4 bytes) 2024-09-17T21:30:54.373 INFO:teuthology.orchestra.run.smithi129.stdout:5554 punch from 0x707bfe5 to 0x708a248, (0xe263 bytes) 2024-09-17T21:30:54.373 INFO:teuthology.orchestra.run.smithi129.stdout:5555 writesame 0x354a1ab thru 0x3556e54 data_size 0x2e(0xccaa bytes) 2024-09-17T21:30:54.375 INFO:teuthology.orchestra.run.smithi129.stdout:5556 write 0x6d57afa thru 0x6d5e604 (0x6b0b bytes) 2024-09-17T21:30:54.378 INFO:teuthology.orchestra.run.smithi129.stdout:5560 writesame 0xc4eb60 thru 0xc55c81 data_size 0x9(0x7122 bytes) 2024-09-17T21:30:54.381 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:30:54.382 INFO:teuthology.orchestra.run.smithi129.stdout:5561 punch from 0x6e169bc to 0x6e24811, (0xde55 bytes) 2024-09-17T21:30:54.383 INFO:teuthology.orchestra.run.smithi129.stdout:5562 trunc from 0x7934178 to 0x94f189f 2024-09-17T21:30:54.390 INFO:teuthology.orchestra.run.smithi129.stdout:5563 punch from 0x871b8d to 0x87354a, (0x19bd bytes) 2024-09-17T21:30:54.390 INFO:teuthology.orchestra.run.smithi129.stdout:5564 read 0x3f02c89 thru 0x3f126e3 (0xfa5b bytes) 2024-09-17T21:30:54.391 INFO:teuthology.orchestra.run.smithi129.stdout:5565 punch from 0x5e7abf1 to 0x5e89b24, (0xef33 bytes) 2024-09-17T21:30:54.391 INFO:teuthology.orchestra.run.smithi129.stdout:5567 read 0x20e54ed thru 0x20ea3fa (0x4f0e bytes) 2024-09-17T21:30:54.401 INFO:teuthology.orchestra.run.smithi129.stdout:5568 trunc from 0x94f189f to 0x2fac77 2024-09-17T21:30:54.407 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:54.405+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:54.436 INFO:teuthology.orchestra.run.smithi129.stdout:5569 writesame 0xa66e4d5 thru 0xa67ab47 data_size 0x1(0xc673 bytes) 2024-09-17T21:30:54.436 INFO:teuthology.orchestra.run.smithi129.stdout:5570 punch from 0x71473e3 to 0x714e605, (0x7222 bytes) 2024-09-17T21:30:54.436 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:54.436 INFO:teuthology.orchestra.run.smithi129.stdout:5572 writesame 0x609c17a thru 0x609d797 data_size 0x2(0x161e bytes) 2024-09-17T21:30:54.439 INFO:teuthology.orchestra.run.smithi129.stdout:5573 read 0x56b6430 thru 0x56c2e41 (0xca12 bytes) 2024-09-17T21:30:54.439 INFO:teuthology.orchestra.run.smithi129.stdout:5574 writesame 0x3ad53c5 thru 0x3adc2c3 data_size 0x1(0x6eff bytes) 2024-09-17T21:30:54.454 INFO:teuthology.orchestra.run.smithi129.stdout:5575 punch from 0x1428509 to 0x14355d3, (0xd0ca bytes) 2024-09-17T21:30:54.454 INFO:teuthology.orchestra.run.smithi129.stdout:5576 read 0x981f823 thru 0x9825f7f (0x675d bytes) 2024-09-17T21:30:54.454 INFO:teuthology.orchestra.run.smithi129.stdout:5577 read 0x3f3c067 thru 0x3f4056f (0x4509 bytes) 2024-09-17T21:30:54.454 INFO:teuthology.orchestra.run.smithi129.stdout:5578 writesame 0x60bbcc3 thru 0x60c9fa2 data_size 0x3c(0xe2e0 bytes) 2024-09-17T21:30:54.470 INFO:teuthology.orchestra.run.smithi129.stdout:5580 trunc from 0xa67ab48 to 0xd1317f 2024-09-17T21:30:54.491 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:54.491 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:54.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:54.491 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:54.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.492 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:30:54.492 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:54.492 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.548822356000002, 2024-09-17T21:30:54.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61572000699999996, 2024-09-17T21:30:54.492 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.492 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.492 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:54.492 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.492 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:30:54.492 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:30:54.492 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T21:30:54.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.492 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:54.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:54.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939745+0000", 2024-09-17T21:30:54.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6899999999999999e-06 2024-09-17T21:30:54.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939753+0000", 2024-09-17T21:30:54.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1950000000000003e-06 2024-09-17T21:30:54.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939767+0000", 2024-09-17T21:30:54.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3429e-05 2024-09-17T21:30:54.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939776+0000", 2024-09-17T21:30:54.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:30:54.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939858+0000", 2024-09-17T21:30:54.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2496000000000004e-05 2024-09-17T21:30:54.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T21:30:54.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939868+0000", 2024-09-17T21:30:54.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1910000000000006e-06 2024-09-17T21:30:54.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940612+0000", 2024-09-17T21:30:54.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074429599999999998 2024-09-17T21:30:54.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:30:54.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940621+0000", 2024-09-17T21:30:54.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9139999999999997e-06 2024-09-17T21:30:54.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554006+0000", 2024-09-17T21:30:54.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61338505099999996 2024-09-17T21:30:54.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554166+0000", 2024-09-17T21:30:54.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015982600000000001 2024-09-17T21:30:54.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:30:54.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554296+0000", 2024-09-17T21:30:54.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013002399999999999 2024-09-17T21:30:54.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:54.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554936+0000", 2024-09-17T21:30:54.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064009400000000001 2024-09-17T21:30:54.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:54.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555389+0000", 2024-09-17T21:30:54.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045355200000000002 2024-09-17T21:30:54.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:54.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555432+0000", 2024-09-17T21:30:54.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2477e-05 2024-09-17T21:30:54.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555463+0000", 2024-09-17T21:30:54.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1612000000000002e-05 2024-09-17T21:30:54.499 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.499 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.499 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.499 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17527 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1005568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:54.499 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:54.499 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.083594524999995, 2024-09-17T21:30:54.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046452090000000001, 2024-09-17T21:30:54.500 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.500 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.500 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.500 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.500 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.500 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.500 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17527 2024-09-17T21:30:54.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.500 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:54.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:54.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405496+0000", 2024-09-17T21:30:54.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052498400000000004 2024-09-17T21:30:54.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405500+0000", 2024-09-17T21:30:54.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8410000000000002e-06 2024-09-17T21:30:54.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405511+0000", 2024-09-17T21:30:54.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0749e-05 2024-09-17T21:30:54.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405517+0000", 2024-09-17T21:30:54.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:30:54.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451275+0000", 2024-09-17T21:30:54.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045758134999999998 2024-09-17T21:30:54.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451355+0000", 2024-09-17T21:30:54.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9453e-05 2024-09-17T21:30:54.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451423+0000", 2024-09-17T21:30:54.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8782999999999995e-05 2024-09-17T21:30:54.504 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.504 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.504 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.504 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17615 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:54.504 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:54.504 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.078354036999997, 2024-09-17T21:30:54.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046741309000000002, 2024-09-17T21:30:54.504 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.504 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.505 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.505 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.505 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.505 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.505 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17615 2024-09-17T21:30:54.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.505 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:54.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:54.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:54.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:30:54.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410213+0000", 2024-09-17T21:30:54.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:30:54.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410217+0000", 2024-09-17T21:30:54.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8199999999999998e-06 2024-09-17T21:30:54.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410220+0000", 2024-09-17T21:30:54.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6900000000000001e-06 2024-09-17T21:30:54.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455264+0000", 2024-09-17T21:30:54.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045044002999999999 2024-09-17T21:30:54.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455338+0000", 2024-09-17T21:30:54.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4182000000000002e-05 2024-09-17T21:30:54.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.456953+0000", 2024-09-17T21:30:54.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016150089999999999 2024-09-17T21:30:54.508 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.509 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.509 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.509 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17616 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 872448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:54.509 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:54.509 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.078341984999994, 2024-09-17T21:30:54.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946039000000002, 2024-09-17T21:30:54.509 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.509 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.509 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.509 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.509 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.510 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.510 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17616 2024-09-17T21:30:54.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.510 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:54.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:54.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:54.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T21:30:54.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410225+0000", 2024-09-17T21:30:54.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3459999999999999e-06 2024-09-17T21:30:54.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410228+0000", 2024-09-17T21:30:54.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.083e-06 2024-09-17T21:30:54.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410232+0000", 2024-09-17T21:30:54.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4419999999999998e-06 2024-09-17T21:30:54.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457004+0000", 2024-09-17T21:30:54.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046771873999999998 2024-09-17T21:30:54.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457084+0000", 2024-09-17T21:30:54.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0506000000000005e-05 2024-09-17T21:30:54.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457170+0000", 2024-09-17T21:30:54.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5494000000000005e-05 2024-09-17T21:30:54.513 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.513 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.513 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.514 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17617 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:54.514 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:54.514 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.078330582999996, 2024-09-17T21:30:54.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047084168000000003, 2024-09-17T21:30:54.514 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.514 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.514 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.514 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.514 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.514 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.514 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17617 2024-09-17T21:30:54.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.514 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:54.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:54.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:54.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-17T21:30:54.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410237+0000", 2024-09-17T21:30:54.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1960000000000001e-06 2024-09-17T21:30:54.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410246+0000", 2024-09-17T21:30:54.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7880000000000001e-06 2024-09-17T21:30:54.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410250+0000", 2024-09-17T21:30:54.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.518e-06 2024-09-17T21:30:54.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457209+0000", 2024-09-17T21:30:54.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046959283999999997 2024-09-17T21:30:54.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457264+0000", 2024-09-17T21:30:54.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673999999999999e-05 2024-09-17T21:30:54.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457319+0000", 2024-09-17T21:30:54.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5497000000000001e-05 2024-09-17T21:30:54.518 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.518 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.518 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.518 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17618 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 899072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:54.518 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:54.518 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.078311138999993, 2024-09-17T21:30:54.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047204208999999997, 2024-09-17T21:30:54.518 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.519 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.519 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.519 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.519 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.519 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.519 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17618 2024-09-17T21:30:54.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.519 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:54.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:54.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:54.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8799999999999998e-07 2024-09-17T21:30:54.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410256+0000", 2024-09-17T21:30:54.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T21:30:54.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410260+0000", 2024-09-17T21:30:54.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6500000000000002e-06 2024-09-17T21:30:54.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410263+0000", 2024-09-17T21:30:54.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7089999999999998e-06 2024-09-17T21:30:54.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457355+0000", 2024-09-17T21:30:54.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047092756999999999 2024-09-17T21:30:54.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457407+0000", 2024-09-17T21:30:54.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1808999999999998e-05 2024-09-17T21:30:54.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457459+0000", 2024-09-17T21:30:54.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1684000000000002e-05 2024-09-17T21:30:54.522 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.522 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.522 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.523 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17619 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 913408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:54.523 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:54.523 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.078299900999994, 2024-09-17T21:30:54.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047293443999999997, 2024-09-17T21:30:54.523 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.523 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.523 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.523 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.523 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.523 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.523 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17619 2024-09-17T21:30:54.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.523 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:54.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:54.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:54.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:30:54.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410267+0000", 2024-09-17T21:30:54.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:30:54.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410270+0000", 2024-09-17T21:30:54.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.01e-06 2024-09-17T21:30:54.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410273+0000", 2024-09-17T21:30:54.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6410000000000002e-06 2024-09-17T21:30:54.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457473+0000", 2024-09-17T21:30:54.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047199829999999998 2024-09-17T21:30:54.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457517+0000", 2024-09-17T21:30:54.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4058999999999999e-05 2024-09-17T21:30:54.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457559+0000", 2024-09-17T21:30:54.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2404999999999998e-05 2024-09-17T21:30:54.527 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.527 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.527 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.527 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17620 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 927744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:54.527 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:54.527 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.078288728999993, 2024-09-17T21:30:54.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047402409, 2024-09-17T21:30:54.527 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.527 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.528 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.528 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.528 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.528 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.528 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17620 2024-09-17T21:30:54.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.528 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:54.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:54.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:54.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:30:54.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410278+0000", 2024-09-17T21:30:54.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1790000000000001e-06 2024-09-17T21:30:54.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410281+0000", 2024-09-17T21:30:54.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.061e-06 2024-09-17T21:30:54.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410284+0000", 2024-09-17T21:30:54.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.65e-06 2024-09-17T21:30:54.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457587+0000", 2024-09-17T21:30:54.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047303206 2024-09-17T21:30:54.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457633+0000", 2024-09-17T21:30:54.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6063999999999998e-05 2024-09-17T21:30:54.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457679+0000", 2024-09-17T21:30:54.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6007000000000002e-05 2024-09-17T21:30:54.531 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.531 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.531 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.532 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17621 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 940032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:54.532 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:54.532 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.078278312999998, 2024-09-17T21:30:54.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047486550000000002, 2024-09-17T21:30:54.532 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.532 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.532 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.532 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.532 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.532 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.532 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17621 2024-09-17T21:30:54.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.532 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:54.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:54.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410288+0000", 2024-09-17T21:30:54.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:30:54.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410289+0000", 2024-09-17T21:30:54.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1179999999999999e-06 2024-09-17T21:30:54.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410292+0000", 2024-09-17T21:30:54.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:30:54.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410294+0000", 2024-09-17T21:30:54.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T21:30:54.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457694+0000", 2024-09-17T21:30:54.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047399416999999999 2024-09-17T21:30:54.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457734+0000", 2024-09-17T21:30:54.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0528999999999998e-05 2024-09-17T21:30:54.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457774+0000", 2024-09-17T21:30:54.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9529e-05 2024-09-17T21:30:54.536 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.536 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.536 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.536 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17622 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:54.536 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:54.536 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.078268131000002, 2024-09-17T21:30:54.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047596619, 2024-09-17T21:30:54.536 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.537 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.537 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.537 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.537 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.537 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.537 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17622 2024-09-17T21:30:54.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.537 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:54.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:54.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:54.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:54.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410303+0000", 2024-09-17T21:30:54.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2979999999999999e-06 2024-09-17T21:30:54.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410307+0000", 2024-09-17T21:30:54.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:30:54.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410310+0000", 2024-09-17T21:30:54.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.146e-06 2024-09-17T21:30:54.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457801+0000", 2024-09-17T21:30:54.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047491322000000002 2024-09-17T21:30:54.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457850+0000", 2024-09-17T21:30:54.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8585000000000003e-05 2024-09-17T21:30:54.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457894+0000", 2024-09-17T21:30:54.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4455e-05 2024-09-17T21:30:54.540 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.540 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.540 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.541 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17623 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 968704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:54.541 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:54.541 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.078251746999996, 2024-09-17T21:30:54.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047689377999999998, 2024-09-17T21:30:54.541 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.541 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.541 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.541 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17623 2024-09-17T21:30:54.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.542 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:54.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:54.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:54.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:30:54.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410315+0000", 2024-09-17T21:30:54.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0499999999999999e-06 2024-09-17T21:30:54.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410319+0000", 2024-09-17T21:30:54.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.517e-06 2024-09-17T21:30:54.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410321+0000", 2024-09-17T21:30:54.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:30:54.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457922+0000", 2024-09-17T21:30:54.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047600638000000001 2024-09-17T21:30:54.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457964+0000", 2024-09-17T21:30:54.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2126000000000003e-05 2024-09-17T21:30:54.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458003+0000", 2024-09-17T21:30:54.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9138999999999999e-05 2024-09-17T21:30:54.545 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.545 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.545 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.545 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17624 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:54.546 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:54.546 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.078241163000001, 2024-09-17T21:30:54.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047798292999999999, 2024-09-17T21:30:54.546 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.546 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.546 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.546 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.546 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.546 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.546 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17624 2024-09-17T21:30:54.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.546 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:54.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:54.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:54.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:30:54.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410326+0000", 2024-09-17T21:30:54.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0610000000000001e-06 2024-09-17T21:30:54.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410329+0000", 2024-09-17T21:30:54.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0319999999999999e-06 2024-09-17T21:30:54.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410331+0000", 2024-09-17T21:30:54.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5749999999999999e-06 2024-09-17T21:30:54.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458033+0000", 2024-09-17T21:30:54.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047701371999999999 2024-09-17T21:30:54.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458079+0000", 2024-09-17T21:30:54.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:30:54.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458123+0000", 2024-09-17T21:30:54.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850999999999997e-05 2024-09-17T21:30:54.550 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.550 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.550 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.550 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17625 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:54.550 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:54.550 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.078231002999999, 2024-09-17T21:30:54.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047883051000000003, 2024-09-17T21:30:54.550 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.550 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.550 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.551 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.551 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.551 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.551 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17625 2024-09-17T21:30:54.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.551 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:54.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:54.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:54.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:54.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410336+0000", 2024-09-17T21:30:54.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1310000000000001e-06 2024-09-17T21:30:54.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410339+0000", 2024-09-17T21:30:54.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000001e-06 2024-09-17T21:30:54.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410342+0000", 2024-09-17T21:30:54.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7460000000000001e-06 2024-09-17T21:30:54.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458137+0000", 2024-09-17T21:30:54.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047794931999999998 2024-09-17T21:30:54.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458178+0000", 2024-09-17T21:30:54.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1013999999999997e-05 2024-09-17T21:30:54.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458218+0000", 2024-09-17T21:30:54.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9675999999999998e-05 2024-09-17T21:30:54.554 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.554 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.554 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.555 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17626 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1009664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:54.555 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:54.555 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.078220381999998, 2024-09-17T21:30:54.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047991601000000002, 2024-09-17T21:30:54.555 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.555 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.555 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.555 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.555 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.555 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.555 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17626 2024-09-17T21:30:54.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.556 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:54.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:54.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410346+0000", 2024-09-17T21:30:54.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:30:54.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410347+0000", 2024-09-17T21:30:54.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.114e-06 2024-09-17T21:30:54.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410350+0000", 2024-09-17T21:30:54.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.101e-06 2024-09-17T21:30:54.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410353+0000", 2024-09-17T21:30:54.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T21:30:54.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458245+0000", 2024-09-17T21:30:54.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0478921 2024-09-17T21:30:54.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458292+0000", 2024-09-17T21:30:54.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397999999999999e-05 2024-09-17T21:30:54.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458337+0000", 2024-09-17T21:30:54.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5221e-05 2024-09-17T21:30:54.559 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.559 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.559 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.559 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17627 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:54.559 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:54.559 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.078183280999994, 2024-09-17T21:30:54.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048047383999999999, 2024-09-17T21:30:54.560 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.560 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.560 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.560 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.560 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.560 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.560 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17627 2024-09-17T21:30:54.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.560 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:54.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:54.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410383+0000", 2024-09-17T21:30:54.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:30:54.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410385+0000", 2024-09-17T21:30:54.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:30:54.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410391+0000", 2024-09-17T21:30:54.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:30:54.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410395+0000", 2024-09-17T21:30:54.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6539999999999998e-06 2024-09-17T21:30:54.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458350+0000", 2024-09-17T21:30:54.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047954855999999997 2024-09-17T21:30:54.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458392+0000", 2024-09-17T21:30:54.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2094000000000001e-05 2024-09-17T21:30:54.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458430+0000", 2024-09-17T21:30:54.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7474000000000001e-05 2024-09-17T21:30:54.563 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.564 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.564 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.564 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17628 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:54.564 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:54.564 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.078163720000006, 2024-09-17T21:30:54.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048142676000000002, 2024-09-17T21:30:54.564 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.564 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.564 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.564 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.564 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.565 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.565 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17628 2024-09-17T21:30:54.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.565 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:54.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:54.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:54.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:30:54.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410404+0000", 2024-09-17T21:30:54.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-06 2024-09-17T21:30:54.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410411+0000", 2024-09-17T21:30:54.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6850000000000004e-06 2024-09-17T21:30:54.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410415+0000", 2024-09-17T21:30:54.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6750000000000001e-06 2024-09-17T21:30:54.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458457+0000", 2024-09-17T21:30:54.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048042045999999998 2024-09-17T21:30:54.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458501+0000", 2024-09-17T21:30:54.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3661000000000001e-05 2024-09-17T21:30:54.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458545+0000", 2024-09-17T21:30:54.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3529000000000003e-05 2024-09-17T21:30:54.568 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.569 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.569 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.569 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17687 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:54.569 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:54.569 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.075718766999998, 2024-09-17T21:30:54.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050670448, 2024-09-17T21:30:54.569 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.569 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.569 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:54.569 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.569 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.570 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.570 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17687 2024-09-17T21:30:54.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.570 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:54.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:54.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412848+0000", 2024-09-17T21:30:54.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9100000000000003e-07 2024-09-17T21:30:54.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413102+0000", 2024-09-17T21:30:54.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025415600000000001 2024-09-17T21:30:54.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413111+0000", 2024-09-17T21:30:54.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2010000000000005e-06 2024-09-17T21:30:54.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413116+0000", 2024-09-17T21:30:54.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:30:54.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458559+0000", 2024-09-17T21:30:54.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045442914000000001 2024-09-17T21:30:54.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458632+0000", 2024-09-17T21:30:54.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3207000000000006e-05 2024-09-17T21:30:54.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:54.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458785+0000", 2024-09-17T21:30:54.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015226700000000001 2024-09-17T21:30:54.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:54.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.461648+0000", 2024-09-17T21:30:54.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028633069999999998 2024-09-17T21:30:54.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:54.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463416+0000", 2024-09-17T21:30:54.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017680230000000001 2024-09-17T21:30:54.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:54.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463475+0000", 2024-09-17T21:30:54.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9333000000000003e-05 2024-09-17T21:30:54.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463517+0000", 2024-09-17T21:30:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2162000000000001e-05 2024-09-17T21:30:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17688 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [write 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.075439899000003, 2024-09-17T21:30:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050625098, 2024-09-17T21:30:54.575 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.576 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.576 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:54.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.576 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17688 2024-09-17T21:30:54.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.576 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:54.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:54.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:54.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:30:54.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413128+0000", 2024-09-17T21:30:54.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.813e-06 2024-09-17T21:30:54.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413132+0000", 2024-09-17T21:30:54.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:30:54.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413135+0000", 2024-09-17T21:30:54.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:30:54.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459756+0000", 2024-09-17T21:30:54.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046621293000000001 2024-09-17T21:30:54.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459816+0000", 2024-09-17T21:30:54.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9156000000000001e-05 2024-09-17T21:30:54.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:54.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459912+0000", 2024-09-17T21:30:54.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6302000000000001e-05 2024-09-17T21:30:54.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:54.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463231+0000", 2024-09-17T21:30:54.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033188290000000001 2024-09-17T21:30:54.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:54.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463701+0000", 2024-09-17T21:30:54.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047068199999999999 2024-09-17T21:30:54.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:54.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463728+0000", 2024-09-17T21:30:54.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6726000000000001e-05 2024-09-17T21:30:54.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463751+0000", 2024-09-17T21:30:54.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2824e-05 2024-09-17T21:30:54.581 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.581 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.581 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.581 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17691 3.d 3:b3a055f3:::rbd_data.11207744cc2c.0000000000000083:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:54.581 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:54.581 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.071894995999997, 2024-09-17T21:30:54.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049309370999999998, 2024-09-17T21:30:54.581 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.582 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.582 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:54.582 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.582 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.582 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.582 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17691 2024-09-17T21:30:54.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.582 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:54.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:54.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416672+0000", 2024-09-17T21:30:54.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.47e-07 2024-09-17T21:30:54.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416915+0000", 2024-09-17T21:30:54.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024344500000000001 2024-09-17T21:30:54.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416924+0000", 2024-09-17T21:30:54.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7739999999999993e-06 2024-09-17T21:30:54.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416929+0000", 2024-09-17T21:30:54.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6990000000000002e-06 2024-09-17T21:30:54.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460414+0000", 2024-09-17T21:30:54.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043485466 2024-09-17T21:30:54.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460540+0000", 2024-09-17T21:30:54.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012608899999999999 2024-09-17T21:30:54.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:54.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460746+0000", 2024-09-17T21:30:54.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000205577 2024-09-17T21:30:54.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:54.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463800+0000", 2024-09-17T21:30:54.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030542030000000001 2024-09-17T21:30:54.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:54.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465867+0000", 2024-09-17T21:30:54.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020665929999999998 2024-09-17T21:30:54.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:54.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465914+0000", 2024-09-17T21:30:54.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497000000000003e-05 2024-09-17T21:30:54.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465980+0000", 2024-09-17T21:30:54.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6081000000000006e-05 2024-09-17T21:30:54.587 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.587 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.587 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.587 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:70218 3.16 3:68692d8f:::rbd_data.1120b9193b9f.000000000000001b:head [stat,delete] snapc 1d=[1d] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-17T21:30:54.587 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:54.588 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.682927907, 2024-09-17T21:30:54.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.951400786, 2024-09-17T21:30:54.588 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:54.588 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:54.588 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:54.588 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:54.588 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:54.588 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.588 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 70218 2024-09-17T21:30:54.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.588 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:54.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:54.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:54.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:54.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:54.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:54.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:54.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:54.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4799999999999998e-07 2024-09-17T21:30:54.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:54.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805642+0000", 2024-09-17T21:30:54.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2009999999999999e-06 2024-09-17T21:30:54.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:54.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805650+0000", 2024-09-17T21:30:54.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1000000000000004e-06 2024-09-17T21:30:54.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805655+0000", 2024-09-17T21:30:54.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:30:54.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805722+0000", 2024-09-17T21:30:54.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6878000000000006e-05 2024-09-17T21:30:54.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:30:54.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805733+0000", 2024-09-17T21:30:54.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0895e-05 2024-09-17T21:30:54.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:54.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755252+0000", 2024-09-17T21:30:54.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.94951843800000002 2024-09-17T21:30:54.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:54.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755398+0000", 2024-09-17T21:30:54.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014622600000000001 2024-09-17T21:30:54.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:54.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:54.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:54.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.757039+0000", 2024-09-17T21:30:54.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001640819 2024-09-17T21:30:54.593 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.593 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.593 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.593 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:54.593 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:54.593 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:54.595 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:30:54.595 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:54.592+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:54.596 INFO:teuthology.orchestra.run.smithi129.stdout:5581 write 0x64a894d thru 0x64ad966 (0x501a bytes) 2024-09-17T21:30:54.596 INFO:teuthology.orchestra.run.smithi129.stdout:5582 writesame 0xbceb9f3 thru 0xbcf97e6 data_size 0x1e(0xddf4 bytes) 2024-09-17T21:30:54.597 INFO:teuthology.orchestra.run.smithi129.stdout:5583 punch from 0xb43d997 to 0xb43ea26, (0x108f bytes) 2024-09-17T21:30:54.597 INFO:teuthology.orchestra.run.smithi129.stdout:5584 write 0xb4766e2 thru 0xb47769d (0xfbc bytes) 2024-09-17T21:30:54.597 INFO:teuthology.orchestra.run.smithi129.stdout:5585 punch from 0xa2e9a62 to 0xa2f4a59, (0xaff7 bytes) 2024-09-17T21:30:54.597 INFO:teuthology.orchestra.run.smithi129.stdout:5587 read 0x83422c thru 0x839580 (0x5355 bytes) 2024-09-17T21:30:54.597 INFO:teuthology.orchestra.run.smithi129.stdout:5589 trunc from 0xbcf97e7 to 0x78ccb6 2024-09-17T21:30:54.597 INFO:teuthology.orchestra.run.smithi129.stdout:5591 write 0x3cfd504 thru 0x3d0922b (0xbd28 bytes) 2024-09-17T21:30:54.597 INFO:teuthology.orchestra.run.smithi129.stdout:5593 write 0xc0e4e58 thru 0xc0f3fe9 (0xf192 bytes) 2024-09-17T21:30:54.597 INFO:teuthology.orchestra.run.smithi129.stdout:5594 read 0x4145e68 thru 0x4152503 (0xc69c bytes) 2024-09-17T21:30:54.597 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:54.597 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:54.597 INFO:teuthology.orchestra.run.smithi129.stdout:5599 write 0x6ab46a4 thru 0x6abbb6a (0x74c7 bytes) 2024-09-17T21:30:54.597 INFO:teuthology.orchestra.run.smithi129.stdout:5601 write 0x3e9d11d thru 0x3ea2927 (0x580b bytes) 2024-09-17T21:30:54.597 INFO:teuthology.orchestra.run.smithi129.stdout:5606 trunc from 0xc0f3fea to 0x6624ada 2024-09-17T21:30:54.611 INFO:teuthology.orchestra.run.smithi129.stdout:5607 write 0xd6b038f thru 0xd6b623e (0x5eb0 bytes) 2024-09-17T21:30:54.615 INFO:teuthology.orchestra.run.smithi129.stdout:5608 read 0x9a1851 thru 0x9a90db (0x788b bytes) 2024-09-17T21:30:54.615 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:54.615 INFO:teuthology.orchestra.run.smithi129.stdout:5610 write 0x24c3198 thru 0x24c5314 (0x217d bytes) 2024-09-17T21:30:54.617 INFO:teuthology.orchestra.run.smithi129.stdout:5613 punch from 0x9a561ba to 0x9a58554, (0x239a bytes) 2024-09-17T21:30:54.617 INFO:teuthology.orchestra.run.smithi129.stdout:5614 read 0x8ccfae5 thru 0x8cd53e2 (0x58fe bytes) 2024-09-17T21:30:54.617 INFO:teuthology.orchestra.run.smithi129.stdout:5616 read 0x5420a23 thru 0x542bdd1 (0xb3af bytes) 2024-09-17T21:30:54.617 INFO:teuthology.orchestra.run.smithi129.stdout:5617 punch from 0x18b684f to 0x18b9896, (0x3047 bytes) 2024-09-17T21:30:54.617 INFO:teuthology.orchestra.run.smithi129.stdout:5618 writesame 0x3b15dc6 thru 0x3b22161 data_size 0x9c(0xc39c bytes) 2024-09-17T21:30:54.632 INFO:teuthology.orchestra.run.smithi129.stdout:5619 trunc from 0xd6b623f to 0x4b64544 2024-09-17T21:30:54.644 INFO:teuthology.orchestra.run.smithi129.stdout:5620 write 0x4954f7e thru 0x4963e4f (0xeed2 bytes) 2024-09-17T21:30:54.663 INFO:teuthology.orchestra.run.smithi129.stdout:5621 write 0xdeabe0e thru 0xdeb056e (0x4761 bytes) 2024-09-17T21:30:54.666 INFO:teuthology.orchestra.run.smithi129.stdout:5622 punch from 0xd935ced to 0xd9459d3, (0xfce6 bytes) 2024-09-17T21:30:54.666 INFO:teuthology.orchestra.run.smithi129.stdout:5623 read 0xcfa03d3 thru 0xcfa1d77 (0x19a5 bytes) 2024-09-17T21:30:54.667 INFO:teuthology.orchestra.run.smithi129.stdout:5624 read 0x77c3c5d thru 0x77d2a9a (0xee3e bytes) 2024-09-17T21:30:54.667 INFO:teuthology.orchestra.run.smithi129.stdout:5626 read 0x50867ab thru 0x508b422 (0x4c78 bytes) 2024-09-17T21:30:54.667 INFO:teuthology.orchestra.run.smithi129.stdout:5627 write 0x47ab4b thru 0x48066c (0x5b22 bytes) 2024-09-17T21:30:54.668 INFO:teuthology.orchestra.run.smithi129.stdout:5628 write 0x868d1dc thru 0x8697e1d (0xac42 bytes) 2024-09-17T21:30:54.671 INFO:teuthology.orchestra.run.smithi129.stdout:5629 trunc from 0xdeb056f to 0xdd7fd94 2024-09-17T21:30:54.679 INFO:teuthology.orchestra.run.smithi129.stdout:5630 punch from 0x42b9747 to 0x42ba1c3, (0xa7c bytes) 2024-09-17T21:30:54.679 INFO:teuthology.orchestra.run.smithi129.stdout:5633 read 0x91ec759 thru 0x91fb058 (0xe900 bytes) 2024-09-17T21:30:54.679 INFO:teuthology.orchestra.run.smithi129.stdout:5634 writesame 0x1023644 thru 0x1028664 data_size 0x1(0x5021 bytes) 2024-09-17T21:30:54.693 INFO:teuthology.orchestra.run.smithi129.stdout:5635 trunc from 0xdd7fd94 to 0xcbf3160 2024-09-17T21:30:54.694 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:54.692+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:54.705 INFO:teuthology.orchestra.run.smithi129.stdout:5636 write 0x9a0cc7f thru 0x9a11d16 (0x5098 bytes) 2024-09-17T21:30:54.709 INFO:teuthology.orchestra.run.smithi129.stdout:5637 read 0x50fc46e thru 0x5108f5d (0xcaf0 bytes) 2024-09-17T21:30:54.709 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:54.709 INFO:teuthology.orchestra.run.smithi129.stdout:5639 write 0xa1eda09 thru 0xa1f8ed2 (0xb4ca bytes) 2024-09-17T21:30:54.713 INFO:teuthology.orchestra.run.smithi129.stdout:5640 punch from 0x6ebee07 to 0x6ec44f5, (0x56ee bytes) 2024-09-17T21:30:54.713 INFO:teuthology.orchestra.run.smithi129.stdout:5641 punch from 0x32b8948 to 0x32bc7cd, (0x3e85 bytes) 2024-09-17T21:30:54.713 INFO:teuthology.orchestra.run.smithi129.stdout:5642 read 0x83472c7 thru 0x8348b02 (0x183c bytes) 2024-09-17T21:30:54.713 INFO:teuthology.orchestra.run.smithi129.stdout:5644 write 0x171b3e6 thru 0x1722538 (0x7153 bytes) 2024-09-17T21:30:54.715 INFO:teuthology.orchestra.run.smithi129.stdout:5645 read 0x8d3cd6b thru 0x8d3fce4 (0x2f7a bytes) 2024-09-17T21:30:54.715 INFO:teuthology.orchestra.run.smithi129.stdout:5647 read 0x20fa189 thru 0x2103efe (0x9d76 bytes) 2024-09-17T21:30:54.716 INFO:teuthology.orchestra.run.smithi129.stdout:5648 write 0x6087127 thru 0x6090639 (0x9513 bytes) 2024-09-17T21:30:54.719 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:54.719 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:54.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:54.719 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:108191 3.0 3:0640231f:::rbd_data.1120928f1bcf.0000000000000018:head [set-alloc-hint object_size 4194304 write_size 4194304,write 815399~38163 in=38163b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-17T21:30:54.719 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:54.717231+0000", 2024-09-17T21:30:54.719 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.00024284799999999999, 2024-09-17T21:30:54.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026801700000000001, 2024-09-17T21:30:54.719 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:54.719 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:54.719 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:30:54.720 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:54.720 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:54.720 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:54.720 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 108191 2024-09-17T21:30:54.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:54.720 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:54.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:54.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:54.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:54.717231+0000", 2024-09-17T21:30:54.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:54.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:54.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:54.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:54.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:54.717231+0000", 2024-09-17T21:30:54.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:54.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:54.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:54.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:54.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:54.717233+0000", 2024-09-17T21:30:54.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.643e-06 2024-09-17T21:30:54.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:54.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:54.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:54.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:54.717252+0000", 2024-09-17T21:30:54.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8923000000000002e-05 2024-09-17T21:30:54.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:54.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:54.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:54.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:54.717265+0000", 2024-09-17T21:30:54.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3332e-05 2024-09-17T21:30:54.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:54.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:54.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:54.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:54.717271+0000", 2024-09-17T21:30:54.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9599999999999997e-06 2024-09-17T21:30:54.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:54.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:54.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:54.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:54.717319+0000", 2024-09-17T21:30:54.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8214000000000003e-05 2024-09-17T21:30:54.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:54.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:54.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:54.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:54.717372+0000", 2024-09-17T21:30:54.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2703e-05 2024-09-17T21:30:54.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:54.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:54.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:54.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:54.717454+0000", 2024-09-17T21:30:54.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2633000000000007e-05 2024-09-17T21:30:54.723 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:54.723 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:54.723 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:54.723 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:54.723 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:30:54.723 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 1 2024-09-17T21:30:54.724 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:54.724 INFO:teuthology.orchestra.run.smithi129.stdout:5650 read 0x27145cb thru 0x271cade (0x8514 bytes) 2024-09-17T21:30:54.724 INFO:teuthology.orchestra.run.smithi129.stdout:5651 read 0x32b27cc thru 0x32bb70e (0x8f43 bytes) 2024-09-17T21:30:54.733 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:30:54.735 INFO:teuthology.orchestra.run.smithi129.stdout:5652 trunc from 0xcbf3160 to 0x6eb0fa 2024-09-17T21:30:54.770 INFO:teuthology.orchestra.run.smithi129.stdout:5653 writesame 0xb0c45fe thru 0xb0ca762 data_size 0x1(0x6165 bytes) 2024-09-17T21:30:54.771 INFO:teuthology.orchestra.run.smithi129.stdout:5655 trunc from 0xb0ca763 to 0xc1040e0 2024-09-17T21:30:54.777 INFO:teuthology.orchestra.run.smithi129.stdout:5657 write 0x6361594 thru 0x6369d1f (0x878c bytes) 2024-09-17T21:30:54.783 INFO:teuthology.orchestra.run.smithi129.stdout:5660 trunc from 0xc1040e0 to 0x9a6194c 2024-09-17T21:30:54.790 INFO:teuthology.orchestra.run.smithi129.stdout:5661 read 0x568f1ca thru 0x569835a (0x9191 bytes) 2024-09-17T21:30:54.796 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:54.794+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:54.801 INFO:teuthology.orchestra.run.smithi129.stdout:5664 write 0xdfeabe6 thru 0xdfed4e7 (0x2902 bytes) 2024-09-17T21:30:54.805 INFO:teuthology.orchestra.run.smithi129.stdout:5665 read 0x8de003e thru 0x8de58ac (0x586f bytes) 2024-09-17T21:30:54.805 INFO:teuthology.orchestra.run.smithi129.stdout:5666 read 0x8d50281 thru 0x8d5a6aa (0xa42a bytes) 2024-09-17T21:30:54.805 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:54.805 INFO:teuthology.orchestra.run.smithi129.stdout:5669 punch from 0x212229e to 0x2122bfc, (0x95e bytes) 2024-09-17T21:30:54.805 INFO:teuthology.orchestra.run.smithi129.stdout:5670 read 0x6e3536c thru 0x6e42851 (0xd4e6 bytes) 2024-09-17T21:30:54.805 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:54.805 INFO:teuthology.orchestra.run.smithi129.stdout:5673 write 0x1d7b978 thru 0x1d8146d (0x5af6 bytes) 2024-09-17T21:30:54.810 INFO:teuthology.orchestra.run.smithi129.stdout:5675 trunc from 0xdfed4e8 to 0xb600e75 2024-09-17T21:30:54.821 INFO:teuthology.orchestra.run.smithi129.stdout:5676 read 0x826fe1f thru 0x82712ae (0x1490 bytes) 2024-09-17T21:30:54.821 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:54.822 INFO:teuthology.orchestra.run.smithi129.stdout:5678 writesame 0x5cfc33e thru 0x5d005c9 data_size 0x2(0x428c bytes) 2024-09-17T21:30:54.827 INFO:teuthology.orchestra.run.smithi129.stdout:5681 punch from 0x6dbda68 to 0x6dc597b, (0x7f13 bytes) 2024-09-17T21:30:54.827 INFO:teuthology.orchestra.run.smithi129.stdout:5682 punch from 0xb5d5af3 to 0xb5db248, (0x5755 bytes) 2024-09-17T21:30:54.835 INFO:teuthology.orchestra.run.smithi129.stdout:5684 writesame 0xc7e2fd8 thru 0xc7ed7ff data_size 0x4(0xa828 bytes) 2024-09-17T21:30:54.836 INFO:teuthology.orchestra.run.smithi129.stdout:5685 read 0xae9627b thru 0xaea189e (0xb624 bytes) 2024-09-17T21:30:54.836 INFO:teuthology.orchestra.run.smithi129.stdout:5687 read 0x26a0c5f thru 0x26b035f (0xf701 bytes) 2024-09-17T21:30:54.836 INFO:teuthology.orchestra.run.smithi129.stdout:5688 writesame 0xba91a5f thru 0xba9a567 data_size 0x1(0x8b09 bytes) 2024-09-17T21:30:54.851 INFO:teuthology.orchestra.run.smithi129.stdout:5689 write 0xb83405c thru 0xb8431a3 (0xf148 bytes) 2024-09-17T21:30:54.855 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:54.855 INFO:teuthology.orchestra.run.smithi129.stdout:5691 writesame 0x3dec860 thru 0x3df5b83 data_size 0x81(0x9324 bytes) 2024-09-17T21:30:54.857 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:54.857 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:54.857 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:54.857 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:54.857 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:54.858 INFO:teuthology.orchestra.run.smithi129.stdout:5692 punch from 0x3fcdb30 to 0x3fdcf61, (0xf431 bytes) 2024-09-17T21:30:54.859 INFO:teuthology.orchestra.run.smithi129.stdout:5693 writesame 0x6f1dc5b thru 0x6f28c22 data_size 0x96(0xafc8 bytes) 2024-09-17T21:30:54.862 INFO:teuthology.orchestra.run.smithi129.stdout:5694 write 0x1f6bbb6 thru 0x1f712e5 (0x5730 bytes) 2024-09-17T21:30:54.864 INFO:teuthology.orchestra.run.smithi129.stdout:5695 write 0xc6f453a thru 0xc6f815a (0x3c21 bytes) 2024-09-17T21:30:54.865 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:54.868 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:30:54.875 INFO:teuthology.orchestra.run.smithi129.stdout:5697 trunc from 0xc7ed800 to 0x4d81342 2024-09-17T21:30:54.897 INFO:teuthology.orchestra.run.smithi129.stdout:5699 trunc from 0x4d81342 to 0xc845026 2024-09-17T21:30:54.897 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:54.895+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:54.903 INFO:teuthology.orchestra.run.smithi129.stdout:5700 punch from 0x5849dc3 to 0x5853299, (0x94d6 bytes) 2024-09-17T21:30:54.903 INFO:teuthology.orchestra.run.smithi129.stdout:5702 read 0x89b6d2a thru 0x89bc87c (0x5b53 bytes) 2024-09-17T21:30:54.903 INFO:teuthology.orchestra.run.smithi129.stdout:5703 write 0x689cb05 thru 0x68aae08 (0xe304 bytes) 2024-09-17T21:30:54.910 INFO:teuthology.orchestra.run.smithi129.stdout:5704 read 0x498d5cc thru 0x4995512 (0x7f47 bytes) 2024-09-17T21:30:54.910 INFO:teuthology.orchestra.run.smithi129.stdout:5705 read 0xbb16a78 thru 0xbb1ed20 (0x82a9 bytes) 2024-09-17T21:30:54.914 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:30:54.918 INFO:teuthology.orchestra.run.smithi129.stdout:5707 writesame 0xe3fa0bd thru 0xe3fcbd8 data_size 0x4(0x2b1c bytes) 2024-09-17T21:30:54.918 INFO:teuthology.orchestra.run.smithi129.stdout:5709 writesame 0xce388d4 thru 0xce40255 data_size 0x2(0x7982 bytes) 2024-09-17T21:30:54.930 INFO:teuthology.orchestra.run.smithi129.stdout:5712 write 0x5b6e7b5 thru 0x5b764f6 (0x7d42 bytes) 2024-09-17T21:30:54.933 INFO:teuthology.orchestra.run.smithi129.stdout:5713 write 0xd283142 thru 0xd2918d5 (0xe794 bytes) 2024-09-17T21:30:54.937 INFO:teuthology.orchestra.run.smithi129.stdout:5714 read 0xb52eb3a thru 0xb5372ea (0x87b1 bytes) 2024-09-17T21:30:54.937 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:54.937 INFO:teuthology.orchestra.run.smithi129.stdout:5716 read 0x787fcaf thru 0x788b5e2 (0xb934 bytes) 2024-09-17T21:30:54.937 INFO:teuthology.orchestra.run.smithi129.stdout:5717 write 0xe1f5331 thru 0xe1f8dee (0x3abe bytes) 2024-09-17T21:30:54.941 INFO:teuthology.orchestra.run.smithi129.stdout:5718 read 0x43a0b01 thru 0x43a97d9 (0x8cd9 bytes) 2024-09-17T21:30:54.945 INFO:teuthology.orchestra.run.smithi129.stdout:5719 trunc from 0xe3fcbd9 to 0xa713f78 2024-09-17T21:30:54.957 INFO:teuthology.orchestra.run.smithi129.stdout:5720 read 0x7ad71cc thru 0x7adecc3 (0x7af8 bytes) 2024-09-17T21:30:54.967 INFO:teuthology.orchestra.run.smithi129.stdout:5722 trunc from 0xa713f78 to 0x23e5127 2024-09-17T21:30:54.981 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:54.982 INFO:teuthology.orchestra.run.smithi129.stdout:5724 read 0x80419f thru 0x8090d3 (0x4f35 bytes) 2024-09-17T21:30:54.982 INFO:teuthology.orchestra.run.smithi129.stdout:5725 read 0x158a8f3 thru 0x15902e5 (0x59f3 bytes) 2024-09-17T21:30:54.992 INFO:teuthology.orchestra.run.smithi129.stdout:5727 writesame 0x55a2079 thru 0x55a26cf data_size 0x1(0x657 bytes) 2024-09-17T21:30:54.992 INFO:teuthology.orchestra.run.smithi129.stdout:5728 read 0x49c21e thru 0x4a7fcc (0xbdaf bytes) 2024-09-17T21:30:54.997 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:54.995+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:55.000 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:55.001 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:55.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:55.001 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:55.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.001 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97777 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 196608~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.001 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966872+0000", 2024-09-17T21:30:55.001 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.031018537, 2024-09-17T21:30:55.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050666048999999998, 2024-09-17T21:30:55.001 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.001 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.001 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.001 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.001 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.002 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.002 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97777 2024-09-17T21:30:55.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.002 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966872+0000", 2024-09-17T21:30:55.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966872+0000", 2024-09-17T21:30:55.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966872+0000", 2024-09-17T21:30:55.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0399999999999996e-07 2024-09-17T21:30:55.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966874+0000", 2024-09-17T21:30:55.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0930000000000001e-06 2024-09-17T21:30:55.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966881+0000", 2024-09-17T21:30:55.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9070000000000001e-06 2024-09-17T21:30:55.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966886+0000", 2024-09-17T21:30:55.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7700000000000001e-06 2024-09-17T21:30:55.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017200+0000", 2024-09-17T21:30:55.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050314336000000001 2024-09-17T21:30:55.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017284+0000", 2024-09-17T21:30:55.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3968000000000007e-05 2024-09-17T21:30:55.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017538+0000", 2024-09-17T21:30:55.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025347100000000001 2024-09-17T21:30:55.006 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.006 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.006 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.006 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97778 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 245760~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.006 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:30:55.006 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.030991995000001, 2024-09-17T21:30:55.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050933728999999997, 2024-09-17T21:30:55.006 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.006 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.007 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.007 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.007 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.007 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.007 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97778 2024-09-17T21:30:55.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.007 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:30:55.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:30:55.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966899+0000", 2024-09-17T21:30:55.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:30:55.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966900+0000", 2024-09-17T21:30:55.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.384e-06 2024-09-17T21:30:55.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966905+0000", 2024-09-17T21:30:55.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6530000000000002e-06 2024-09-17T21:30:55.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966909+0000", 2024-09-17T21:30:55.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2889999999999999e-06 2024-09-17T21:30:55.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017585+0000", 2024-09-17T21:30:55.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050675691000000002 2024-09-17T21:30:55.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017642+0000", 2024-09-17T21:30:55.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7167999999999999e-05 2024-09-17T21:30:55.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017832+0000", 2024-09-17T21:30:55.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019022 2024-09-17T21:30:55.011 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.011 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.011 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.011 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97779 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.011 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:30:55.011 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.030974927000001, 2024-09-17T21:30:55.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051245027999999998, 2024-09-17T21:30:55.012 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.012 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.012 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.012 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.012 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.012 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.012 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97779 2024-09-17T21:30:55.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.012 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:30:55.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:30:55.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966916+0000", 2024-09-17T21:30:55.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:30:55.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966918+0000", 2024-09-17T21:30:55.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8369999999999999e-06 2024-09-17T21:30:55.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966924+0000", 2024-09-17T21:30:55.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2260000000000002e-06 2024-09-17T21:30:55.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966928+0000", 2024-09-17T21:30:55.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3819999999999997e-06 2024-09-17T21:30:55.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017866+0000", 2024-09-17T21:30:55.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050938079999999997 2024-09-17T21:30:55.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017945+0000", 2024-09-17T21:30:55.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8275000000000005e-05 2024-09-17T21:30:55.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018161+0000", 2024-09-17T21:30:55.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021579499999999999 2024-09-17T21:30:55.016 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.016 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.016 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.016 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97780 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 344064~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.016 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:30:55.016 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.030950648999999, 2024-09-17T21:30:55.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051535301999999998, 2024-09-17T21:30:55.016 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.016 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.016 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.016 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.017 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.017 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.017 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97780 2024-09-17T21:30:55.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.017 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:30:55.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:30:55.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:30:55.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:30:55.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966942+0000", 2024-09-17T21:30:55.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7439999999999999e-06 2024-09-17T21:30:55.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966946+0000", 2024-09-17T21:30:55.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5510000000000001e-06 2024-09-17T21:30:55.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966951+0000", 2024-09-17T21:30:55.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:30:55.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018191+0000", 2024-09-17T21:30:55.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051240290000000001 2024-09-17T21:30:55.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018269+0000", 2024-09-17T21:30:55.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7416999999999994e-05 2024-09-17T21:30:55.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018475+0000", 2024-09-17T21:30:55.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206486 2024-09-17T21:30:55.020 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.020 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.020 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.021 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97789 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 40960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.021 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:30:55.021 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.03093048, 2024-09-17T21:30:55.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051862143999999999, 2024-09-17T21:30:55.021 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.021 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.021 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.021 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.021 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.021 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.021 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97789 2024-09-17T21:30:55.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.021 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:30:55.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:30:55.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:30:55.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5899999999999996e-07 2024-09-17T21:30:55.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966962+0000", 2024-09-17T21:30:55.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9700000000000002e-06 2024-09-17T21:30:55.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966969+0000", 2024-09-17T21:30:55.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6939999999999998e-06 2024-09-17T21:30:55.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966980+0000", 2024-09-17T21:30:55.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0679999999999999e-05 2024-09-17T21:30:55.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018513+0000", 2024-09-17T21:30:55.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051533420000000003 2024-09-17T21:30:55.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018595+0000", 2024-09-17T21:30:55.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1260999999999993e-05 2024-09-17T21:30:55.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018822+0000", 2024-09-17T21:30:55.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022756 2024-09-17T21:30:55.025 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.025 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.025 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.025 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97790 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 90112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.025 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:30:55.025 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.030899981999999, 2024-09-17T21:30:55.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052173479000000002, 2024-09-17T21:30:55.025 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.025 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.026 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.026 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.026 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.026 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.026 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97790 2024-09-17T21:30:55.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.026 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:30:55.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:30:55.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966991+0000", 2024-09-17T21:30:55.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9500000000000003e-07 2024-09-17T21:30:55.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966993+0000", 2024-09-17T21:30:55.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.238e-06 2024-09-17T21:30:55.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967000+0000", 2024-09-17T21:30:55.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9500000000000004e-06 2024-09-17T21:30:55.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967005+0000", 2024-09-17T21:30:55.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:30:55.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018852+0000", 2024-09-17T21:30:55.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051847250999999997 2024-09-17T21:30:55.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018927+0000", 2024-09-17T21:30:55.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4944000000000006e-05 2024-09-17T21:30:55.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019164+0000", 2024-09-17T21:30:55.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000236803 2024-09-17T21:30:55.030 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.030 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.030 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.030 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97791 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 139264~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.030 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:30:55.030 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.030878238, 2024-09-17T21:30:55.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052464594000000003, 2024-09-17T21:30:55.030 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.030 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.031 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.031 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.031 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.031 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.031 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97791 2024-09-17T21:30:55.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.031 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:30:55.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:30:55.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967013+0000", 2024-09-17T21:30:55.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:30:55.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967014+0000", 2024-09-17T21:30:55.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8679999999999999e-06 2024-09-17T21:30:55.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967020+0000", 2024-09-17T21:30:55.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.322e-06 2024-09-17T21:30:55.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967024+0000", 2024-09-17T21:30:55.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2660000000000003e-06 2024-09-17T21:30:55.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019193+0000", 2024-09-17T21:30:55.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052169010000000002 2024-09-17T21:30:55.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019269+0000", 2024-09-17T21:30:55.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6149999999999994e-05 2024-09-17T21:30:55.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019477+0000", 2024-09-17T21:30:55.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020753699999999999 2024-09-17T21:30:55.034 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.035 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.035 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.035 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97792 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 188416~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.035 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:30:55.035 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.030860498999999, 2024-09-17T21:30:55.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052764958000000001, 2024-09-17T21:30:55.035 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.035 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.035 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.035 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.035 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.036 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.036 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97792 2024-09-17T21:30:55.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.036 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:30:55.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:30:55.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:30:55.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0900000000000002e-07 2024-09-17T21:30:55.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967032+0000", 2024-09-17T21:30:55.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9740000000000001e-06 2024-09-17T21:30:55.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967039+0000", 2024-09-17T21:30:55.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3679999999999998e-06 2024-09-17T21:30:55.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967043+0000", 2024-09-17T21:30:55.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3379999999999998e-06 2024-09-17T21:30:55.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019505+0000", 2024-09-17T21:30:55.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052462264000000002 2024-09-17T21:30:55.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019582+0000", 2024-09-17T21:30:55.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7146999999999998e-05 2024-09-17T21:30:55.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019795+0000", 2024-09-17T21:30:55.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000212458 2024-09-17T21:30:55.039 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.039 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.039 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.039 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97793 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 237568~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.039 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:30:55.040 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.030838999, 2024-09-17T21:30:55.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053070436999999998, 2024-09-17T21:30:55.040 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.040 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.040 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.040 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.040 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.040 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.040 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97793 2024-09-17T21:30:55.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.040 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:30:55.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:30:55.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967052+0000", 2024-09-17T21:30:55.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7500000000000001e-07 2024-09-17T21:30:55.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967054+0000", 2024-09-17T21:30:55.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:30:55.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967059+0000", 2024-09-17T21:30:55.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5840000000000002e-06 2024-09-17T21:30:55.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967064+0000", 2024-09-17T21:30:55.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:30:55.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019826+0000", 2024-09-17T21:30:55.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052762396000000003 2024-09-17T21:30:55.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019900+0000", 2024-09-17T21:30:55.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3594e-05 2024-09-17T21:30:55.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020122+0000", 2024-09-17T21:30:55.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022185999999999999 2024-09-17T21:30:55.044 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.044 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.044 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.044 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97794 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 286720~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.044 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:30:55.044 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.030818863, 2024-09-17T21:30:55.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053453734000000003, 2024-09-17T21:30:55.044 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.044 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.045 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.045 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.045 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.045 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.045 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97794 2024-09-17T21:30:55.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.045 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:30:55.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:30:55.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:30:55.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2199999999999999e-07 2024-09-17T21:30:55.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967074+0000", 2024-09-17T21:30:55.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8419999999999999e-06 2024-09-17T21:30:55.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967080+0000", 2024-09-17T21:30:55.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5660000000000003e-06 2024-09-17T21:30:55.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967120+0000", 2024-09-17T21:30:55.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9504e-05 2024-09-17T21:30:55.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020189+0000", 2024-09-17T21:30:55.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053069144999999998 2024-09-17T21:30:55.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020287+0000", 2024-09-17T21:30:55.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8259999999999998e-05 2024-09-17T21:30:55.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020525+0000", 2024-09-17T21:30:55.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023799499999999999 2024-09-17T21:30:55.048 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.048 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.049 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.049 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97795 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 335872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.049 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:30:55.049 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.030760729000001, 2024-09-17T21:30:55.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053707207999999999, 2024-09-17T21:30:55.049 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.049 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.049 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.049 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.049 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.049 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.049 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97795 2024-09-17T21:30:55.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.050 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:30:55.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:30:55.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:30:55.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:30:55.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967139+0000", 2024-09-17T21:30:55.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5790000000000004e-06 2024-09-17T21:30:55.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967145+0000", 2024-09-17T21:30:55.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3550000000000001e-06 2024-09-17T21:30:55.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967155+0000", 2024-09-17T21:30:55.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8859999999999999e-06 2024-09-17T21:30:55.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020555+0000", 2024-09-17T21:30:55.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053400359000000001 2024-09-17T21:30:55.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020630+0000", 2024-09-17T21:30:55.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4894999999999994e-05 2024-09-17T21:30:55.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020837+0000", 2024-09-17T21:30:55.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020670500000000001 2024-09-17T21:30:55.053 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.053 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.053 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.053 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97796 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 385024~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.053 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:30:55.053 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.030728469, 2024-09-17T21:30:55.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053990227000000002, 2024-09-17T21:30:55.054 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.054 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.054 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.054 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.054 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.054 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.054 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97796 2024-09-17T21:30:55.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.054 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:30:55.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:30:55.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:30:55.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:30:55.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967164+0000", 2024-09-17T21:30:55.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0370000000000001e-06 2024-09-17T21:30:55.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967171+0000", 2024-09-17T21:30:55.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2190000000000001e-06 2024-09-17T21:30:55.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967175+0000", 2024-09-17T21:30:55.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2599999999999999e-06 2024-09-17T21:30:55.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020866+0000", 2024-09-17T21:30:55.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053691539000000003 2024-09-17T21:30:55.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020942+0000", 2024-09-17T21:30:55.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5487000000000004e-05 2024-09-17T21:30:55.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021152+0000", 2024-09-17T21:30:55.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021021599999999999 2024-09-17T21:30:55.058 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.058 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.058 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.058 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97805 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 8192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.058 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:30:55.058 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.030709788999999, 2024-09-17T21:30:55.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054331771000000001, 2024-09-17T21:30:55.059 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.059 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.059 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.059 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.059 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.059 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.059 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97805 2024-09-17T21:30:55.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.059 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:30:55.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:30:55.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:30:55.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:30:55.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967183+0000", 2024-09-17T21:30:55.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6840000000000001e-06 2024-09-17T21:30:55.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967189+0000", 2024-09-17T21:30:55.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0299999999999999e-06 2024-09-17T21:30:55.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967193+0000", 2024-09-17T21:30:55.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-17T21:30:55.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021182+0000", 2024-09-17T21:30:55.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053989319000000001 2024-09-17T21:30:55.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021261+0000", 2024-09-17T21:30:55.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8460000000000004e-05 2024-09-17T21:30:55.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021512+0000", 2024-09-17T21:30:55.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025158500000000002 2024-09-17T21:30:55.063 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.063 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.063 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.063 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97806 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.063 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:30:55.063 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.030691786, 2024-09-17T21:30:55.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054654343000000001, 2024-09-17T21:30:55.063 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.064 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.064 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.064 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.064 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.064 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.064 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97806 2024-09-17T21:30:55.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.064 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:30:55.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:30:55.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:30:55.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:30:55.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967201+0000", 2024-09-17T21:30:55.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.691e-06 2024-09-17T21:30:55.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967206+0000", 2024-09-17T21:30:55.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7780000000000004e-06 2024-09-17T21:30:55.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967224+0000", 2024-09-17T21:30:55.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7357999999999999e-05 2024-09-17T21:30:55.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021546+0000", 2024-09-17T21:30:55.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054322237000000002 2024-09-17T21:30:55.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021618+0000", 2024-09-17T21:30:55.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2007000000000004e-05 2024-09-17T21:30:55.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021853+0000", 2024-09-17T21:30:55.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023495800000000001 2024-09-17T21:30:55.068 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.068 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.068 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.068 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97807 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 106496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.068 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:30:55.068 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.030659386, 2024-09-17T21:30:55.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055014183000000001, 2024-09-17T21:30:55.068 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.068 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.068 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.069 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.069 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.069 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.069 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97807 2024-09-17T21:30:55.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.069 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:30:55.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:30:55.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967232+0000", 2024-09-17T21:30:55.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:30:55.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967234+0000", 2024-09-17T21:30:55.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.283e-06 2024-09-17T21:30:55.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967240+0000", 2024-09-17T21:30:55.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.144e-06 2024-09-17T21:30:55.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967252+0000", 2024-09-17T21:30:55.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2322e-05 2024-09-17T21:30:55.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021882+0000", 2024-09-17T21:30:55.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054629952000000002 2024-09-17T21:30:55.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021952+0000", 2024-09-17T21:30:55.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9919999999999995e-05 2024-09-17T21:30:55.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022245+0000", 2024-09-17T21:30:55.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00029302100000000003 2024-09-17T21:30:55.072 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.072 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.072 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.073 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97808 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 155648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.073 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:30:55.073 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.030630787, 2024-09-17T21:30:55.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055447485999999997, 2024-09-17T21:30:55.073 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.073 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.073 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.073 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.073 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.073 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.073 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97808 2024-09-17T21:30:55.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.073 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:30:55.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:30:55.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:30:55.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4299999999999998e-07 2024-09-17T21:30:55.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967267+0000", 2024-09-17T21:30:55.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9820000000000002e-06 2024-09-17T21:30:55.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967273+0000", 2024-09-17T21:30:55.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:30:55.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967277+0000", 2024-09-17T21:30:55.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5050000000000001e-06 2024-09-17T21:30:55.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022297+0000", 2024-09-17T21:30:55.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055020131999999999 2024-09-17T21:30:55.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022379+0000", 2024-09-17T21:30:55.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1810000000000004e-05 2024-09-17T21:30:55.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022707+0000", 2024-09-17T21:30:55.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00032784300000000003 2024-09-17T21:30:55.077 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.077 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.077 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.077 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97809 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.077 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:30:55.077 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.030607271999999, 2024-09-17T21:30:55.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056012109999999997, 2024-09-17T21:30:55.077 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.077 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.077 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.078 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.078 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.078 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.078 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97809 2024-09-17T21:30:55.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.078 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:30:55.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:30:55.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967284+0000", 2024-09-17T21:30:55.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:30:55.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967285+0000", 2024-09-17T21:30:55.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.826e-06 2024-09-17T21:30:55.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967291+0000", 2024-09-17T21:30:55.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:30:55.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967302+0000", 2024-09-17T21:30:55.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1114999999999999e-05 2024-09-17T21:30:55.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022901+0000", 2024-09-17T21:30:55.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055599335 2024-09-17T21:30:55.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022990+0000", 2024-09-17T21:30:55.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8671999999999994e-05 2024-09-17T21:30:55.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023295+0000", 2024-09-17T21:30:55.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030554900000000001 2024-09-17T21:30:55.081 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.082 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.082 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.082 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97810 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 253952~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.082 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:30:55.082 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.030580843999999, 2024-09-17T21:30:55.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056355668999999997, 2024-09-17T21:30:55.082 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.082 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.082 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.082 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.082 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.082 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.083 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97810 2024-09-17T21:30:55.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.083 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:30:55.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:30:55.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:30:55.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0200000000000002e-07 2024-09-17T21:30:55.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967313+0000", 2024-09-17T21:30:55.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4549999999999998e-06 2024-09-17T21:30:55.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967319+0000", 2024-09-17T21:30:55.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3080000000000004e-06 2024-09-17T21:30:55.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967324+0000", 2024-09-17T21:30:55.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0159999999999999e-06 2024-09-17T21:30:55.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023342+0000", 2024-09-17T21:30:55.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056018635999999997 2024-09-17T21:30:55.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023423+0000", 2024-09-17T21:30:55.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0473e-05 2024-09-17T21:30:55.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023665+0000", 2024-09-17T21:30:55.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242279 2024-09-17T21:30:55.086 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.087 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.087 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.087 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97811 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 303104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.087 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:30:55.087 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.030554585999999, 2024-09-17T21:30:55.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056743978, 2024-09-17T21:30:55.087 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.087 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.087 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.087 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.087 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.088 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.088 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97811 2024-09-17T21:30:55.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.088 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:30:55.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:30:55.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:30:55.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1300000000000001e-07 2024-09-17T21:30:55.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967338+0000", 2024-09-17T21:30:55.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9740000000000001e-06 2024-09-17T21:30:55.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967346+0000", 2024-09-17T21:30:55.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8129999999999996e-06 2024-09-17T21:30:55.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967351+0000", 2024-09-17T21:30:55.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6129999999999997e-06 2024-09-17T21:30:55.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023750+0000", 2024-09-17T21:30:55.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056399796000000002 2024-09-17T21:30:55.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023839+0000", 2024-09-17T21:30:55.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.852e-05 2024-09-17T21:30:55.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024080+0000", 2024-09-17T21:30:55.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024084900000000001 2024-09-17T21:30:55.091 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.091 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.092 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.092 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97812 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 352256~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:55.092 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:30:55.092 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.03053397, 2024-09-17T21:30:55.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.057024455000000002, 2024-09-17T21:30:55.092 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.092 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.092 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.092 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.092 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.092 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.093 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97812 2024-09-17T21:30:55.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.093 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:30:55.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:30:55.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967357+0000", 2024-09-17T21:30:55.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:30:55.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967359+0000", 2024-09-17T21:30:55.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:30:55.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967365+0000", 2024-09-17T21:30:55.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:30:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967369+0000", 2024-09-17T21:30:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5029999999999999e-06 2024-09-17T21:30:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024109+0000", 2024-09-17T21:30:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056740038 2024-09-17T21:30:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024175+0000", 2024-09-17T21:30:55.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5106999999999999e-05 2024-09-17T21:30:55.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024381+0000", 2024-09-17T21:30:55.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020629700000000001 2024-09-17T21:30:55.096 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.096 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.096 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.096 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.096 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.096 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:55.097 INFO:teuthology.orchestra.run.smithi129.stdout:5729 writesame 0x7746368 thru 0x7748982 data_size 0x1(0x261b bytes) 2024-09-17T21:30:55.097 INFO:teuthology.orchestra.run.smithi129.stdout:5730 read 0x1f5b196 thru 0x1f66a7f (0xb8ea bytes) 2024-09-17T21:30:55.097 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:55.097 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:55.098 INFO:teuthology.orchestra.run.smithi129.stdout:5734 writesame 0x62afacb thru 0x62b9641 data_size 0x1(0x9b77 bytes) 2024-09-17T21:30:55.099 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:30:55.100 INFO:teuthology.orchestra.run.smithi129.stdout:5735 write 0x9e19fe6 thru 0x9e270a1 (0xd0bc bytes) 2024-09-17T21:30:55.100 INFO:teuthology.orchestra.run.smithi129.stdout:5736 read 0x7aeceeb thru 0x7af3343 (0x6459 bytes) 2024-09-17T21:30:55.100 INFO:teuthology.orchestra.run.smithi129.stdout:5738 read 0x9356db0 thru 0x9364b5c (0xddad bytes) 2024-09-17T21:30:55.100 INFO:teuthology.orchestra.run.smithi129.stdout:5739 write 0x196fd6c thru 0x197a7b0 (0xaa45 bytes) 2024-09-17T21:30:55.100 INFO:teuthology.orchestra.run.smithi129.stdout:5740 writesame 0xba2f2a9 thru 0xba3b38d data_size 0x17(0xc0e5 bytes) 2024-09-17T21:30:55.100 INFO:teuthology.orchestra.run.smithi129.stdout:5742 write 0x566135d thru 0x566ff79 (0xec1d bytes) 2024-09-17T21:30:55.100 INFO:teuthology.orchestra.run.smithi129.stdout:5744 write 0xd405674 thru 0xd412f48 (0xd8d5 bytes) 2024-09-17T21:30:55.100 INFO:teuthology.orchestra.run.smithi129.stdout:5746 trunc from 0xd412f49 to 0x8128f4 2024-09-17T21:30:55.106 INFO:teuthology.orchestra.run.smithi129.stdout:5747 read 0x215b60 thru 0x21fe4d (0xa2ee bytes) 2024-09-17T21:30:55.106 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:55.104+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:55.107 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:55.107 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:55.128 INFO:teuthology.orchestra.run.smithi129.stdout:5750 writesame 0xd7aefca thru 0xd7bd9c9 data_size 0x48(0xea00 bytes) 2024-09-17T21:30:55.128 INFO:teuthology.orchestra.run.smithi129.stdout:5751 write 0x5ce8954 thru 0x5cef3a3 (0x6a50 bytes) 2024-09-17T21:30:55.131 INFO:teuthology.orchestra.run.smithi129.stdout:5754 writesame 0x66f0c64 thru 0x66f489c data_size 0x9(0x3c39 bytes) 2024-09-17T21:30:55.134 INFO:teuthology.orchestra.run.smithi129.stdout:5760 read 0x920d0c9 thru 0x92121a6 (0x50de bytes) 2024-09-17T21:30:55.145 INFO:teuthology.orchestra.run.smithi129.stdout:5762 trunc from 0xd7bd9ca to 0x4c016f8 2024-09-17T21:30:55.156 INFO:teuthology.orchestra.run.smithi129.stdout:5763 read 0x4bf8b3f thru 0x4c016f7 (0x8bb9 bytes) 2024-09-17T21:30:55.162 INFO:teuthology.orchestra.run.smithi129.stdout:5764 write 0x4ef02bc thru 0x4ef2dfe (0x2b43 bytes) 2024-09-17T21:30:55.165 INFO:teuthology.orchestra.run.smithi129.stdout:5765 flatten 2024-09-17T21:30:55.171 INFO:teuthology.orchestra.run.smithi129.stdout:5766 read 0x1396283 thru 0x13a4033 (0xddb1 bytes) 2024-09-17T21:30:55.173 INFO:teuthology.orchestra.run.smithi129.stdout:5767 trunc from 0x4ef2dff to 0x62b5536 2024-09-17T21:30:55.180 INFO:teuthology.orchestra.run.smithi129.stdout:5768 trunc from 0x62b5536 to 0x5f730f6 2024-09-17T21:30:55.186 INFO:teuthology.orchestra.run.smithi129.stdout:5769 read 0x4380306 thru 0x438448e (0x4189 bytes) 2024-09-17T21:30:55.186 INFO:teuthology.orchestra.run.smithi129.stdout:5770 trunc from 0x5f730f6 to 0x5f2c3a0 2024-09-17T21:30:55.202 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:55.200+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:55.203 INFO:teuthology.orchestra.run.smithi129.stdout:5771 write 0xab6ba3a thru 0xab78f09 (0xd4d0 bytes) 2024-09-17T21:30:55.206 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:55.207 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:55.210 INFO:teuthology.orchestra.run.smithi129.stdout:5774 trunc from 0xab78f0a to 0xd79dbfd 2024-09-17T21:30:55.217 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:55.224 INFO:teuthology.orchestra.run.smithi129.stdout:5777 trunc from 0xd79dbfd to 0x7b47232 2024-09-17T21:30:55.227 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:55.227 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:55.227 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:55.227 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:55.234 INFO:teuthology.orchestra.run.smithi129.stdout:5778 read 0xa93f1a thru 0xaa388f (0xf976 bytes) 2024-09-17T21:30:55.235 INFO:teuthology.orchestra.run.smithi129.stdout:5781 punch from 0x29abb0e to 0x29b9cd8, (0xe1ca bytes) 2024-09-17T21:30:55.237 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:30:55.241 INFO:teuthology.orchestra.run.smithi129.stdout:5782 write 0x88f1366 thru 0x88ffc1b (0xe8b6 bytes) 2024-09-17T21:30:55.244 INFO:teuthology.orchestra.run.smithi129.stdout:5784 write 0x7283489 thru 0x7286c11 (0x3789 bytes) 2024-09-17T21:30:55.248 INFO:teuthology.orchestra.run.smithi129.stdout:5786 read 0x2f91cb4 thru 0x2f92e4b (0x1198 bytes) 2024-09-17T21:30:55.248 INFO:teuthology.orchestra.run.smithi129.stdout:5787 read 0x5530e48 thru 0x553d8d4 (0xca8d bytes) 2024-09-17T21:30:55.259 INFO:teuthology.orchestra.run.smithi129.stdout:5788 writesame 0xb976b1f thru 0xb97c8df data_size 0x1(0x5dc1 bytes) 2024-09-17T21:30:55.259 INFO:teuthology.orchestra.run.smithi129.stdout:5789 write 0x1d4385d thru 0x1d49570 (0x5d14 bytes) 2024-09-17T21:30:55.260 INFO:teuthology.orchestra.run.smithi129.stdout:5790 writesame 0xa04a2e8 thru 0xa0516d9 data_size 0x66(0x73f2 bytes) 2024-09-17T21:30:55.262 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:55.262 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:55.271 INFO:teuthology.orchestra.run.smithi129.stdout:5793 trunc from 0xb97c8e0 to 0x46ac266 2024-09-17T21:30:55.282 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:55.282 INFO:teuthology.orchestra.run.smithi129.stdout:5795 write 0x28b9a89 thru 0x28c5fa6 (0xc51e bytes) 2024-09-17T21:30:55.284 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:55.298 INFO:teuthology.orchestra.run.smithi129.stdout:5798 write 0x98f932d thru 0x98fb97e (0x2652 bytes) 2024-09-17T21:30:55.301 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:55.301 INFO:teuthology.orchestra.run.smithi129.stdout:5800 write 0x62ad9b3 thru 0x62b8813 (0xae61 bytes) 2024-09-17T21:30:55.303 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:55.301+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:55.305 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:55.305 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:55.305 INFO:teuthology.orchestra.run.smithi129.stdout:5805 read 0x3d9a9b4 thru 0x3d9e4df (0x3b2c bytes) 2024-09-17T21:30:55.310 INFO:teuthology.orchestra.run.smithi129.stdout:5806 writesame 0x99e6999 thru 0x99eedcb data_size 0x1d(0x8433 bytes) 2024-09-17T21:30:55.311 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:55.311 INFO:teuthology.orchestra.run.smithi129.stdout:5809 read 0x2f2d4dd thru 0x2f2fa43 (0x2567 bytes) 2024-09-17T21:30:55.311 INFO:teuthology.orchestra.run.smithi129.stdout:5811 punch from 0x5290aea to 0x529f7e3, (0xecf9 bytes) 2024-09-17T21:30:55.316 INFO:teuthology.orchestra.run.smithi129.stdout:5812 write 0x9d79b72 thru 0x9d86cf6 (0xd185 bytes) 2024-09-17T21:30:55.322 INFO:teuthology.orchestra.run.smithi129.stdout:5814 punch from 0x9821acd to 0x9823fed, (0x2520 bytes) 2024-09-17T21:30:55.322 INFO:teuthology.orchestra.run.smithi129.stdout:5815 writesame 0x831f377 thru 0x832655d data_size 0x1(0x71e7 bytes) 2024-09-17T21:30:55.340 INFO:teuthology.orchestra.run.smithi129.stdout:5816 punch from 0x5c8171a to 0x5c899be, (0x82a4 bytes) 2024-09-17T21:30:55.345 INFO:teuthology.orchestra.run.smithi129.stdout:5817 trunc from 0x9d86cf7 to 0xd71b164 2024-09-17T21:30:55.350 INFO:teuthology.orchestra.run.smithi129.stdout:5818 clone 44 order 25 su 4194304 sc 8 2024-09-17T21:30:55.362 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:55.363 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:55.363 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:55.363 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:55.363 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:55.371 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:30:55.404 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:30:55.405 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:55.403+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:55.506 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:55.503+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:55.519 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:37817 3.17 3:eab85a8d:::rbd_header.1120e62eea89:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:30:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.821423381000002, 2024-09-17T21:30:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86635970399999995, 2024-09-17T21:30:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37817 2024-09-17T21:30:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695457+0000", 2024-09-17T21:30:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5799999999999998e-07 2024-09-17T21:30:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695461+0000", 2024-09-17T21:30:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1110000000000001e-06 2024-09-17T21:30:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695471+0000", 2024-09-17T21:30:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0441e-05 2024-09-17T21:30:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695477+0000", 2024-09-17T21:30:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7599999999999999e-06 2024-09-17T21:30:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695533+0000", 2024-09-17T21:30:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6570999999999998e-05 2024-09-17T21:30:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:30:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695543+0000", 2024-09-17T21:30:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9979999999999998e-06 2024-09-17T21:30:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561472+0000", 2024-09-17T21:30:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86592841899999995 2024-09-17T21:30:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561610+0000", 2024-09-17T21:30:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013854699999999999 2024-09-17T21:30:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561816+0000", 2024-09-17T21:30:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020509900000000001 2024-09-17T21:30:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50202 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.409252590999998, 2024-09-17T21:30:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.076620261999999995, 2024-09-17T21:30:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50202 2024-09-17T21:30:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107629+0000", 2024-09-17T21:30:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3709999999999998e-06 2024-09-17T21:30:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119765+0000", 2024-09-17T21:30:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012135683 2024-09-17T21:30:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119787+0000", 2024-09-17T21:30:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2748999999999999e-05 2024-09-17T21:30:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119797+0000", 2024-09-17T21:30:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8169999999999994e-06 2024-09-17T21:30:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119872+0000", 2024-09-17T21:30:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270999999999994e-05 2024-09-17T21:30:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119987+0000", 2024-09-17T21:30:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011557699999999999 2024-09-17T21:30:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.121766+0000", 2024-09-17T21:30:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00177853 2024-09-17T21:30:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178307+0000", 2024-09-17T21:30:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056541559999999998 2024-09-17T21:30:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184097+0000", 2024-09-17T21:30:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057899029999999999 2024-09-17T21:30:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184172+0000", 2024-09-17T21:30:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5226000000000001e-05 2024-09-17T21:30:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184247+0000", 2024-09-17T21:30:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4574999999999997e-05 2024-09-17T21:30:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50203 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [truncate 20971520] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.397052686000002, 2024-09-17T21:30:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065009779000000004, 2024-09-17T21:30:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50203 2024-09-17T21:30:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119828+0000", 2024-09-17T21:30:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6390000000000001e-06 2024-09-17T21:30:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119832+0000", 2024-09-17T21:30:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7699999999999999e-06 2024-09-17T21:30:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119845+0000", 2024-09-17T21:30:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3093e-05 2024-09-17T21:30:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119852+0000", 2024-09-17T21:30:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6950000000000003e-06 2024-09-17T21:30:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.160945+0000", 2024-09-17T21:30:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041093076999999999 2024-09-17T21:30:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161071+0000", 2024-09-17T21:30:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012569799999999999 2024-09-17T21:30:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161283+0000", 2024-09-17T21:30:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021201999999999999 2024-09-17T21:30:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.179360+0000", 2024-09-17T21:30:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018077784999999999 2024-09-17T21:30:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184726+0000", 2024-09-17T21:30:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053652960000000003 2024-09-17T21:30:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184787+0000", 2024-09-17T21:30:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1402000000000006e-05 2024-09-17T21:30:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184836+0000", 2024-09-17T21:30:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9304000000000001e-05 2024-09-17T21:30:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50206 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.359971145999999, 2024-09-17T21:30:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.090361968000000001, 2024-09-17T21:30:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50206 2024-09-17T21:30:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156911+0000", 2024-09-17T21:30:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-17T21:30:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173670+0000", 2024-09-17T21:30:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016758907 2024-09-17T21:30:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173696+0000", 2024-09-17T21:30:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6329000000000002e-05 2024-09-17T21:30:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173708+0000", 2024-09-17T21:30:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2512e-05 2024-09-17T21:30:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173788+0000", 2024-09-17T21:30:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9332999999999994e-05 2024-09-17T21:30:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173918+0000", 2024-09-17T21:30:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013063300000000001 2024-09-17T21:30:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.176228+0000", 2024-09-17T21:30:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002309757 2024-09-17T21:30:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.245164+0000", 2024-09-17T21:30:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068936301000000005 2024-09-17T21:30:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247176+0000", 2024-09-17T21:30:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020117590000000001 2024-09-17T21:30:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247215+0000", 2024-09-17T21:30:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9094000000000002e-05 2024-09-17T21:30:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247270+0000", 2024-09-17T21:30:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4793999999999997e-05 2024-09-17T21:30:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50207 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [truncate 21556327] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.343122166000001, 2024-09-17T21:30:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.074055363999999999, 2024-09-17T21:30:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50207 2024-09-17T21:30:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173759+0000", 2024-09-17T21:30:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8160000000000001e-06 2024-09-17T21:30:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173763+0000", 2024-09-17T21:30:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5360000000000003e-06 2024-09-17T21:30:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173771+0000", 2024-09-17T21:30:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8849999999999999e-06 2024-09-17T21:30:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173777+0000", 2024-09-17T21:30:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7799999999999997e-06 2024-09-17T21:30:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221363+0000", 2024-09-17T21:30:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047585493 2024-09-17T21:30:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221430+0000", 2024-09-17T21:30:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7129000000000001e-05 2024-09-17T21:30:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221550+0000", 2024-09-17T21:30:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120658 2024-09-17T21:30:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246113+0000", 2024-09-17T21:30:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02456262 2024-09-17T21:30:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247744+0000", 2024-09-17T21:30:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001630692 2024-09-17T21:30:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247777+0000", 2024-09-17T21:30:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3015000000000002e-05 2024-09-17T21:30:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247812+0000", 2024-09-17T21:30:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.574e-05 2024-09-17T21:30:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50828 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.035010233999998, 2024-09-17T21:30:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047324185999999997, 2024-09-17T21:30:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50828 2024-09-17T21:30:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.46e-07 2024-09-17T21:30:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481871+0000", 2024-09-17T21:30:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T21:30:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481878+0000", 2024-09-17T21:30:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:30:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481882+0000", 2024-09-17T21:30:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:30:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529038+0000", 2024-09-17T21:30:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047156034999999999 2024-09-17T21:30:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529099+0000", 2024-09-17T21:30:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0316000000000003e-05 2024-09-17T21:30:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529193+0000", 2024-09-17T21:30:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4369000000000004e-05 2024-09-17T21:30:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50829 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 12845056~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.034991439999999, 2024-09-17T21:30:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047887596999999997, 2024-09-17T21:30:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50829 2024-09-17T21:30:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:30:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481890+0000", 2024-09-17T21:30:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.877e-06 2024-09-17T21:30:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481895+0000", 2024-09-17T21:30:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:30:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481900+0000", 2024-09-17T21:30:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4440000000000001e-06 2024-09-17T21:30:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529225+0000", 2024-09-17T21:30:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047325117999999999 2024-09-17T21:30:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529292+0000", 2024-09-17T21:30:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7100000000000005e-05 2024-09-17T21:30:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529775+0000", 2024-09-17T21:30:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048322799999999999 2024-09-17T21:30:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50830 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.034973037999997, 2024-09-17T21:30:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048062123999999998, 2024-09-17T21:30:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50830 2024-09-17T21:30:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481907+0000", 2024-09-17T21:30:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:30:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481909+0000", 2024-09-17T21:30:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T21:30:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481915+0000", 2024-09-17T21:30:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.02e-06 2024-09-17T21:30:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481920+0000", 2024-09-17T21:30:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6799999999999998e-06 2024-09-17T21:30:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529793+0000", 2024-09-17T21:30:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047873042999999997 2024-09-17T21:30:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529870+0000", 2024-09-17T21:30:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6967999999999999e-05 2024-09-17T21:30:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529968+0000", 2024-09-17T21:30:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8085999999999996e-05 2024-09-17T21:30:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50831 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 18087936~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.034952691999997, 2024-09-17T21:30:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048663304999999997, 2024-09-17T21:30:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50831 2024-09-17T21:30:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:30:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481929+0000", 2024-09-17T21:30:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2189999999999998e-06 2024-09-17T21:30:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481935+0000", 2024-09-17T21:30:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.383e-06 2024-09-17T21:30:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481939+0000", 2024-09-17T21:30:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2479999999999998e-06 2024-09-17T21:30:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530095+0000", 2024-09-17T21:30:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048156617999999998 2024-09-17T21:30:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530159+0000", 2024-09-17T21:30:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3394999999999999e-05 2024-09-17T21:30:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530590+0000", 2024-09-17T21:30:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043101199999999999 2024-09-17T21:30:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50832 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 19660800~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.034934045, 2024-09-17T21:30:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049339466999999998, 2024-09-17T21:30:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50832 2024-09-17T21:30:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481946+0000", 2024-09-17T21:30:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481951+0000", 2024-09-17T21:30:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1479999999999997e-06 2024-09-17T21:30:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481956+0000", 2024-09-17T21:30:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7339999999999996e-06 2024-09-17T21:30:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481961+0000", 2024-09-17T21:30:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4889999999999997e-06 2024-09-17T21:30:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530630+0000", 2024-09-17T21:30:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048668596000000001 2024-09-17T21:30:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530720+0000", 2024-09-17T21:30:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0163999999999999e-05 2024-09-17T21:30:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531285+0000", 2024-09-17T21:30:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056493199999999998 2024-09-17T21:30:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50833 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.034911901000001, 2024-09-17T21:30:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049908069999999999, 2024-09-17T21:30:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50833 2024-09-17T21:30:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481968+0000", 2024-09-17T21:30:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5600000000000001e-07 2024-09-17T21:30:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481970+0000", 2024-09-17T21:30:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:30:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481975+0000", 2024-09-17T21:30:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8499999999999999e-06 2024-09-17T21:30:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481980+0000", 2024-09-17T21:30:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-17T21:30:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531337+0000", 2024-09-17T21:30:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049356874000000002 2024-09-17T21:30:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531392+0000", 2024-09-17T21:30:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4826e-05 2024-09-17T21:30:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531875+0000", 2024-09-17T21:30:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048345099999999998 2024-09-17T21:30:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50834 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 24903680~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.034887132999998, 2024-09-17T21:30:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050534015000000002, 2024-09-17T21:30:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50834 2024-09-17T21:30:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481993+0000", 2024-09-17T21:30:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:30:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481995+0000", 2024-09-17T21:30:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.199e-06 2024-09-17T21:30:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482001+0000", 2024-09-17T21:30:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.772e-06 2024-09-17T21:30:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482006+0000", 2024-09-17T21:30:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0660000000000003e-06 2024-09-17T21:30:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531923+0000", 2024-09-17T21:30:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049917200000000002 2024-09-17T21:30:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532004+0000", 2024-09-17T21:30:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0758000000000002e-05 2024-09-17T21:30:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532526+0000", 2024-09-17T21:30:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00052258899999999995 2024-09-17T21:30:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50835 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 28573696~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.034867149999997, 2024-09-17T21:30:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051204701999999998, 2024-09-17T21:30:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50835 2024-09-17T21:30:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482013+0000", 2024-09-17T21:30:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:30:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482015+0000", 2024-09-17T21:30:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.272e-06 2024-09-17T21:30:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482020+0000", 2024-09-17T21:30:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3040000000000004e-06 2024-09-17T21:30:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482025+0000", 2024-09-17T21:30:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5650000000000003e-06 2024-09-17T21:30:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532569+0000", 2024-09-17T21:30:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050544661999999997 2024-09-17T21:30:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532649+0000", 2024-09-17T21:30:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9640000000000003e-05 2024-09-17T21:30:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533217+0000", 2024-09-17T21:30:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056784100000000003 2024-09-17T21:30:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50836 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 30146560~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.034847532999997, 2024-09-17T21:30:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051875589, 2024-09-17T21:30:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50836 2024-09-17T21:30:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3700000000000001e-07 2024-09-17T21:30:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482034+0000", 2024-09-17T21:30:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0059999999999999e-06 2024-09-17T21:30:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482040+0000", 2024-09-17T21:30:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:30:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482044+0000", 2024-09-17T21:30:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:30:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533260+0000", 2024-09-17T21:30:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051215416 2024-09-17T21:30:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533348+0000", 2024-09-17T21:30:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8695000000000004e-05 2024-09-17T21:30:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533907+0000", 2024-09-17T21:30:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055902199999999995 2024-09-17T21:30:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50844 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.034684106999997, 2024-09-17T21:30:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0475644, 2024-09-17T21:30:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50844 2024-09-17T21:30:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482196+0000", 2024-09-17T21:30:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.39e-07 2024-09-17T21:30:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482198+0000", 2024-09-17T21:30:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0099999999999998e-06 2024-09-17T21:30:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482204+0000", 2024-09-17T21:30:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.93e-06 2024-09-17T21:30:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482208+0000", 2024-09-17T21:30:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7439999999999998e-06 2024-09-17T21:30:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529318+0000", 2024-09-17T21:30:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047110051999999999 2024-09-17T21:30:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529396+0000", 2024-09-17T21:30:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7712000000000004e-05 2024-09-17T21:30:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529760+0000", 2024-09-17T21:30:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00036351299999999999 2024-09-17T21:30:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50845 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 21757952~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.034665003999997, 2024-09-17T21:30:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048084625999999998, 2024-09-17T21:30:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50845 2024-09-17T21:30:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482215+0000", 2024-09-17T21:30:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:30:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482216+0000", 2024-09-17T21:30:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.823e-06 2024-09-17T21:30:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482222+0000", 2024-09-17T21:30:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.507e-06 2024-09-17T21:30:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482227+0000", 2024-09-17T21:30:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6609999999999999e-06 2024-09-17T21:30:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529792+0000", 2024-09-17T21:30:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047565075999999998 2024-09-17T21:30:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529871+0000", 2024-09-17T21:30:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8855000000000003e-05 2024-09-17T21:30:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530299+0000", 2024-09-17T21:30:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000428307 2024-09-17T21:30:55.599 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.599 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.599 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.599 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50846 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 25427968~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.599 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:55.599 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.034642617000003, 2024-09-17T21:30:55.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048735142000000002, 2024-09-17T21:30:55.599 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.599 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.599 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.599 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.599 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.600 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.600 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50846 2024-09-17T21:30:55.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.600 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:55.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:55.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:55.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:30:55.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482239+0000", 2024-09-17T21:30:55.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.871e-06 2024-09-17T21:30:55.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482244+0000", 2024-09-17T21:30:55.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.49e-06 2024-09-17T21:30:55.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482249+0000", 2024-09-17T21:30:55.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6269999999999999e-06 2024-09-17T21:30:55.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530347+0000", 2024-09-17T21:30:55.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048097721000000003 2024-09-17T21:30:55.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530437+0000", 2024-09-17T21:30:55.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0604000000000002e-05 2024-09-17T21:30:55.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530972+0000", 2024-09-17T21:30:55.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053445000000000001 2024-09-17T21:30:55.603 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.603 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.603 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.603 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50847 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 27000832~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.603 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:55.604 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.034624721, 2024-09-17T21:30:55.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049308668, 2024-09-17T21:30:55.604 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.604 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.604 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.604 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.604 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.604 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.604 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50847 2024-09-17T21:30:55.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.604 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:55.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:55.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:55.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:30:55.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482257+0000", 2024-09-17T21:30:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.846e-06 2024-09-17T21:30:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482264+0000", 2024-09-17T21:30:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8170000000000001e-06 2024-09-17T21:30:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482283+0000", 2024-09-17T21:30:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9718e-05 2024-09-17T21:30:55.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531018+0000", 2024-09-17T21:30:55.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048734497000000002 2024-09-17T21:30:55.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531080+0000", 2024-09-17T21:30:55.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1795e-05 2024-09-17T21:30:55.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531563+0000", 2024-09-17T21:30:55.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048366499999999998 2024-09-17T21:30:55.608 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.608 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.608 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.608 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50848 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 30670848~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.608 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:55.608 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.034588178999996, 2024-09-17T21:30:55.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049893337000000003, 2024-09-17T21:30:55.608 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.608 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.608 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.609 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.609 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.609 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.609 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50848 2024-09-17T21:30:55.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.609 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:55.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:55.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482292+0000", 2024-09-17T21:30:55.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T21:30:55.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482294+0000", 2024-09-17T21:30:55.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3209999999999999e-06 2024-09-17T21:30:55.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482300+0000", 2024-09-17T21:30:55.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7189999999999998e-06 2024-09-17T21:30:55.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482305+0000", 2024-09-17T21:30:55.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T21:30:55.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531606+0000", 2024-09-17T21:30:55.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049301273999999999 2024-09-17T21:30:55.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531696+0000", 2024-09-17T21:30:55.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0161999999999995e-05 2024-09-17T21:30:55.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532184+0000", 2024-09-17T21:30:55.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048806400000000002 2024-09-17T21:30:55.612 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.612 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.612 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.612 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50849 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 32243712~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:55.613 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:55.613 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.034568737999997, 2024-09-17T21:30:55.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050444607000000002, 2024-09-17T21:30:55.613 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:55.613 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:55.613 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:55.613 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:55.613 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:55.613 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:55.613 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50849 2024-09-17T21:30:55.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.613 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:55.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:55.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:55.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:55.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:55.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:55.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:55.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482311+0000", 2024-09-17T21:30:55.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:30:55.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482316+0000", 2024-09-17T21:30:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1209999999999998e-06 2024-09-17T21:30:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482321+0000", 2024-09-17T21:30:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4779999999999999e-06 2024-09-17T21:30:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482326+0000", 2024-09-17T21:30:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7099999999999998e-06 2024-09-17T21:30:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532221+0000", 2024-09-17T21:30:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049894936000000001 2024-09-17T21:30:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532287+0000", 2024-09-17T21:30:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5914000000000006e-05 2024-09-17T21:30:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532755+0000", 2024-09-17T21:30:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000468107 2024-09-17T21:30:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:55.617 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:55.618 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:55.616+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:55.619 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:30:55.619 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:55.619 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:55.620 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:30:55.620 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:30:55.620 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:55.621 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:55.621 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:55.621 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:55.621 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:55.622 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:55.622 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:55.622 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:55.631 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:55.631 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:55.648 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:55.648 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:55.719 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:55.717+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:55.735 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:55.735 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:55.743 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:30:55.820 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:55.817+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:55.846 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:55.846 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:55.855 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:30:55.913 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is set 2024-09-17T21:30:55.920 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:55.918+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:56.010 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:56.010 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:56.020 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:30:56.021 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:56.018+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:56.115 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-17T21:30:56.116 INFO:teuthology.orchestra.run.smithi129.stdout:5819 writesame 0x71580b6 thru 0x7167e98 data_size 0x23(0xfde3 bytes) 2024-09-17T21:30:56.122 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:56.119+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:56.124 INFO:teuthology.orchestra.run.smithi129.stdout:5820 read 0x98d5047 thru 0x98d5895 (0x84f bytes) 2024-09-17T21:30:56.124 INFO:teuthology.orchestra.run.smithi129.stdout:5821 writesame 0xb4aa868 thru 0xb4b1c27 data_size 0x20(0x73c0 bytes) 2024-09-17T21:30:56.128 INFO:teuthology.orchestra.run.smithi129.stdout:5823 writesame 0x7b9fd89 thru 0x7bad528 data_size 0x73(0xd7a0 bytes) 2024-09-17T21:30:56.132 INFO:teuthology.orchestra.run.smithi129.stdout:5824 punch from 0x641d64b to 0x641f187, (0x1b3c bytes) 2024-09-17T21:30:56.132 INFO:teuthology.orchestra.run.smithi129.stdout:5825 read 0xd1d99a3 thru 0xd1dc7a5 (0x2e03 bytes) 2024-09-17T21:30:56.132 INFO:teuthology.orchestra.run.smithi129.stdout:5826 read 0x54d9dbb thru 0x54dc0e1 (0x2327 bytes) 2024-09-17T21:30:56.133 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:56.133 INFO:teuthology.orchestra.run.smithi129.stdout:5828 write 0xc7c5ab1 thru 0xc7c6030 (0x580 bytes) 2024-09-17T21:30:56.137 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:56.137 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:56.137 INFO:teuthology.orchestra.run.smithi129.stdout:5829 write 0x4ca986e thru 0x4cb2e38 (0x95cb bytes) 2024-09-17T21:30:56.142 INFO:teuthology.orchestra.run.smithi129.stdout:5831 read 0x2653621 thru 0x26548df (0x12bf bytes) 2024-09-17T21:30:56.142 INFO:teuthology.orchestra.run.smithi129.stdout:5833 write 0x1905d6d thru 0x1908253 (0x24e7 bytes) 2024-09-17T21:30:56.144 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:56.144 INFO:teuthology.orchestra.run.smithi129.stdout:5835 write 0x43e2c7b thru 0x43e5047 (0x23cd bytes) 2024-09-17T21:30:56.148 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:30:56.160 INFO:teuthology.orchestra.run.smithi129.stdout:5837 write 0x386b610 thru 0x3870bdb (0x55cc bytes) 2024-09-17T21:30:56.162 INFO:teuthology.orchestra.run.smithi129.stdout:5838 writesame 0x1bf7ce9 thru 0x1c04681 data_size 0x1(0xc999 bytes) 2024-09-17T21:30:56.191 INFO:teuthology.orchestra.run.smithi129.stdout:5840 writesame 0x232c7c1 thru 0x2330182 data_size 0x1(0x39c2 bytes) 2024-09-17T21:30:56.198 INFO:teuthology.orchestra.run.smithi129.stdout:5841 write 0x82ad1b8 thru 0x82b9393 (0xc1dc bytes) 2024-09-17T21:30:56.200 INFO:teuthology.orchestra.run.smithi129.stdout:5842 read 0xa4ca04e thru 0xa4cdcac (0x3c5f bytes) 2024-09-17T21:30:56.200 INFO:teuthology.orchestra.run.smithi129.stdout:5844 punch from 0x3159a48 to 0x315a28f, (0x847 bytes) 2024-09-17T21:30:56.201 INFO:teuthology.orchestra.run.smithi129.stdout:5846 read 0x221b42d thru 0x221d59f (0x2173 bytes) 2024-09-17T21:30:56.201 INFO:teuthology.orchestra.run.smithi129.stdout:5847 punch from 0xc6b6e89 to 0xc6c307a, (0xc1f1 bytes) 2024-09-17T21:30:56.201 INFO:teuthology.orchestra.run.smithi129.stdout:5848 read 0x8eefb52 thru 0x8efb639 (0xbae8 bytes) 2024-09-17T21:30:56.201 INFO:teuthology.orchestra.run.smithi129.stdout:5849 write 0x25c6dd2 thru 0x25ce5d5 (0x7804 bytes) 2024-09-17T21:30:56.203 INFO:teuthology.orchestra.run.smithi129.stdout:5850 write 0xd119c71 thru 0xd123a25 (0x9db5 bytes) 2024-09-17T21:30:56.205 INFO:teuthology.orchestra.run.smithi129.stdout:5851 write 0x2be9316 thru 0x2bf5957 (0xc642 bytes) 2024-09-17T21:30:56.210 INFO:teuthology.orchestra.run.smithi129.stdout:5852 writesame 0xca5db72 thru 0xca608b6 data_size 0x1(0x2d45 bytes) 2024-09-17T21:30:56.216 INFO:teuthology.orchestra.run.smithi129.stdout:5853 punch from 0x692f685 to 0x693a765, (0xb0e0 bytes) 2024-09-17T21:30:56.217 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:56.217 INFO:teuthology.orchestra.run.smithi129.stdout:5855 read 0x712699f thru 0x712b60c (0x4c6e bytes) 2024-09-17T21:30:56.217 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:56.223 INFO:teuthology.orchestra.run.smithi129.stdout:5858 writesame 0xdee393a thru 0xdee5fc4 data_size 0x21(0x268b bytes) 2024-09-17T21:30:56.223 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:56.221+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:56.224 INFO:teuthology.orchestra.run.smithi129.stdout:5859 read 0xbe34a44 thru 0xbe3ddf7 (0x93b4 bytes) 2024-09-17T21:30:56.224 INFO:teuthology.orchestra.run.smithi129.stdout:5860 read 0x5576d3b thru 0x5576d8d (0x53 bytes) 2024-09-17T21:30:56.225 INFO:teuthology.orchestra.run.smithi129.stdout:5862 read 0xbca896 thru 0xbd01a1 (0x590c bytes) 2024-09-17T21:30:56.239 INFO:teuthology.orchestra.run.smithi129.stdout:5863 trunc from 0xdee5fc5 to 0x241b271 2024-09-17T21:30:56.251 INFO:teuthology.orchestra.run.smithi129.stdout:5865 read 0x223443f thru 0x2237e83 (0x3a45 bytes) 2024-09-17T21:30:56.269 INFO:teuthology.orchestra.run.smithi129.stdout:5866 write 0xbc674d6 thru 0xbc6ee3b (0x7966 bytes) 2024-09-17T21:30:56.273 INFO:teuthology.orchestra.run.smithi129.stdout:5867 trunc from 0xbc6ee3c to 0xe20ad56 2024-09-17T21:30:56.278 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:56.278 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:56.278 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:56.278 INFO:teuthology.orchestra.run.smithi129.stdout:5870 write 0x70af6a thru 0x70d2b1 (0x2348 bytes) 2024-09-17T21:30:56.281 INFO:teuthology.orchestra.run.smithi129.stdout:5871 punch from 0x2799a95 to 0x27a526d, (0xb7d8 bytes) 2024-09-17T21:30:56.281 INFO:teuthology.orchestra.run.smithi129.stdout:5872 read 0xbed4fdb thru 0xbee1812 (0xc838 bytes) 2024-09-17T21:30:56.281 INFO:teuthology.orchestra.run.smithi129.stdout:5873 writesame 0x3525fbb thru 0x352df97 data_size 0x3(0x7fdd bytes) 2024-09-17T21:30:56.288 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:30:56.288 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:56.289 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:56.289 INFO:teuthology.orchestra.run.smithi129.stdout:5877 write 0x6f00ed0 thru 0x6f0e086 (0xd1b7 bytes) 2024-09-17T21:30:56.289 INFO:teuthology.orchestra.run.smithi129.stdout:5878 clone 45 order 25 su 2097152 sc 15 2024-09-17T21:30:56.334 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:56.331+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:56.392 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:56.392 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:56.401 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:30:56.441 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:56.439+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:56.504 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:56.504 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:56.513 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:30:56.526 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:56.524+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:56.616 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:56.616 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:56.624 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:30:56.627 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:56.624+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:56.727 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:56.727 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:56.728 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:56.725+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:56.735 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:30:56.829 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:56.827+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:56.838 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:56.838 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:56.848 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:30:56.911 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-17T21:30:56.911 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:30:56.911 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-17T21:30:56.912 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T21:30:56.912 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T21:30:56.912 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-17T21:30:56.929 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:56.927+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:56.954 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:56.955 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:56.964 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:30:56.986 INFO:teuthology.orchestra.run.smithi129.stdout:truncating image image_client.0-clone44 from 0xe20ad56 (overlap 0x241b271) to 0x15a9e43 2024-09-17T21:30:57.030 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:57.031 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T21:30:57.031 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:57.031 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:57.028+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:57.046 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-17T21:30:57.110 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:57.110 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:57.119 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:30:57.132 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:57.129+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:57.223 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:57.223 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:57.232 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:30:57.233 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:57.231+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:57.282 INFO:teuthology.orchestra.run.smithi129.stdout:5880 trunc from 0xe20ad56 to 0xa535740 2024-09-17T21:30:57.331 INFO:teuthology.orchestra.run.smithi129.stdout:5882 write 0xbf60b7c thru 0xbf689ee (0x7e73 bytes) 2024-09-17T21:30:57.332 INFO:teuthology.orchestra.run.smithi129.stdout:5883 read 0x2c9220f thru 0x2c9ff36 (0xdd28 bytes) 2024-09-17T21:30:57.332 INFO:teuthology.orchestra.run.smithi129.stdout:5884 writesame 0xbb69840 thru 0xbb7723a data_size 0x3(0xd9fb bytes) 2024-09-17T21:30:57.334 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:57.332+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:57.338 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:57.339 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:57.348 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:30:57.348 INFO:teuthology.orchestra.run.smithi129.stdout:5885 writesame 0xe1e361c thru 0xe1ec56d data_size 0xf(0x8f52 bytes) 2024-09-17T21:30:57.350 INFO:teuthology.orchestra.run.smithi129.stdout:5886 punch from 0x63f41cc to 0x64040e2, (0xff16 bytes) 2024-09-17T21:30:57.350 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:57.350 INFO:teuthology.orchestra.run.smithi129.stdout:5888 read 0xbe9b29 thru 0xbebff2 (0x24ca bytes) 2024-09-17T21:30:57.350 INFO:teuthology.orchestra.run.smithi129.stdout:5889 writesame 0x843ceaf thru 0x843e2ff data_size 0x1(0x1451 bytes) 2024-09-17T21:30:57.353 INFO:teuthology.orchestra.run.smithi129.stdout:5890 read 0x6cae308 thru 0x6cbced8 (0xebd1 bytes) 2024-09-17T21:30:57.353 INFO:teuthology.orchestra.run.smithi129.stdout:5892 write 0x8bca9ef thru 0x8bd1d69 (0x737b bytes) 2024-09-17T21:30:57.355 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:57.355 INFO:teuthology.orchestra.run.smithi129.stdout:5894 punch from 0x7b38483 to 0x7b460af, (0xdc2c bytes) 2024-09-17T21:30:57.355 INFO:teuthology.orchestra.run.smithi129.stdout:5895 read 0xb2d1ba4 thru 0xb2dacfd (0x915a bytes) 2024-09-17T21:30:57.356 INFO:teuthology.orchestra.run.smithi129.stdout:5897 read 0xb2069eb thru 0xb20bc80 (0x5296 bytes) 2024-09-17T21:30:57.356 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:57.356 INFO:teuthology.orchestra.run.smithi129.stdout:5901 punch from 0xe17a007 to 0xe17e63a, (0x4633 bytes) 2024-09-17T21:30:57.356 INFO:teuthology.orchestra.run.smithi129.stdout:5902 write 0x1c8ad6b thru 0x1c9a191 (0xf427 bytes) 2024-09-17T21:30:57.358 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:57.358 INFO:teuthology.orchestra.run.smithi129.stdout:5905 read 0x76acb7e thru 0x76aef92 (0x2415 bytes) 2024-09-17T21:30:57.358 INFO:teuthology.orchestra.run.smithi129.stdout:5906 read 0x5edb524 thru 0x5edd066 (0x1b43 bytes) 2024-09-17T21:30:57.359 INFO:teuthology.orchestra.run.smithi129.stdout:5907 read 0x96b14f5 thru 0x96bb109 (0x9c15 bytes) 2024-09-17T21:30:57.359 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:57.359 INFO:teuthology.orchestra.run.smithi129.stdout:5911 read 0x26f30c1 thru 0x26f6abb (0x39fb bytes) 2024-09-17T21:30:57.359 INFO:teuthology.orchestra.run.smithi129.stdout:5915 read 0x8a9b255 thru 0x8aa7f40 (0xccec bytes) 2024-09-17T21:30:57.360 INFO:teuthology.orchestra.run.smithi129.stdout:5917 read 0x45d368b thru 0x45d7a37 (0x43ad bytes) 2024-09-17T21:30:57.360 INFO:teuthology.orchestra.run.smithi129.stdout:5919 write 0x28ad0ac thru 0x28ad9f5 (0x94a bytes) 2024-09-17T21:30:57.361 INFO:teuthology.orchestra.run.smithi129.stdout:5920 write 0xb6e74ab thru 0xb6ed70a (0x6260 bytes) 2024-09-17T21:30:57.362 INFO:teuthology.orchestra.run.smithi129.stdout:5921 writesame 0x9fe604b thru 0x9ff5795 data_size 0x25(0xf74b bytes) 2024-09-17T21:30:57.364 INFO:teuthology.orchestra.run.smithi129.stdout:5922 write 0x53bd498 thru 0x53cc89c (0xf405 bytes) 2024-09-17T21:30:57.366 INFO:teuthology.orchestra.run.smithi129.stdout:5924 writesame 0xc466699 thru 0xc469953 data_size 0x27(0x32bb bytes) 2024-09-17T21:30:57.383 INFO:teuthology.orchestra.run.smithi129.stdout:5926 trunc from 0xe1ec56e to 0xa75654 2024-09-17T21:30:57.409 INFO:teuthology.orchestra.run.smithi129.stdout:5927 read 0x56b722 thru 0x56dc96 (0x2575 bytes) 2024-09-17T21:30:57.409 INFO:teuthology.orchestra.run.smithi129.stdout:5928 read 0x9dd45f thru 0x9ec3b7 (0xef59 bytes) 2024-09-17T21:30:57.414 INFO:teuthology.orchestra.run.smithi129.stdout:5929 writesame 0xb15076 thru 0xb1e35e data_size 0xd(0x92e9 bytes) 2024-09-17T21:30:57.428 INFO:teuthology.orchestra.run.smithi129.stdout:5932 write 0x6118394 thru 0x61206b9 (0x8326 bytes) 2024-09-17T21:30:57.430 INFO:teuthology.orchestra.run.smithi129.stdout:5933 punch from 0x4644d54 to 0x464a0d2, (0x537e bytes) 2024-09-17T21:30:57.435 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:57.433+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:57.438 INFO:teuthology.orchestra.run.smithi129.stdout:5934 write 0x8b83fa3 thru 0x8b8ad2c (0x6d8a bytes) 2024-09-17T21:30:57.441 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:57.441 INFO:teuthology.orchestra.run.smithi129.stdout:5937 read 0x1f6d4a1 thru 0x1f70811 (0x3371 bytes) 2024-09-17T21:30:57.441 INFO:teuthology.orchestra.run.smithi129.stdout:5938 punch from 0x409aace to 0x40a505a, (0xa58c bytes) 2024-09-17T21:30:57.442 INFO:teuthology.orchestra.run.smithi129.stdout:5942 writesame 0x10d26bf thru 0x10d5a9b data_size 0x11(0x33dd bytes) 2024-09-17T21:30:57.444 INFO:teuthology.orchestra.run.smithi129.stdout:5943 punch from 0x2060e3 to 0x20a8da, (0x47f7 bytes) 2024-09-17T21:30:57.444 INFO:teuthology.orchestra.run.smithi129.stdout:5944 read 0xbdaddb thru 0xbe234b (0x7571 bytes) 2024-09-17T21:30:57.454 INFO:teuthology.orchestra.run.smithi129.stdout:5946 write 0xc7d9ed2 thru 0xc7dc2c0 (0x23ef bytes) 2024-09-17T21:30:57.456 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:57.456 INFO:teuthology.orchestra.run.smithi129.stdout:5948 writesame 0x33753f7 thru 0x337ba49 data_size 0xd(0x6653 bytes) 2024-09-17T21:30:57.460 INFO:teuthology.orchestra.run.smithi129.stdout:5949 writesame 0x5f1922c thru 0x5f1d487 data_size 0x3e(0x425c bytes) 2024-09-17T21:30:57.461 INFO:teuthology.orchestra.run.smithi129.stdout:5950 write 0x9b04c50 thru 0x9b0e521 (0x98d2 bytes) 2024-09-17T21:30:57.463 INFO:teuthology.orchestra.run.smithi129.stdout:5952 read 0x1297e99 thru 0x12a4d5e (0xcec6 bytes) 2024-09-17T21:30:57.463 INFO:teuthology.orchestra.run.smithi129.stdout:5953 read 0x16fd37f thru 0x1708147 (0xadc9 bytes) 2024-09-17T21:30:57.463 INFO:teuthology.orchestra.run.smithi129.stdout:5954 write 0x4aa5f8c thru 0x4aaae7a (0x4eef bytes) 2024-09-17T21:30:57.466 INFO:teuthology.orchestra.run.smithi129.stdout:5955 read 0xc2dcfd0 thru 0xc2e38b9 (0x68ea bytes) 2024-09-17T21:30:57.466 INFO:teuthology.orchestra.run.smithi129.stdout:5956 read 0xc019d97 thru 0xc01f7b3 (0x5a1d bytes) 2024-09-17T21:30:57.466 INFO:teuthology.orchestra.run.smithi129.stdout:5958 write 0x8e210a4 thru 0x8e28629 (0x7586 bytes) 2024-09-17T21:30:57.469 INFO:teuthology.orchestra.run.smithi129.stdout:5959 read 0x8081898 thru 0x8081c3e (0x3a7 bytes) 2024-09-17T21:30:57.469 INFO:teuthology.orchestra.run.smithi129.stdout:5960 read 0x9452e21 thru 0x9462dce (0xffae bytes) 2024-09-17T21:30:57.469 INFO:teuthology.orchestra.run.smithi129.stdout:compare and write disabled 2024-09-17T21:30:57.470 INFO:teuthology.orchestra.run.smithi129.stdout:5962 writesame 0x4122019 thru 0x4126b2a data_size 0x3(0x4b12 bytes) 2024-09-17T21:30:57.474 INFO:teuthology.orchestra.run.smithi129.stdout:5963 read 0x23e0d2f thru 0x23e195a (0xc2c bytes) 2024-09-17T21:30:57.474 INFO:teuthology.orchestra.run.smithi129.stdout:5964 write 0x8ae8db0 thru 0x8af5d82 (0xcfd3 bytes) 2024-09-17T21:30:57.476 INFO:teuthology.orchestra.run.smithi129.stdout:5965 read 0x350c5c5 thru 0x35112fe (0x4d3a bytes) 2024-09-17T21:30:57.476 INFO:teuthology.orchestra.run.smithi129.stdout:5968 writesame 0x44fc46d thru 0x4509e34 data_size 0x65(0xd9c8 bytes) 2024-09-17T21:30:57.478 INFO:teuthology.orchestra.run.smithi129.stdout:5969 write 0x93c3013 thru 0x93c6cba (0x3ca8 bytes) 2024-09-17T21:30:57.487 INFO:teuthology.orchestra.run.smithi129.stdout:5971 write 0xe29db06 thru 0xe2a034a (0x2845 bytes) 2024-09-17T21:30:57.489 INFO:teuthology.orchestra.run.smithi129.stdout:5972 writesame 0xda567dc thru 0xda59229 data_size 0x39(0x2a4e bytes) 2024-09-17T21:30:57.491 INFO:teuthology.orchestra.run.smithi129.stdout:5975 write 0x430ec2e thru 0x4312343 (0x3716 bytes) 2024-09-17T21:30:57.496 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:57.496 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:57.505 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:30:57.508 INFO:teuthology.orchestra.run.smithi129.stdout:5977 trunc from 0xe2a034b to 0x13d853c 2024-09-17T21:30:57.540 INFO:teuthology.orchestra.run.smithi129.stdout:5978 writesame 0x63c95a5 thru 0x63d83f2 data_size 0x5e(0xee4e bytes) 2024-09-17T21:30:57.542 INFO:teuthology.orchestra.run.smithi129.stdout:5979 punch from 0x44e7859 to 0x44eccda, (0x5481 bytes) 2024-09-17T21:30:57.542 INFO:teuthology.orchestra.run.smithi129.stdout:5980 read 0x5d78814 thru 0x5d797fc (0xfe9 bytes) 2024-09-17T21:30:57.542 INFO:teuthology.orchestra.run.smithi129.stdout:5982 write 0xce4529 thru 0xce8bd2 (0x46aa bytes) 2024-09-17T21:30:57.546 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:57.544+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:57.547 INFO:teuthology.orchestra.run.smithi129.stdout:5983 punch from 0x3776f34 to 0x37866bf, (0xf78b bytes) 2024-09-17T21:30:57.556 INFO:teuthology.orchestra.run.smithi129.stdout:5984 trunc from 0x63d83f3 to 0xdb6b75a 2024-09-17T21:30:57.561 INFO:teuthology.orchestra.run.smithi129.stdout:5986 read 0x1030127 thru 0x10383b7 (0x8291 bytes) 2024-09-17T21:30:57.562 INFO:teuthology.orchestra.run.smithi129.stdout:5988 read 0x556579b thru 0x5568e69 (0x36cf bytes) 2024-09-17T21:30:57.562 INFO:teuthology.orchestra.run.smithi129.stdout:5989 writesame 0x7d44785 thru 0x7d4f37e data_size 0x1(0xabfa bytes) 2024-09-17T21:30:57.582 INFO:teuthology.orchestra.run.smithi129.stdout:5990 write 0xd2cf0e8 thru 0xd2dc72c (0xd645 bytes) 2024-09-17T21:30:57.584 INFO:teuthology.orchestra.run.smithi129.stdout:5992 write 0xe1d14 thru 0xe54e6 (0x37d3 bytes) 2024-09-17T21:30:57.585 INFO:teuthology.orchestra.run.smithi129.stdout:5995 read 0x9f63979 thru 0x9f6af37 (0x75bf bytes) 2024-09-17T21:30:57.585 INFO:teuthology.orchestra.run.smithi129.stdout:5996 writesame 0x20269ec thru 0x202c597 data_size 0x2(0x5bac bytes) 2024-09-17T21:30:57.595 INFO:teuthology.orchestra.run.smithi129.stdout:5997 trunc from 0xdb6b75a to 0xa426ae0 2024-09-17T21:30:57.602 INFO:teuthology.orchestra.run.smithi129.stdout:5998 writesame 0x54b0704 thru 0x54bb097 data_size 0x2(0xa994 bytes) 2024-09-17T21:30:57.613 INFO:teuthology.orchestra.run.smithi129.stdout:6000 read 0x53a9513 thru 0x53aeaac (0x559a bytes) 2024-09-17T21:30:57.632 INFO:teuthology.orchestra.run.smithi129.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-17T21:30:57.637 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:57.634+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:57.654 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:57.654 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:57.659 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:30:57.664 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:30:57.738 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:57.736+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:57.771 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:30:57.771 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:30:57.780 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:30:57.795 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:30:57.796 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:30:57.812 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:30:57.812 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:30:57.828 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:30:57.828 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:30:57.839 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:57.836+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:57.844 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:30:57.845 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:30:57.861 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:30:57.861 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:30:57.878 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:30:57.878 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:30:57.894 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:30:57.894 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:30:57.911 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:30:57.911 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:30:57.929 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:30:57.930 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:30:57.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:30:57.930 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:30:57.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.930 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:30:57.930 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:57.930 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 89.986711787999994, 2024-09-17T21:30:57.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61572000699999996, 2024-09-17T21:30:57.930 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:57.930 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:57.930 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:57.930 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:57.930 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:30:57.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:30:57.931 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T21:30:57.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.931 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:57.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:57.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:57.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:57.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:30:57.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:57.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939745+0000", 2024-09-17T21:30:57.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6899999999999999e-06 2024-09-17T21:30:57.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:57.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939753+0000", 2024-09-17T21:30:57.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1950000000000003e-06 2024-09-17T21:30:57.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:57.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939767+0000", 2024-09-17T21:30:57.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3429e-05 2024-09-17T21:30:57.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:57.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939776+0000", 2024-09-17T21:30:57.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:30:57.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:57.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939858+0000", 2024-09-17T21:30:57.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2496000000000004e-05 2024-09-17T21:30:57.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T21:30:57.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939868+0000", 2024-09-17T21:30:57.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1910000000000006e-06 2024-09-17T21:30:57.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:57.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940612+0000", 2024-09-17T21:30:57.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074429599999999998 2024-09-17T21:30:57.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:30:57.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940621+0000", 2024-09-17T21:30:57.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9139999999999997e-06 2024-09-17T21:30:57.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:57.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554006+0000", 2024-09-17T21:30:57.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61338505099999996 2024-09-17T21:30:57.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:57.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554166+0000", 2024-09-17T21:30:57.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015982600000000001 2024-09-17T21:30:57.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:30:57.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554296+0000", 2024-09-17T21:30:57.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013002399999999999 2024-09-17T21:30:57.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:57.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554936+0000", 2024-09-17T21:30:57.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064009400000000001 2024-09-17T21:30:57.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:57.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555389+0000", 2024-09-17T21:30:57.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045355200000000002 2024-09-17T21:30:57.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:57.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555432+0000", 2024-09-17T21:30:57.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2477e-05 2024-09-17T21:30:57.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:57.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555463+0000", 2024-09-17T21:30:57.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1612000000000002e-05 2024-09-17T21:30:57.937 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.937 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:57.937 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.937 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17527 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1005568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:57.937 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:57.937 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.521483957000001, 2024-09-17T21:30:57.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046452090000000001, 2024-09-17T21:30:57.938 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:57.938 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:57.938 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:57.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:57.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:57.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:57.938 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17527 2024-09-17T21:30:57.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.938 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:57.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:57.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:57.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:57.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:30:57.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:57.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405496+0000", 2024-09-17T21:30:57.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052498400000000004 2024-09-17T21:30:57.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:57.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405500+0000", 2024-09-17T21:30:57.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8410000000000002e-06 2024-09-17T21:30:57.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:57.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405511+0000", 2024-09-17T21:30:57.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0749e-05 2024-09-17T21:30:57.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:57.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405517+0000", 2024-09-17T21:30:57.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:30:57.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:57.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451275+0000", 2024-09-17T21:30:57.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045758134999999998 2024-09-17T21:30:57.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:57.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451355+0000", 2024-09-17T21:30:57.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9453e-05 2024-09-17T21:30:57.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:57.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451423+0000", 2024-09-17T21:30:57.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8782999999999995e-05 2024-09-17T21:30:57.943 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.943 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:57.943 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.944 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17615 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:57.944 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:57.944 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.516243469000003, 2024-09-17T21:30:57.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046741309000000002, 2024-09-17T21:30:57.944 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:57.944 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:57.944 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:57.944 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:57.944 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:57.944 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:57.944 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17615 2024-09-17T21:30:57.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.944 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:57.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:57.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:57.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:57.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:57.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:57.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:30:57.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:30:57.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:57.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410213+0000", 2024-09-17T21:30:57.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:30:57.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:57.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410217+0000", 2024-09-17T21:30:57.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8199999999999998e-06 2024-09-17T21:30:57.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:57.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410220+0000", 2024-09-17T21:30:57.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6900000000000001e-06 2024-09-17T21:30:57.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:57.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455264+0000", 2024-09-17T21:30:57.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045044002999999999 2024-09-17T21:30:57.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:57.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455338+0000", 2024-09-17T21:30:57.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4182000000000002e-05 2024-09-17T21:30:57.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:57.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.456953+0000", 2024-09-17T21:30:57.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016150089999999999 2024-09-17T21:30:57.947 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.948 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:57.948 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.948 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17616 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 872448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:57.948 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:57.948 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.516231417, 2024-09-17T21:30:57.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946039000000002, 2024-09-17T21:30:57.948 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:57.948 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:57.948 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:57.948 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:57.948 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:57.948 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:57.949 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17616 2024-09-17T21:30:57.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.949 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:57.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:57.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:57.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:57.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:57.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:57.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:30:57.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T21:30:57.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:57.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410225+0000", 2024-09-17T21:30:57.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3459999999999999e-06 2024-09-17T21:30:57.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:57.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410228+0000", 2024-09-17T21:30:57.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.083e-06 2024-09-17T21:30:57.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:57.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410232+0000", 2024-09-17T21:30:57.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4419999999999998e-06 2024-09-17T21:30:57.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:57.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457004+0000", 2024-09-17T21:30:57.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046771873999999998 2024-09-17T21:30:57.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:57.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457084+0000", 2024-09-17T21:30:57.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0506000000000005e-05 2024-09-17T21:30:57.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:57.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457170+0000", 2024-09-17T21:30:57.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5494000000000005e-05 2024-09-17T21:30:57.952 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.952 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:57.952 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.952 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17617 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:57.952 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:57.952 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.516220015000002, 2024-09-17T21:30:57.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047084168000000003, 2024-09-17T21:30:57.952 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:57.952 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:57.953 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:57.953 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:57.953 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:57.953 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:57.953 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17617 2024-09-17T21:30:57.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.953 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:57.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:57.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:57.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:57.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:57.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:57.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:30:57.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-17T21:30:57.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:57.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410237+0000", 2024-09-17T21:30:57.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1960000000000001e-06 2024-09-17T21:30:57.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:57.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410246+0000", 2024-09-17T21:30:57.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7880000000000001e-06 2024-09-17T21:30:57.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:57.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410250+0000", 2024-09-17T21:30:57.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.518e-06 2024-09-17T21:30:57.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:57.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457209+0000", 2024-09-17T21:30:57.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046959283999999997 2024-09-17T21:30:57.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:57.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457264+0000", 2024-09-17T21:30:57.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673999999999999e-05 2024-09-17T21:30:57.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:57.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457319+0000", 2024-09-17T21:30:57.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5497000000000001e-05 2024-09-17T21:30:57.956 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.956 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:57.956 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.956 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17618 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 899072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:57.956 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:57.956 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.516200570999999, 2024-09-17T21:30:57.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047204208999999997, 2024-09-17T21:30:57.957 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:57.957 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:57.957 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:57.957 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:57.957 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:57.957 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:57.957 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17618 2024-09-17T21:30:57.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.957 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:57.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:57.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:57.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:57.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:57.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:57.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:30:57.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8799999999999998e-07 2024-09-17T21:30:57.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:57.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410256+0000", 2024-09-17T21:30:57.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T21:30:57.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:57.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410260+0000", 2024-09-17T21:30:57.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6500000000000002e-06 2024-09-17T21:30:57.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:57.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410263+0000", 2024-09-17T21:30:57.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7089999999999998e-06 2024-09-17T21:30:57.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:57.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457355+0000", 2024-09-17T21:30:57.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047092756999999999 2024-09-17T21:30:57.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:57.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457407+0000", 2024-09-17T21:30:57.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1808999999999998e-05 2024-09-17T21:30:57.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:57.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457459+0000", 2024-09-17T21:30:57.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1684000000000002e-05 2024-09-17T21:30:57.960 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.960 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:57.960 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.961 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17619 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 913408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:57.961 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:57.961 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.516189333, 2024-09-17T21:30:57.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047293443999999997, 2024-09-17T21:30:57.961 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:57.961 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:57.961 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:57.961 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:57.961 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:57.961 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:57.961 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17619 2024-09-17T21:30:57.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.961 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:57.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:57.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:57.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:57.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:57.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:57.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:30:57.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:30:57.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:57.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410267+0000", 2024-09-17T21:30:57.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:30:57.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:57.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410270+0000", 2024-09-17T21:30:57.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.01e-06 2024-09-17T21:30:57.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:57.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410273+0000", 2024-09-17T21:30:57.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6410000000000002e-06 2024-09-17T21:30:57.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:57.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457473+0000", 2024-09-17T21:30:57.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047199829999999998 2024-09-17T21:30:57.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:57.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457517+0000", 2024-09-17T21:30:57.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4058999999999999e-05 2024-09-17T21:30:57.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:57.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457559+0000", 2024-09-17T21:30:57.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2404999999999998e-05 2024-09-17T21:30:57.965 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.965 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:57.965 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.965 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17620 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 927744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:57.965 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:57.965 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.516178160999999, 2024-09-17T21:30:57.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047402409, 2024-09-17T21:30:57.966 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:57.966 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:57.966 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:57.966 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:57.966 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:57.966 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:57.966 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17620 2024-09-17T21:30:57.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.966 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:57.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:57.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:57.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:57.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:57.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:57.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:30:57.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:30:57.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:57.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410278+0000", 2024-09-17T21:30:57.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1790000000000001e-06 2024-09-17T21:30:57.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:57.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410281+0000", 2024-09-17T21:30:57.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.061e-06 2024-09-17T21:30:57.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:57.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410284+0000", 2024-09-17T21:30:57.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.65e-06 2024-09-17T21:30:57.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:57.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457587+0000", 2024-09-17T21:30:57.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047303206 2024-09-17T21:30:57.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:57.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457633+0000", 2024-09-17T21:30:57.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6063999999999998e-05 2024-09-17T21:30:57.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:57.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457679+0000", 2024-09-17T21:30:57.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6007000000000002e-05 2024-09-17T21:30:57.969 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.969 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:57.969 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.970 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17621 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 940032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:57.970 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:57.970 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.516167745000004, 2024-09-17T21:30:57.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047486550000000002, 2024-09-17T21:30:57.970 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:57.970 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:57.970 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:57.970 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:57.970 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:57.970 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:57.970 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17621 2024-09-17T21:30:57.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.970 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:57.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:57.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:57.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:57.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:30:57.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:57.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410288+0000", 2024-09-17T21:30:57.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:30:57.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:57.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410289+0000", 2024-09-17T21:30:57.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1179999999999999e-06 2024-09-17T21:30:57.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:57.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410292+0000", 2024-09-17T21:30:57.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:30:57.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:57.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410294+0000", 2024-09-17T21:30:57.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T21:30:57.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:57.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457694+0000", 2024-09-17T21:30:57.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047399416999999999 2024-09-17T21:30:57.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:57.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457734+0000", 2024-09-17T21:30:57.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0528999999999998e-05 2024-09-17T21:30:57.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:57.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457774+0000", 2024-09-17T21:30:57.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9529e-05 2024-09-17T21:30:57.974 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.974 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:57.974 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.974 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17622 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:57.974 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:57.974 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.516157562999993, 2024-09-17T21:30:57.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047596619, 2024-09-17T21:30:57.974 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:57.974 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:57.974 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:57.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:57.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:57.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:57.975 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17622 2024-09-17T21:30:57.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.975 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:57.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:57.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:57.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:57.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:57.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:57.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:30:57.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:57.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:57.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410303+0000", 2024-09-17T21:30:57.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2979999999999999e-06 2024-09-17T21:30:57.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:57.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410307+0000", 2024-09-17T21:30:57.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:30:57.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:57.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410310+0000", 2024-09-17T21:30:57.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.146e-06 2024-09-17T21:30:57.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:57.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457801+0000", 2024-09-17T21:30:57.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047491322000000002 2024-09-17T21:30:57.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:57.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457850+0000", 2024-09-17T21:30:57.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8585000000000003e-05 2024-09-17T21:30:57.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:57.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457894+0000", 2024-09-17T21:30:57.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4455e-05 2024-09-17T21:30:57.978 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.978 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:57.978 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.978 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17623 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 968704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:57.978 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:57.978 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.516141179000002, 2024-09-17T21:30:57.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047689377999999998, 2024-09-17T21:30:57.978 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:57.979 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:57.979 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:57.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:57.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:57.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:57.979 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17623 2024-09-17T21:30:57.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.979 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:57.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:57.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:57.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:57.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:57.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:57.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:30:57.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:30:57.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:57.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410315+0000", 2024-09-17T21:30:57.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0499999999999999e-06 2024-09-17T21:30:57.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:57.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410319+0000", 2024-09-17T21:30:57.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.517e-06 2024-09-17T21:30:57.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:57.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410321+0000", 2024-09-17T21:30:57.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:30:57.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:57.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457922+0000", 2024-09-17T21:30:57.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047600638000000001 2024-09-17T21:30:57.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:57.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457964+0000", 2024-09-17T21:30:57.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2126000000000003e-05 2024-09-17T21:30:57.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:57.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458003+0000", 2024-09-17T21:30:57.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9138999999999999e-05 2024-09-17T21:30:57.982 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.982 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:57.982 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.983 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17624 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:57.983 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:57.983 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.516130595000007, 2024-09-17T21:30:57.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047798292999999999, 2024-09-17T21:30:57.983 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:57.983 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:57.983 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:57.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:57.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:57.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:57.983 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17624 2024-09-17T21:30:57.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.984 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:57.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:57.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:57.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:57.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:57.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:57.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:30:57.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:30:57.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:57.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410326+0000", 2024-09-17T21:30:57.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0610000000000001e-06 2024-09-17T21:30:57.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:57.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410329+0000", 2024-09-17T21:30:57.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0319999999999999e-06 2024-09-17T21:30:57.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:57.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410331+0000", 2024-09-17T21:30:57.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5749999999999999e-06 2024-09-17T21:30:57.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:57.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458033+0000", 2024-09-17T21:30:57.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047701371999999999 2024-09-17T21:30:57.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:57.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458079+0000", 2024-09-17T21:30:57.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:30:57.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:57.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458123+0000", 2024-09-17T21:30:57.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850999999999997e-05 2024-09-17T21:30:57.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.987 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:57.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.987 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17625 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:57.987 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:57.988 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.516120435000005, 2024-09-17T21:30:57.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047883051000000003, 2024-09-17T21:30:57.988 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:57.988 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:57.988 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:57.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:57.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:57.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:57.988 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17625 2024-09-17T21:30:57.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.988 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:57.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:57.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:57.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:57.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:57.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:57.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:30:57.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:30:57.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:57.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410336+0000", 2024-09-17T21:30:57.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1310000000000001e-06 2024-09-17T21:30:57.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:57.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410339+0000", 2024-09-17T21:30:57.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000001e-06 2024-09-17T21:30:57.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:57.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410342+0000", 2024-09-17T21:30:57.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7460000000000001e-06 2024-09-17T21:30:57.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:57.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458137+0000", 2024-09-17T21:30:57.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047794931999999998 2024-09-17T21:30:57.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:57.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458178+0000", 2024-09-17T21:30:57.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1013999999999997e-05 2024-09-17T21:30:57.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:57.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458218+0000", 2024-09-17T21:30:57.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9675999999999998e-05 2024-09-17T21:30:57.991 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.991 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:57.992 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.992 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17626 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1009664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:57.992 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:57.992 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.516109814000004, 2024-09-17T21:30:57.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047991601000000002, 2024-09-17T21:30:57.992 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:57.992 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:57.992 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:57.992 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:57.992 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:57.992 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:57.993 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17626 2024-09-17T21:30:57.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.993 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:57.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:57.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:57.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:57.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:30:57.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:57.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410346+0000", 2024-09-17T21:30:57.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:30:57.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:57.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410347+0000", 2024-09-17T21:30:57.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.114e-06 2024-09-17T21:30:57.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:57.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410350+0000", 2024-09-17T21:30:57.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.101e-06 2024-09-17T21:30:57.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:57.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410353+0000", 2024-09-17T21:30:57.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T21:30:57.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:57.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458245+0000", 2024-09-17T21:30:57.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0478921 2024-09-17T21:30:57.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:57.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458292+0000", 2024-09-17T21:30:57.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397999999999999e-05 2024-09-17T21:30:57.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:57.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458337+0000", 2024-09-17T21:30:57.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5221e-05 2024-09-17T21:30:57.996 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.996 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:57.996 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:57.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.997 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17627 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:57.997 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:57.997 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.516072713, 2024-09-17T21:30:57.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048047383999999999, 2024-09-17T21:30:57.997 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:57.997 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:57.997 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:57.997 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:57.997 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:57.997 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:57.997 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17627 2024-09-17T21:30:57.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.997 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:57.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:57.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:57.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:57.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:30:57.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:57.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:57.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410383+0000", 2024-09-17T21:30:57.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:30:57.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:57.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410385+0000", 2024-09-17T21:30:57.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:30:57.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:57.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410391+0000", 2024-09-17T21:30:57.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:30:57.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:57.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:57.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:57.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410395+0000", 2024-09-17T21:30:57.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6539999999999998e-06 2024-09-17T21:30:57.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:58.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458350+0000", 2024-09-17T21:30:58.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047954855999999997 2024-09-17T21:30:58.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:58.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458392+0000", 2024-09-17T21:30:58.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2094000000000001e-05 2024-09-17T21:30:58.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:58.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458430+0000", 2024-09-17T21:30:58.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7474000000000001e-05 2024-09-17T21:30:58.001 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:58.001 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:58.001 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:58.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.001 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17628 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:58.001 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:58.001 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.516053151999998, 2024-09-17T21:30:58.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048142676000000002, 2024-09-17T21:30:58.001 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:58.001 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:58.001 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:58.001 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:58.001 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:58.002 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.002 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17628 2024-09-17T21:30:58.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.002 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:58.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:58.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:58.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:58.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:58.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:58.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:58.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:58.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:30:58.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:30:58.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:58.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410404+0000", 2024-09-17T21:30:58.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-06 2024-09-17T21:30:58.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:58.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410411+0000", 2024-09-17T21:30:58.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6850000000000004e-06 2024-09-17T21:30:58.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410415+0000", 2024-09-17T21:30:58.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6750000000000001e-06 2024-09-17T21:30:58.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:58.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458457+0000", 2024-09-17T21:30:58.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048042045999999998 2024-09-17T21:30:58.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:58.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458501+0000", 2024-09-17T21:30:58.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3661000000000001e-05 2024-09-17T21:30:58.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:58.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458545+0000", 2024-09-17T21:30:58.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3529000000000003e-05 2024-09-17T21:30:58.005 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:58.005 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:58.005 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:58.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.005 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17687 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:58.005 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:58.005 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.513608199000004, 2024-09-17T21:30:58.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050670448, 2024-09-17T21:30:58.006 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:58.006 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:58.006 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:58.006 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:58.006 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:58.006 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.006 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17687 2024-09-17T21:30:58.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.006 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:58.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:58.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:58.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:58.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:58.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:30:58.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:58.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:58.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412848+0000", 2024-09-17T21:30:58.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9100000000000003e-07 2024-09-17T21:30:58.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:58.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413102+0000", 2024-09-17T21:30:58.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025415600000000001 2024-09-17T21:30:58.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:58.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413111+0000", 2024-09-17T21:30:58.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2010000000000005e-06 2024-09-17T21:30:58.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413116+0000", 2024-09-17T21:30:58.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:30:58.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:58.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458559+0000", 2024-09-17T21:30:58.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045442914000000001 2024-09-17T21:30:58.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:58.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458632+0000", 2024-09-17T21:30:58.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3207000000000006e-05 2024-09-17T21:30:58.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:58.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458785+0000", 2024-09-17T21:30:58.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015226700000000001 2024-09-17T21:30:58.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:58.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.461648+0000", 2024-09-17T21:30:58.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028633069999999998 2024-09-17T21:30:58.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:58.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463416+0000", 2024-09-17T21:30:58.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017680230000000001 2024-09-17T21:30:58.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:58.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463475+0000", 2024-09-17T21:30:58.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9333000000000003e-05 2024-09-17T21:30:58.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:58.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463517+0000", 2024-09-17T21:30:58.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2162000000000001e-05 2024-09-17T21:30:58.011 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:58.011 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:58.011 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:58.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.011 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17688 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [write 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:58.011 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:58.011 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.513329330999994, 2024-09-17T21:30:58.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050625098, 2024-09-17T21:30:58.011 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:58.011 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:58.011 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:58.012 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:58.012 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:58.012 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.012 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17688 2024-09-17T21:30:58.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.012 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:58.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:58.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:58.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:58.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:58.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:58.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:58.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:58.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:30:58.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:30:58.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:58.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413128+0000", 2024-09-17T21:30:58.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.813e-06 2024-09-17T21:30:58.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:58.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413132+0000", 2024-09-17T21:30:58.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:30:58.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413135+0000", 2024-09-17T21:30:58.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:30:58.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:58.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459756+0000", 2024-09-17T21:30:58.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046621293000000001 2024-09-17T21:30:58.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:58.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459816+0000", 2024-09-17T21:30:58.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9156000000000001e-05 2024-09-17T21:30:58.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:58.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459912+0000", 2024-09-17T21:30:58.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6302000000000001e-05 2024-09-17T21:30:58.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:58.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463231+0000", 2024-09-17T21:30:58.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033188290000000001 2024-09-17T21:30:58.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:58.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463701+0000", 2024-09-17T21:30:58.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047068199999999999 2024-09-17T21:30:58.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:58.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463728+0000", 2024-09-17T21:30:58.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6726000000000001e-05 2024-09-17T21:30:58.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:58.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463751+0000", 2024-09-17T21:30:58.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2824e-05 2024-09-17T21:30:58.016 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:58.016 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:58.017 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:58.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.017 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17691 3.d 3:b3a055f3:::rbd_data.11207744cc2c.0000000000000083:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:30:58.017 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:58.017 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.509784428000003, 2024-09-17T21:30:58.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049309370999999998, 2024-09-17T21:30:58.017 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:58.017 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:58.017 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:58.017 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:58.017 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:58.017 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.017 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17691 2024-09-17T21:30:58.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.018 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:58.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:58.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:58.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:58.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:58.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:30:58.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:58.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:58.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416672+0000", 2024-09-17T21:30:58.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.47e-07 2024-09-17T21:30:58.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:58.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416915+0000", 2024-09-17T21:30:58.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024344500000000001 2024-09-17T21:30:58.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:58.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416924+0000", 2024-09-17T21:30:58.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7739999999999993e-06 2024-09-17T21:30:58.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416929+0000", 2024-09-17T21:30:58.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6990000000000002e-06 2024-09-17T21:30:58.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:58.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460414+0000", 2024-09-17T21:30:58.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043485466 2024-09-17T21:30:58.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:58.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460540+0000", 2024-09-17T21:30:58.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012608899999999999 2024-09-17T21:30:58.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:30:58.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460746+0000", 2024-09-17T21:30:58.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000205577 2024-09-17T21:30:58.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:58.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463800+0000", 2024-09-17T21:30:58.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030542030000000001 2024-09-17T21:30:58.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:30:58.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465867+0000", 2024-09-17T21:30:58.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020665929999999998 2024-09-17T21:30:58.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:30:58.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465914+0000", 2024-09-17T21:30:58.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497000000000003e-05 2024-09-17T21:30:58.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:58.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465980+0000", 2024-09-17T21:30:58.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6081000000000006e-05 2024-09-17T21:30:58.022 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:58.022 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:58.022 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:58.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.022 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:70218 3.16 3:68692d8f:::rbd_data.1120b9193b9f.000000000000001b:head [stat,delete] snapc 1d=[1d] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-17T21:30:58.022 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:58.022 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.120817338999998, 2024-09-17T21:30:58.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.951400786, 2024-09-17T21:30:58.023 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:30:58.023 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:30:58.023 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:30:58.023 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:30:58.023 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:30:58.023 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.023 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 70218 2024-09-17T21:30:58.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.023 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:30:58.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:30:58.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:58.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:58.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:30:58.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:58.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:30:58.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:30:58.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:30:58.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4799999999999998e-07 2024-09-17T21:30:58.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:30:58.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805642+0000", 2024-09-17T21:30:58.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2009999999999999e-06 2024-09-17T21:30:58.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:30:58.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805650+0000", 2024-09-17T21:30:58.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1000000000000004e-06 2024-09-17T21:30:58.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805655+0000", 2024-09-17T21:30:58.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:30:58.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:58.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805722+0000", 2024-09-17T21:30:58.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6878000000000006e-05 2024-09-17T21:30:58.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:30:58.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805733+0000", 2024-09-17T21:30:58.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0895e-05 2024-09-17T21:30:58.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:30:58.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755252+0000", 2024-09-17T21:30:58.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.94951843800000002 2024-09-17T21:30:58.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:30:58.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755398+0000", 2024-09-17T21:30:58.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014622600000000001 2024-09-17T21:30:58.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:30:58.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:30:58.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:30:58.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.757039+0000", 2024-09-17T21:30:58.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001640819 2024-09-17T21:30:58.027 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:58.027 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:58.027 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:58.027 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:30:58.027 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:30:58.028 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:30:58.028 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:30:58.030 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:30:58.030 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:30:58.027+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:30:58.130 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:58.128+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:58.137 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:58.137 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:58.137 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:58.137 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:58.147 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:30:58.231 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:30:58.228+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:30:58.251 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:58.251 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:58.251 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:58.251 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:58.251 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:58.260 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:30:58.332 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:58.330+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:58.404 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:58.404 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:58.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:58.404 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:58.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.404 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97777 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 196608~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.404 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966872+0000", 2024-09-17T21:30:58.404 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.434252996, 2024-09-17T21:30:58.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050666048999999998, 2024-09-17T21:30:58.404 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.404 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.404 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.405 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.405 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.405 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.405 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97777 2024-09-17T21:30:58.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.405 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966872+0000", 2024-09-17T21:30:58.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966872+0000", 2024-09-17T21:30:58.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966872+0000", 2024-09-17T21:30:58.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0399999999999996e-07 2024-09-17T21:30:58.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966874+0000", 2024-09-17T21:30:58.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0930000000000001e-06 2024-09-17T21:30:58.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966881+0000", 2024-09-17T21:30:58.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9070000000000001e-06 2024-09-17T21:30:58.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966886+0000", 2024-09-17T21:30:58.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7700000000000001e-06 2024-09-17T21:30:58.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017200+0000", 2024-09-17T21:30:58.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050314336000000001 2024-09-17T21:30:58.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017284+0000", 2024-09-17T21:30:58.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3968000000000007e-05 2024-09-17T21:30:58.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017538+0000", 2024-09-17T21:30:58.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025347100000000001 2024-09-17T21:30:58.408 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.408 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.408 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.408 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97778 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 245760~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.408 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:30:58.408 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.434226453999999, 2024-09-17T21:30:58.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050933728999999997, 2024-09-17T21:30:58.409 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.409 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.409 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.409 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.409 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.409 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.409 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97778 2024-09-17T21:30:58.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.409 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:30:58.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:30:58.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966899+0000", 2024-09-17T21:30:58.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:30:58.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966900+0000", 2024-09-17T21:30:58.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.384e-06 2024-09-17T21:30:58.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966905+0000", 2024-09-17T21:30:58.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6530000000000002e-06 2024-09-17T21:30:58.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966909+0000", 2024-09-17T21:30:58.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2889999999999999e-06 2024-09-17T21:30:58.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017585+0000", 2024-09-17T21:30:58.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050675691000000002 2024-09-17T21:30:58.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017642+0000", 2024-09-17T21:30:58.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7167999999999999e-05 2024-09-17T21:30:58.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017832+0000", 2024-09-17T21:30:58.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019022 2024-09-17T21:30:58.412 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.412 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.412 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.413 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97779 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.413 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:30:58.413 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.434209385999999, 2024-09-17T21:30:58.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051245027999999998, 2024-09-17T21:30:58.413 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.413 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.413 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.413 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.413 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.413 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.413 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97779 2024-09-17T21:30:58.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:30:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:30:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966916+0000", 2024-09-17T21:30:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:30:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966918+0000", 2024-09-17T21:30:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8369999999999999e-06 2024-09-17T21:30:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966924+0000", 2024-09-17T21:30:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2260000000000002e-06 2024-09-17T21:30:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966928+0000", 2024-09-17T21:30:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3819999999999997e-06 2024-09-17T21:30:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017866+0000", 2024-09-17T21:30:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050938079999999997 2024-09-17T21:30:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017945+0000", 2024-09-17T21:30:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8275000000000005e-05 2024-09-17T21:30:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018161+0000", 2024-09-17T21:30:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021579499999999999 2024-09-17T21:30:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97780 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 344064~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:30:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.434185107999999, 2024-09-17T21:30:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051535301999999998, 2024-09-17T21:30:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97780 2024-09-17T21:30:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:30:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:30:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:30:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:30:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966942+0000", 2024-09-17T21:30:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7439999999999999e-06 2024-09-17T21:30:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966946+0000", 2024-09-17T21:30:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5510000000000001e-06 2024-09-17T21:30:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966951+0000", 2024-09-17T21:30:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:30:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018191+0000", 2024-09-17T21:30:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051240290000000001 2024-09-17T21:30:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018269+0000", 2024-09-17T21:30:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7416999999999994e-05 2024-09-17T21:30:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018475+0000", 2024-09-17T21:30:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206486 2024-09-17T21:30:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97789 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 40960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:30:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.434164939, 2024-09-17T21:30:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051862143999999999, 2024-09-17T21:30:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97789 2024-09-17T21:30:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:30:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:30:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:30:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5899999999999996e-07 2024-09-17T21:30:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966962+0000", 2024-09-17T21:30:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9700000000000002e-06 2024-09-17T21:30:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966969+0000", 2024-09-17T21:30:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6939999999999998e-06 2024-09-17T21:30:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966980+0000", 2024-09-17T21:30:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0679999999999999e-05 2024-09-17T21:30:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018513+0000", 2024-09-17T21:30:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051533420000000003 2024-09-17T21:30:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018595+0000", 2024-09-17T21:30:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1260999999999993e-05 2024-09-17T21:30:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018822+0000", 2024-09-17T21:30:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022756 2024-09-17T21:30:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97790 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 90112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:30:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.434134440999999, 2024-09-17T21:30:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052173479000000002, 2024-09-17T21:30:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97790 2024-09-17T21:30:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:30:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:30:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966991+0000", 2024-09-17T21:30:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9500000000000003e-07 2024-09-17T21:30:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966993+0000", 2024-09-17T21:30:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.238e-06 2024-09-17T21:30:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967000+0000", 2024-09-17T21:30:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9500000000000004e-06 2024-09-17T21:30:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967005+0000", 2024-09-17T21:30:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:30:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018852+0000", 2024-09-17T21:30:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051847250999999997 2024-09-17T21:30:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018927+0000", 2024-09-17T21:30:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4944000000000006e-05 2024-09-17T21:30:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019164+0000", 2024-09-17T21:30:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000236803 2024-09-17T21:30:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97791 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 139264~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:30:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.434112697, 2024-09-17T21:30:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052464594000000003, 2024-09-17T21:30:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97791 2024-09-17T21:30:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:30:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:30:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967013+0000", 2024-09-17T21:30:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:30:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967014+0000", 2024-09-17T21:30:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8679999999999999e-06 2024-09-17T21:30:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967020+0000", 2024-09-17T21:30:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.322e-06 2024-09-17T21:30:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967024+0000", 2024-09-17T21:30:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2660000000000003e-06 2024-09-17T21:30:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019193+0000", 2024-09-17T21:30:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052169010000000002 2024-09-17T21:30:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019269+0000", 2024-09-17T21:30:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6149999999999994e-05 2024-09-17T21:30:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019477+0000", 2024-09-17T21:30:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020753699999999999 2024-09-17T21:30:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97792 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 188416~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:30:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.434094957999999, 2024-09-17T21:30:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052764958000000001, 2024-09-17T21:30:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97792 2024-09-17T21:30:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:30:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:30:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:30:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0900000000000002e-07 2024-09-17T21:30:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967032+0000", 2024-09-17T21:30:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9740000000000001e-06 2024-09-17T21:30:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967039+0000", 2024-09-17T21:30:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3679999999999998e-06 2024-09-17T21:30:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967043+0000", 2024-09-17T21:30:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3379999999999998e-06 2024-09-17T21:30:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019505+0000", 2024-09-17T21:30:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052462264000000002 2024-09-17T21:30:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019582+0000", 2024-09-17T21:30:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7146999999999998e-05 2024-09-17T21:30:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019795+0000", 2024-09-17T21:30:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000212458 2024-09-17T21:30:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97793 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 237568~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:30:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.434073458, 2024-09-17T21:30:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053070436999999998, 2024-09-17T21:30:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97793 2024-09-17T21:30:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:30:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:30:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967052+0000", 2024-09-17T21:30:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7500000000000001e-07 2024-09-17T21:30:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967054+0000", 2024-09-17T21:30:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:30:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967059+0000", 2024-09-17T21:30:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5840000000000002e-06 2024-09-17T21:30:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967064+0000", 2024-09-17T21:30:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:30:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019826+0000", 2024-09-17T21:30:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052762396000000003 2024-09-17T21:30:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019900+0000", 2024-09-17T21:30:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3594e-05 2024-09-17T21:30:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020122+0000", 2024-09-17T21:30:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022185999999999999 2024-09-17T21:30:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97794 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 286720~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:30:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.434053322, 2024-09-17T21:30:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053453734000000003, 2024-09-17T21:30:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97794 2024-09-17T21:30:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:30:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:30:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:30:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2199999999999999e-07 2024-09-17T21:30:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967074+0000", 2024-09-17T21:30:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8419999999999999e-06 2024-09-17T21:30:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967080+0000", 2024-09-17T21:30:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5660000000000003e-06 2024-09-17T21:30:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967120+0000", 2024-09-17T21:30:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9504e-05 2024-09-17T21:30:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020189+0000", 2024-09-17T21:30:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053069144999999998 2024-09-17T21:30:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020287+0000", 2024-09-17T21:30:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8259999999999998e-05 2024-09-17T21:30:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020525+0000", 2024-09-17T21:30:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023799499999999999 2024-09-17T21:30:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97795 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 335872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:30:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.433995188000001, 2024-09-17T21:30:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053707207999999999, 2024-09-17T21:30:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97795 2024-09-17T21:30:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:30:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:30:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:30:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:30:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967139+0000", 2024-09-17T21:30:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5790000000000004e-06 2024-09-17T21:30:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967145+0000", 2024-09-17T21:30:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3550000000000001e-06 2024-09-17T21:30:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967155+0000", 2024-09-17T21:30:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8859999999999999e-06 2024-09-17T21:30:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020555+0000", 2024-09-17T21:30:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053400359000000001 2024-09-17T21:30:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020630+0000", 2024-09-17T21:30:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4894999999999994e-05 2024-09-17T21:30:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020837+0000", 2024-09-17T21:30:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020670500000000001 2024-09-17T21:30:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97796 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 385024~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:30:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.433962928, 2024-09-17T21:30:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053990227000000002, 2024-09-17T21:30:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97796 2024-09-17T21:30:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:30:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:30:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:30:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:30:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967164+0000", 2024-09-17T21:30:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0370000000000001e-06 2024-09-17T21:30:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967171+0000", 2024-09-17T21:30:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2190000000000001e-06 2024-09-17T21:30:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967175+0000", 2024-09-17T21:30:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2599999999999999e-06 2024-09-17T21:30:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020866+0000", 2024-09-17T21:30:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053691539000000003 2024-09-17T21:30:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020942+0000", 2024-09-17T21:30:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5487000000000004e-05 2024-09-17T21:30:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021152+0000", 2024-09-17T21:30:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021021599999999999 2024-09-17T21:30:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97805 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 8192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:30:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.433944248, 2024-09-17T21:30:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054331771000000001, 2024-09-17T21:30:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97805 2024-09-17T21:30:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:30:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:30:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:30:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:30:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967183+0000", 2024-09-17T21:30:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6840000000000001e-06 2024-09-17T21:30:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967189+0000", 2024-09-17T21:30:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0299999999999999e-06 2024-09-17T21:30:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967193+0000", 2024-09-17T21:30:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-17T21:30:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021182+0000", 2024-09-17T21:30:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053989319000000001 2024-09-17T21:30:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021261+0000", 2024-09-17T21:30:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8460000000000004e-05 2024-09-17T21:30:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021512+0000", 2024-09-17T21:30:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025158500000000002 2024-09-17T21:30:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97806 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:30:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.433926245, 2024-09-17T21:30:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054654343000000001, 2024-09-17T21:30:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97806 2024-09-17T21:30:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:30:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:30:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:30:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:30:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967201+0000", 2024-09-17T21:30:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.691e-06 2024-09-17T21:30:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967206+0000", 2024-09-17T21:30:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7780000000000004e-06 2024-09-17T21:30:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967224+0000", 2024-09-17T21:30:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7357999999999999e-05 2024-09-17T21:30:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021546+0000", 2024-09-17T21:30:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054322237000000002 2024-09-17T21:30:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021618+0000", 2024-09-17T21:30:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2007000000000004e-05 2024-09-17T21:30:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021853+0000", 2024-09-17T21:30:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023495800000000001 2024-09-17T21:30:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97807 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 106496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:30:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.433893845, 2024-09-17T21:30:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055014183000000001, 2024-09-17T21:30:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97807 2024-09-17T21:30:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:30:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:30:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967232+0000", 2024-09-17T21:30:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:30:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967234+0000", 2024-09-17T21:30:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.283e-06 2024-09-17T21:30:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967240+0000", 2024-09-17T21:30:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.144e-06 2024-09-17T21:30:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967252+0000", 2024-09-17T21:30:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2322e-05 2024-09-17T21:30:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021882+0000", 2024-09-17T21:30:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054629952000000002 2024-09-17T21:30:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021952+0000", 2024-09-17T21:30:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9919999999999995e-05 2024-09-17T21:30:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022245+0000", 2024-09-17T21:30:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00029302100000000003 2024-09-17T21:30:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97808 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 155648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:30:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.433865246, 2024-09-17T21:30:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055447485999999997, 2024-09-17T21:30:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97808 2024-09-17T21:30:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:30:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:30:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:30:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4299999999999998e-07 2024-09-17T21:30:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967267+0000", 2024-09-17T21:30:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9820000000000002e-06 2024-09-17T21:30:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967273+0000", 2024-09-17T21:30:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:30:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967277+0000", 2024-09-17T21:30:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5050000000000001e-06 2024-09-17T21:30:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022297+0000", 2024-09-17T21:30:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055020131999999999 2024-09-17T21:30:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022379+0000", 2024-09-17T21:30:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1810000000000004e-05 2024-09-17T21:30:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022707+0000", 2024-09-17T21:30:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00032784300000000003 2024-09-17T21:30:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97809 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:30:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.433841730999999, 2024-09-17T21:30:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056012109999999997, 2024-09-17T21:30:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97809 2024-09-17T21:30:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:30:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:30:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967284+0000", 2024-09-17T21:30:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:30:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967285+0000", 2024-09-17T21:30:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.826e-06 2024-09-17T21:30:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967291+0000", 2024-09-17T21:30:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:30:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967302+0000", 2024-09-17T21:30:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1114999999999999e-05 2024-09-17T21:30:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022901+0000", 2024-09-17T21:30:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055599335 2024-09-17T21:30:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022990+0000", 2024-09-17T21:30:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8671999999999994e-05 2024-09-17T21:30:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023295+0000", 2024-09-17T21:30:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030554900000000001 2024-09-17T21:30:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97810 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 253952~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:30:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.433815302999999, 2024-09-17T21:30:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056355668999999997, 2024-09-17T21:30:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97810 2024-09-17T21:30:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:30:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:30:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:30:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0200000000000002e-07 2024-09-17T21:30:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967313+0000", 2024-09-17T21:30:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4549999999999998e-06 2024-09-17T21:30:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967319+0000", 2024-09-17T21:30:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3080000000000004e-06 2024-09-17T21:30:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967324+0000", 2024-09-17T21:30:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0159999999999999e-06 2024-09-17T21:30:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023342+0000", 2024-09-17T21:30:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056018635999999997 2024-09-17T21:30:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023423+0000", 2024-09-17T21:30:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0473e-05 2024-09-17T21:30:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023665+0000", 2024-09-17T21:30:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242279 2024-09-17T21:30:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97811 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 303104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:30:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.433789044999999, 2024-09-17T21:30:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056743978, 2024-09-17T21:30:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97811 2024-09-17T21:30:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:30:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:30:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:30:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1300000000000001e-07 2024-09-17T21:30:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967338+0000", 2024-09-17T21:30:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9740000000000001e-06 2024-09-17T21:30:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967346+0000", 2024-09-17T21:30:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8129999999999996e-06 2024-09-17T21:30:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967351+0000", 2024-09-17T21:30:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6129999999999997e-06 2024-09-17T21:30:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023750+0000", 2024-09-17T21:30:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056399796000000002 2024-09-17T21:30:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023839+0000", 2024-09-17T21:30:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.852e-05 2024-09-17T21:30:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024080+0000", 2024-09-17T21:30:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024084900000000001 2024-09-17T21:30:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97812 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 352256~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:30:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:30:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.433768429000001, 2024-09-17T21:30:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.057024455000000002, 2024-09-17T21:30:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97812 2024-09-17T21:30:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:30:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:30:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967357+0000", 2024-09-17T21:30:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:30:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967359+0000", 2024-09-17T21:30:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:30:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967365+0000", 2024-09-17T21:30:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:30:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967369+0000", 2024-09-17T21:30:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5029999999999999e-06 2024-09-17T21:30:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024109+0000", 2024-09-17T21:30:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056740038 2024-09-17T21:30:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024175+0000", 2024-09-17T21:30:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5106999999999999e-05 2024-09-17T21:30:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024381+0000", 2024-09-17T21:30:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020629700000000001 2024-09-17T21:30:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.494 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.494 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.494 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:58.495 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:58.493+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:58.495 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:30:58.596 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:58.594+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:58.638 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:58.638 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:58.639 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:58.639 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:58.647 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:30:58.696 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:58.694+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:58.750 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:58.750 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:58.750 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:58.750 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:58.751 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:58.758 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:30:58.799 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:58.797+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:58.899 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:58.897+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:58.903 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:58.903 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:58.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:58.903 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:58.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.904 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:37817 3.17 3:eab85a8d:::rbd_header.1120e62eea89:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:30:58.904 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:58.904 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 57.205296707999999, 2024-09-17T21:30:58.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86635970399999995, 2024-09-17T21:30:58.904 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.904 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.904 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.904 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37817 2024-09-17T21:30:58.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.905 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:58.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:30:58.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695457+0000", 2024-09-17T21:30:58.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5799999999999998e-07 2024-09-17T21:30:58.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695461+0000", 2024-09-17T21:30:58.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1110000000000001e-06 2024-09-17T21:30:58.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695471+0000", 2024-09-17T21:30:58.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0441e-05 2024-09-17T21:30:58.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695477+0000", 2024-09-17T21:30:58.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7599999999999999e-06 2024-09-17T21:30:58.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695533+0000", 2024-09-17T21:30:58.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6570999999999998e-05 2024-09-17T21:30:58.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:30:58.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695543+0000", 2024-09-17T21:30:58.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9979999999999998e-06 2024-09-17T21:30:58.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561472+0000", 2024-09-17T21:30:58.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86592841899999995 2024-09-17T21:30:58.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561610+0000", 2024-09-17T21:30:58.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013854699999999999 2024-09-17T21:30:58.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561816+0000", 2024-09-17T21:30:58.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020509900000000001 2024-09-17T21:30:58.908 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.908 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.909 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.909 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50202 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.909 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:58.909 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.793125918000001, 2024-09-17T21:30:58.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.076620261999999995, 2024-09-17T21:30:58.909 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.909 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.909 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:58.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.910 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50202 2024-09-17T21:30:58.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.910 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:58.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:30:58.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107629+0000", 2024-09-17T21:30:58.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3709999999999998e-06 2024-09-17T21:30:58.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119765+0000", 2024-09-17T21:30:58.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012135683 2024-09-17T21:30:58.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119787+0000", 2024-09-17T21:30:58.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2748999999999999e-05 2024-09-17T21:30:58.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119797+0000", 2024-09-17T21:30:58.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8169999999999994e-06 2024-09-17T21:30:58.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119872+0000", 2024-09-17T21:30:58.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270999999999994e-05 2024-09-17T21:30:58.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119987+0000", 2024-09-17T21:30:58.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011557699999999999 2024-09-17T21:30:58.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:58.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.121766+0000", 2024-09-17T21:30:58.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00177853 2024-09-17T21:30:58.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:58.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178307+0000", 2024-09-17T21:30:58.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056541559999999998 2024-09-17T21:30:58.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:58.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184097+0000", 2024-09-17T21:30:58.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057899029999999999 2024-09-17T21:30:58.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:58.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184172+0000", 2024-09-17T21:30:58.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5226000000000001e-05 2024-09-17T21:30:58.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184247+0000", 2024-09-17T21:30:58.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4574999999999997e-05 2024-09-17T21:30:58.915 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.915 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.915 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.915 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50203 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [truncate 20971520] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.915 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:58.916 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.780926012999998, 2024-09-17T21:30:58.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065009779000000004, 2024-09-17T21:30:58.916 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.916 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.916 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:58.916 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.916 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.916 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.916 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50203 2024-09-17T21:30:58.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.916 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:58.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:30:58.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119828+0000", 2024-09-17T21:30:58.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6390000000000001e-06 2024-09-17T21:30:58.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119832+0000", 2024-09-17T21:30:58.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7699999999999999e-06 2024-09-17T21:30:58.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119845+0000", 2024-09-17T21:30:58.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3093e-05 2024-09-17T21:30:58.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119852+0000", 2024-09-17T21:30:58.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6950000000000003e-06 2024-09-17T21:30:58.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.160945+0000", 2024-09-17T21:30:58.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041093076999999999 2024-09-17T21:30:58.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161071+0000", 2024-09-17T21:30:58.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012569799999999999 2024-09-17T21:30:58.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:30:58.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161283+0000", 2024-09-17T21:30:58.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021201999999999999 2024-09-17T21:30:58.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:58.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.179360+0000", 2024-09-17T21:30:58.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018077784999999999 2024-09-17T21:30:58.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:58.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184726+0000", 2024-09-17T21:30:58.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053652960000000003 2024-09-17T21:30:58.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:58.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184787+0000", 2024-09-17T21:30:58.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1402000000000006e-05 2024-09-17T21:30:58.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184836+0000", 2024-09-17T21:30:58.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9304000000000001e-05 2024-09-17T21:30:58.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.922 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.922 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50206 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.922 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:58.923 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.743844473000003, 2024-09-17T21:30:58.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.090361968000000001, 2024-09-17T21:30:58.923 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.923 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.923 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:58.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.923 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50206 2024-09-17T21:30:58.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.923 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:58.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:30:58.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156911+0000", 2024-09-17T21:30:58.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-17T21:30:58.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173670+0000", 2024-09-17T21:30:58.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016758907 2024-09-17T21:30:58.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173696+0000", 2024-09-17T21:30:58.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6329000000000002e-05 2024-09-17T21:30:58.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173708+0000", 2024-09-17T21:30:58.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2512e-05 2024-09-17T21:30:58.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173788+0000", 2024-09-17T21:30:58.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9332999999999994e-05 2024-09-17T21:30:58.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173918+0000", 2024-09-17T21:30:58.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013063300000000001 2024-09-17T21:30:58.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:58.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.176228+0000", 2024-09-17T21:30:58.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002309757 2024-09-17T21:30:58.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:58.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.245164+0000", 2024-09-17T21:30:58.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068936301000000005 2024-09-17T21:30:58.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:58.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247176+0000", 2024-09-17T21:30:58.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020117590000000001 2024-09-17T21:30:58.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:58.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247215+0000", 2024-09-17T21:30:58.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9094000000000002e-05 2024-09-17T21:30:58.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247270+0000", 2024-09-17T21:30:58.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4793999999999997e-05 2024-09-17T21:30:58.928 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.928 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.928 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.929 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50207 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [truncate 21556327] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.929 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:58.929 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.726995492999997, 2024-09-17T21:30:58.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.074055363999999999, 2024-09-17T21:30:58.929 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.929 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.929 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:58.929 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.929 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.929 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.930 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50207 2024-09-17T21:30:58.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.930 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:58.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:30:58.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173759+0000", 2024-09-17T21:30:58.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8160000000000001e-06 2024-09-17T21:30:58.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173763+0000", 2024-09-17T21:30:58.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5360000000000003e-06 2024-09-17T21:30:58.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173771+0000", 2024-09-17T21:30:58.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8849999999999999e-06 2024-09-17T21:30:58.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173777+0000", 2024-09-17T21:30:58.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7799999999999997e-06 2024-09-17T21:30:58.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221363+0000", 2024-09-17T21:30:58.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047585493 2024-09-17T21:30:58.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221430+0000", 2024-09-17T21:30:58.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7129000000000001e-05 2024-09-17T21:30:58.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:30:58.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221550+0000", 2024-09-17T21:30:58.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120658 2024-09-17T21:30:58.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:30:58.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246113+0000", 2024-09-17T21:30:58.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02456262 2024-09-17T21:30:58.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:30:58.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247744+0000", 2024-09-17T21:30:58.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001630692 2024-09-17T21:30:58.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:58.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247777+0000", 2024-09-17T21:30:58.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3015000000000002e-05 2024-09-17T21:30:58.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247812+0000", 2024-09-17T21:30:58.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.574e-05 2024-09-17T21:30:58.936 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.936 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.936 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.936 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50828 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.936 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:58.936 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.418883561000001, 2024-09-17T21:30:58.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047324185999999997, 2024-09-17T21:30:58.937 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.937 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.937 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.937 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50828 2024-09-17T21:30:58.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.937 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:58.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:58.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:30:58.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.46e-07 2024-09-17T21:30:58.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481871+0000", 2024-09-17T21:30:58.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T21:30:58.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481878+0000", 2024-09-17T21:30:58.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:30:58.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481882+0000", 2024-09-17T21:30:58.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:30:58.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529038+0000", 2024-09-17T21:30:58.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047156034999999999 2024-09-17T21:30:58.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529099+0000", 2024-09-17T21:30:58.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0316000000000003e-05 2024-09-17T21:30:58.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529193+0000", 2024-09-17T21:30:58.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4369000000000004e-05 2024-09-17T21:30:58.941 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.941 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.941 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.941 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50829 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 12845056~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.941 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:58.941 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.418864767000002, 2024-09-17T21:30:58.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047887596999999997, 2024-09-17T21:30:58.942 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.942 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.942 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.942 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50829 2024-09-17T21:30:58.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.942 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:58.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:58.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:30:58.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:30:58.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481890+0000", 2024-09-17T21:30:58.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.877e-06 2024-09-17T21:30:58.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481895+0000", 2024-09-17T21:30:58.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:30:58.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481900+0000", 2024-09-17T21:30:58.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4440000000000001e-06 2024-09-17T21:30:58.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529225+0000", 2024-09-17T21:30:58.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047325117999999999 2024-09-17T21:30:58.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529292+0000", 2024-09-17T21:30:58.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7100000000000005e-05 2024-09-17T21:30:58.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529775+0000", 2024-09-17T21:30:58.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048322799999999999 2024-09-17T21:30:58.945 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.945 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.945 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.946 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50830 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.946 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:58.946 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.418846365, 2024-09-17T21:30:58.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048062123999999998, 2024-09-17T21:30:58.946 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.946 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.946 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.946 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.946 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.946 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.946 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50830 2024-09-17T21:30:58.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.946 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:58.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:30:58.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481907+0000", 2024-09-17T21:30:58.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:30:58.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481909+0000", 2024-09-17T21:30:58.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T21:30:58.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481915+0000", 2024-09-17T21:30:58.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.02e-06 2024-09-17T21:30:58.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481920+0000", 2024-09-17T21:30:58.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6799999999999998e-06 2024-09-17T21:30:58.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529793+0000", 2024-09-17T21:30:58.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047873042999999997 2024-09-17T21:30:58.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529870+0000", 2024-09-17T21:30:58.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6967999999999999e-05 2024-09-17T21:30:58.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529968+0000", 2024-09-17T21:30:58.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8085999999999996e-05 2024-09-17T21:30:58.950 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.950 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.950 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.950 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50831 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 18087936~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.950 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:58.950 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.418826019000001, 2024-09-17T21:30:58.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048663304999999997, 2024-09-17T21:30:58.950 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.950 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.950 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50831 2024-09-17T21:30:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:30:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:30:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481929+0000", 2024-09-17T21:30:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2189999999999998e-06 2024-09-17T21:30:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481935+0000", 2024-09-17T21:30:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.383e-06 2024-09-17T21:30:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481939+0000", 2024-09-17T21:30:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2479999999999998e-06 2024-09-17T21:30:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530095+0000", 2024-09-17T21:30:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048156617999999998 2024-09-17T21:30:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530159+0000", 2024-09-17T21:30:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3394999999999999e-05 2024-09-17T21:30:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530590+0000", 2024-09-17T21:30:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043101199999999999 2024-09-17T21:30:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50832 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 19660800~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.418807372000003, 2024-09-17T21:30:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049339466999999998, 2024-09-17T21:30:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50832 2024-09-17T21:30:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:30:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481946+0000", 2024-09-17T21:30:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:30:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481951+0000", 2024-09-17T21:30:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1479999999999997e-06 2024-09-17T21:30:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481956+0000", 2024-09-17T21:30:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7339999999999996e-06 2024-09-17T21:30:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481961+0000", 2024-09-17T21:30:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4889999999999997e-06 2024-09-17T21:30:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530630+0000", 2024-09-17T21:30:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048668596000000001 2024-09-17T21:30:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530720+0000", 2024-09-17T21:30:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0163999999999999e-05 2024-09-17T21:30:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531285+0000", 2024-09-17T21:30:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056493199999999998 2024-09-17T21:30:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50833 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.418785227999997, 2024-09-17T21:30:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049908069999999999, 2024-09-17T21:30:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50833 2024-09-17T21:30:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:30:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481968+0000", 2024-09-17T21:30:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5600000000000001e-07 2024-09-17T21:30:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481970+0000", 2024-09-17T21:30:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:30:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481975+0000", 2024-09-17T21:30:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8499999999999999e-06 2024-09-17T21:30:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481980+0000", 2024-09-17T21:30:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-17T21:30:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531337+0000", 2024-09-17T21:30:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049356874000000002 2024-09-17T21:30:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531392+0000", 2024-09-17T21:30:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4826e-05 2024-09-17T21:30:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531875+0000", 2024-09-17T21:30:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048345099999999998 2024-09-17T21:30:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50834 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 24903680~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.418760460000001, 2024-09-17T21:30:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050534015000000002, 2024-09-17T21:30:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50834 2024-09-17T21:30:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:30:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481993+0000", 2024-09-17T21:30:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:30:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481995+0000", 2024-09-17T21:30:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.199e-06 2024-09-17T21:30:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482001+0000", 2024-09-17T21:30:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.772e-06 2024-09-17T21:30:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482006+0000", 2024-09-17T21:30:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0660000000000003e-06 2024-09-17T21:30:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531923+0000", 2024-09-17T21:30:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049917200000000002 2024-09-17T21:30:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532004+0000", 2024-09-17T21:30:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0758000000000002e-05 2024-09-17T21:30:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532526+0000", 2024-09-17T21:30:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00052258899999999995 2024-09-17T21:30:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50835 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 28573696~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.418740477, 2024-09-17T21:30:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051204701999999998, 2024-09-17T21:30:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50835 2024-09-17T21:30:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:30:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482013+0000", 2024-09-17T21:30:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:30:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482015+0000", 2024-09-17T21:30:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.272e-06 2024-09-17T21:30:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482020+0000", 2024-09-17T21:30:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3040000000000004e-06 2024-09-17T21:30:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482025+0000", 2024-09-17T21:30:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5650000000000003e-06 2024-09-17T21:30:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532569+0000", 2024-09-17T21:30:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050544661999999997 2024-09-17T21:30:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532649+0000", 2024-09-17T21:30:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9640000000000003e-05 2024-09-17T21:30:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533217+0000", 2024-09-17T21:30:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056784100000000003 2024-09-17T21:30:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50836 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 30146560~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.418720860000001, 2024-09-17T21:30:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051875589, 2024-09-17T21:30:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50836 2024-09-17T21:30:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:30:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3700000000000001e-07 2024-09-17T21:30:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482034+0000", 2024-09-17T21:30:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0059999999999999e-06 2024-09-17T21:30:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482040+0000", 2024-09-17T21:30:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:30:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482044+0000", 2024-09-17T21:30:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:30:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533260+0000", 2024-09-17T21:30:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051215416 2024-09-17T21:30:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533348+0000", 2024-09-17T21:30:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8695000000000004e-05 2024-09-17T21:30:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533907+0000", 2024-09-17T21:30:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055902199999999995 2024-09-17T21:30:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50844 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.418557434, 2024-09-17T21:30:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0475644, 2024-09-17T21:30:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50844 2024-09-17T21:30:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:30:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482196+0000", 2024-09-17T21:30:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.39e-07 2024-09-17T21:30:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482198+0000", 2024-09-17T21:30:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0099999999999998e-06 2024-09-17T21:30:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482204+0000", 2024-09-17T21:30:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.93e-06 2024-09-17T21:30:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482208+0000", 2024-09-17T21:30:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7439999999999998e-06 2024-09-17T21:30:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529318+0000", 2024-09-17T21:30:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047110051999999999 2024-09-17T21:30:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529396+0000", 2024-09-17T21:30:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7712000000000004e-05 2024-09-17T21:30:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529760+0000", 2024-09-17T21:30:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00036351299999999999 2024-09-17T21:30:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50845 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 21757952~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.418538331000001, 2024-09-17T21:30:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048084625999999998, 2024-09-17T21:30:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50845 2024-09-17T21:30:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:30:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482215+0000", 2024-09-17T21:30:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:30:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482216+0000", 2024-09-17T21:30:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.823e-06 2024-09-17T21:30:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482222+0000", 2024-09-17T21:30:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.507e-06 2024-09-17T21:30:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482227+0000", 2024-09-17T21:30:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6609999999999999e-06 2024-09-17T21:30:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529792+0000", 2024-09-17T21:30:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047565075999999998 2024-09-17T21:30:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529871+0000", 2024-09-17T21:30:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8855000000000003e-05 2024-09-17T21:30:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530299+0000", 2024-09-17T21:30:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000428307 2024-09-17T21:30:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50846 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 25427968~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.418515943999999, 2024-09-17T21:30:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048735142000000002, 2024-09-17T21:30:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50846 2024-09-17T21:30:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:30:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:30:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482239+0000", 2024-09-17T21:30:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.871e-06 2024-09-17T21:30:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482244+0000", 2024-09-17T21:30:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.49e-06 2024-09-17T21:30:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482249+0000", 2024-09-17T21:30:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6269999999999999e-06 2024-09-17T21:30:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530347+0000", 2024-09-17T21:30:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048097721000000003 2024-09-17T21:30:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530437+0000", 2024-09-17T21:30:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0604000000000002e-05 2024-09-17T21:30:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530972+0000", 2024-09-17T21:30:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053445000000000001 2024-09-17T21:30:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50847 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 27000832~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.418498048000004, 2024-09-17T21:30:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049308668, 2024-09-17T21:30:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50847 2024-09-17T21:30:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:30:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:30:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482257+0000", 2024-09-17T21:30:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.846e-06 2024-09-17T21:30:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482264+0000", 2024-09-17T21:30:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8170000000000001e-06 2024-09-17T21:30:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482283+0000", 2024-09-17T21:30:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9718e-05 2024-09-17T21:30:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531018+0000", 2024-09-17T21:30:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048734497000000002 2024-09-17T21:30:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531080+0000", 2024-09-17T21:30:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1795e-05 2024-09-17T21:30:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531563+0000", 2024-09-17T21:30:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048366499999999998 2024-09-17T21:30:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50848 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 30670848~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.418461506, 2024-09-17T21:30:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049893337000000003, 2024-09-17T21:30:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50848 2024-09-17T21:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482292+0000", 2024-09-17T21:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T21:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482294+0000", 2024-09-17T21:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3209999999999999e-06 2024-09-17T21:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482300+0000", 2024-09-17T21:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7189999999999998e-06 2024-09-17T21:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482305+0000", 2024-09-17T21:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T21:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531606+0000", 2024-09-17T21:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049301273999999999 2024-09-17T21:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531696+0000", 2024-09-17T21:30:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0161999999999995e-05 2024-09-17T21:30:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532184+0000", 2024-09-17T21:30:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048806400000000002 2024-09-17T21:30:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50849 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 32243712~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 47.418442065000001, 2024-09-17T21:30:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050444607000000002, 2024-09-17T21:30:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50849 2024-09-17T21:30:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:30:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482311+0000", 2024-09-17T21:30:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:30:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482316+0000", 2024-09-17T21:30:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1209999999999998e-06 2024-09-17T21:30:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482321+0000", 2024-09-17T21:30:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4779999999999999e-06 2024-09-17T21:30:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482326+0000", 2024-09-17T21:30:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7099999999999998e-06 2024-09-17T21:30:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532221+0000", 2024-09-17T21:30:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049894936000000001 2024-09-17T21:30:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532287+0000", 2024-09-17T21:30:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5914000000000006e-05 2024-09-17T21:30:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532755+0000", 2024-09-17T21:30:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000468107 2024-09-17T21:30:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.003 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.003 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.003 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:59.004 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:30:59.005 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is unset 2024-09-17T21:30:59.006 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:30:59.002+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:30:59.104 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:59.102+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:59.147 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:59.147 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:59.156 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:30:59.206 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:59.203+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:59.298 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:59.299 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:59.299 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:59.299 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:59.299 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:59.307 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:30:59.307 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:30:59.304+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:30:59.407 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:30:59.405+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:30:59.451 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:59.451 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:59.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:59.451 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:59.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.451 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "pg_update_log_missing(3.5 epoch 45/37 rep_tid 844 entries 45'73 (0'0) error 3:a65c40ae:::rbd_data.1120b21c3309.0000000000000126:head by client.4384.0:48900 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 45'70)", 2024-09-17T21:30:59.451 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:59.451 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 52.401062965000001, 2024-09-17T21:30:59.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019554365000000001, 2024-09-17T21:30:59.451 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.452 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.452 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T21:30:59.452 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:59.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:30:59.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047112+0000", 2024-09-17T21:30:59.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:30:59.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047121+0000", 2024-09-17T21:30:59.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4379999999999995e-06 2024-09-17T21:30:59.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047139+0000", 2024-09-17T21:30:59.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8235000000000001e-05 2024-09-17T21:30:59.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047303+0000", 2024-09-17T21:30:59.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016389400000000001 2024-09-17T21:30:59.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047396+0000", 2024-09-17T21:30:59.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3355000000000003e-05 2024-09-17T21:30:59.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066666+0000", 2024-09-17T21:30:59.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019269649999999999 2024-09-17T21:30:59.455 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.455 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.455 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.455 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48906 3.4 e45/25 3:2503fe27:::rbd_data.1120b21c3309.0000000000000127:head v 45'148, mlcod=45'146)", 2024-09-17T21:30:59.455 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:59.455 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 52.398936786, 2024-09-17T21:30:59.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018169739000000001, 2024-09-17T21:30:59.455 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.455 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.455 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:59.455 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:59.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:59.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:30:59.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:30:59.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049244+0000", 2024-09-17T21:30:59.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8390000000000003e-06 2024-09-17T21:30:59.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066411+0000", 2024-09-17T21:30:59.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017166818 2024-09-17T21:30:59.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066426+0000", 2024-09-17T21:30:59.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4888999999999999e-05 2024-09-17T21:30:59.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066611+0000", 2024-09-17T21:30:59.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018464499999999999 2024-09-17T21:30:59.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066630+0000", 2024-09-17T21:30:59.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9201000000000001e-05 2024-09-17T21:30:59.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:59.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067373+0000", 2024-09-17T21:30:59.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00074322600000000004 2024-09-17T21:30:59.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067407+0000", 2024-09-17T21:30:59.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4357999999999999e-05 2024-09-17T21:30:59.459 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.459 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.459 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.459 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50206 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'301, mlcod=45'299)", 2024-09-17T21:30:59.459 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:59.459 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.269459941999997, 2024-09-17T21:30:59.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068285810000000002, 2024-09-17T21:30:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:30:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178716+0000", 2024-09-17T21:30:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.438e-06 2024-09-17T21:30:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192044+0000", 2024-09-17T21:30:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013328370000000001 2024-09-17T21:30:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192069+0000", 2024-09-17T21:30:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4496000000000001e-05 2024-09-17T21:30:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192085+0000", 2024-09-17T21:30:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5758999999999999e-05 2024-09-17T21:30:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192167+0000", 2024-09-17T21:30:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2842000000000004e-05 2024-09-17T21:30:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192201+0000", 2024-09-17T21:30:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3127999999999999e-05 2024-09-17T21:30:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246938+0000", 2024-09-17T21:30:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054736982000000003 2024-09-17T21:30:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247000+0000", 2024-09-17T21:30:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2794999999999998e-05 2024-09-17T21:30:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50207 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'303, mlcod=45'299)", 2024-09-17T21:30:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.226512892999999, 2024-09-17T21:30:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026003433999999999, 2024-09-17T21:30:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:30:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221663+0000", 2024-09-17T21:30:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.271e-06 2024-09-17T21:30:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221672+0000", 2024-09-17T21:30:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6810000000000004e-06 2024-09-17T21:30:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221684+0000", 2024-09-17T21:30:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:30:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221691+0000", 2024-09-17T21:30:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:30:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221743+0000", 2024-09-17T21:30:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2868999999999998e-05 2024-09-17T21:30:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221765+0000", 2024-09-17T21:30:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1316999999999999e-05 2024-09-17T21:30:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247616+0000", 2024-09-17T21:30:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025851504000000001 2024-09-17T21:30:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247665+0000", 2024-09-17T21:30:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8739000000000002e-05 2024-09-17T21:30:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57808 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 196608~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.670213853, 2024-09-17T21:30:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017751475999999999, 2024-09-17T21:30:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57808 2024-09-17T21:30:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:30:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:30:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777968+0000", 2024-09-17T21:30:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8530000000000003e-06 2024-09-17T21:30:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777975+0000", 2024-09-17T21:30:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4410000000000004e-06 2024-09-17T21:30:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777980+0000", 2024-09-17T21:30:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7210000000000002e-06 2024-09-17T21:30:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795323+0000", 2024-09-17T21:30:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017343319999999999 2024-09-17T21:30:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795410+0000", 2024-09-17T21:30:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6567e-05 2024-09-17T21:30:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795712+0000", 2024-09-17T21:30:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030219200000000001 2024-09-17T21:30:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57809 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 458752~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:30:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.670188001, 2024-09-17T21:30:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018092869000000001, 2024-09-17T21:30:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57809 2024-09-17T21:30:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:30:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:30:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777987+0000", 2024-09-17T21:30:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T21:30:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777989+0000", 2024-09-17T21:30:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8649999999999999e-06 2024-09-17T21:30:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777994+0000", 2024-09-17T21:30:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4199999999999998e-06 2024-09-17T21:30:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778004+0000", 2024-09-17T21:30:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6900000000000004e-06 2024-09-17T21:30:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795745+0000", 2024-09-17T21:30:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017740859000000001 2024-09-17T21:30:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795815+0000", 2024-09-17T21:30:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0219e-05 2024-09-17T21:30:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796079+0000", 2024-09-17T21:30:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000264446 2024-09-17T21:30:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57810 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 720896~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:30:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.670164274000001, 2024-09-17T21:30:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018361253000000001, 2024-09-17T21:30:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57810 2024-09-17T21:30:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:30:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:30:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778011+0000", 2024-09-17T21:30:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778012+0000", 2024-09-17T21:30:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.872e-06 2024-09-17T21:30:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778018+0000", 2024-09-17T21:30:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8150000000000002e-06 2024-09-17T21:30:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778023+0000", 2024-09-17T21:30:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3939999999999998e-06 2024-09-17T21:30:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796106+0000", 2024-09-17T21:30:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018083446999999999 2024-09-17T21:30:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796159+0000", 2024-09-17T21:30:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3029999999999999e-05 2024-09-17T21:30:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796371+0000", 2024-09-17T21:30:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021231700000000001 2024-09-17T21:30:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57811 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.670145853999998, 2024-09-17T21:30:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018666035000000001, 2024-09-17T21:30:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57811 2024-09-17T21:30:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:30:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:30:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778031+0000", 2024-09-17T21:30:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.796e-06 2024-09-17T21:30:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778036+0000", 2024-09-17T21:30:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3290000000000001e-06 2024-09-17T21:30:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778041+0000", 2024-09-17T21:30:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5839999999999996e-06 2024-09-17T21:30:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796409+0000", 2024-09-17T21:30:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018368703 2024-09-17T21:30:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796483+0000", 2024-09-17T21:30:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3751999999999995e-05 2024-09-17T21:30:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796695+0000", 2024-09-17T21:30:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021153 2024-09-17T21:30:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57812 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1245184~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.670126957000001, 2024-09-17T21:30:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019000064000000001, 2024-09-17T21:30:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57812 2024-09-17T21:30:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:30:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:30:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778050+0000", 2024-09-17T21:30:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7e-06 2024-09-17T21:30:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778055+0000", 2024-09-17T21:30:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4049999999999999e-06 2024-09-17T21:30:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778059+0000", 2024-09-17T21:30:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1479999999999999e-06 2024-09-17T21:30:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796734+0000", 2024-09-17T21:30:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018674399000000001 2024-09-17T21:30:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796786+0000", 2024-09-17T21:30:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2879e-05 2024-09-17T21:30:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797048+0000", 2024-09-17T21:30:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026118400000000003 2024-09-17T21:30:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57813 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1507328~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:30:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.670109064999998, 2024-09-17T21:30:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019279392999999999, 2024-09-17T21:30:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57813 2024-09-17T21:30:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:30:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:30:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778066+0000", 2024-09-17T21:30:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6199999999999999e-07 2024-09-17T21:30:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778068+0000", 2024-09-17T21:30:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7990000000000001e-06 2024-09-17T21:30:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778074+0000", 2024-09-17T21:30:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:30:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778078+0000", 2024-09-17T21:30:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2579999999999997e-06 2024-09-17T21:30:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797077+0000", 2024-09-17T21:30:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018999180000000001 2024-09-17T21:30:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797123+0000", 2024-09-17T21:30:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5834999999999997e-05 2024-09-17T21:30:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797345+0000", 2024-09-17T21:30:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022191100000000001 2024-09-17T21:30:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57814 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:30:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.670090193, 2024-09-17T21:30:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019604947000000001, 2024-09-17T21:30:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57814 2024-09-17T21:30:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:30:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:30:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778085+0000", 2024-09-17T21:30:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778086+0000", 2024-09-17T21:30:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7850000000000001e-06 2024-09-17T21:30:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778093+0000", 2024-09-17T21:30:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6449999999999999e-06 2024-09-17T21:30:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778098+0000", 2024-09-17T21:30:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:30:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797383+0000", 2024-09-17T21:30:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019284879000000001 2024-09-17T21:30:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797431+0000", 2024-09-17T21:30:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8856000000000001e-05 2024-09-17T21:30:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797689+0000", 2024-09-17T21:30:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025785800000000002 2024-09-17T21:30:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57815 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2031616~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.670069525000002, 2024-09-17T21:30:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019910231, 2024-09-17T21:30:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57815 2024-09-17T21:30:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:30:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:30:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778118+0000", 2024-09-17T21:30:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2967e-05 2024-09-17T21:30:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778125+0000", 2024-09-17T21:30:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.601e-06 2024-09-17T21:30:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778130+0000", 2024-09-17T21:30:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1839999999999998e-06 2024-09-17T21:30:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797720+0000", 2024-09-17T21:30:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019589766000000002 2024-09-17T21:30:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797765+0000", 2024-09-17T21:30:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5281999999999998e-05 2024-09-17T21:30:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798015+0000", 2024-09-17T21:30:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025006200000000001 2024-09-17T21:30:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57816 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2293760~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.670036930999999, 2024-09-17T21:30:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020175322999999998, 2024-09-17T21:30:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57816 2024-09-17T21:30:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:30:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:30:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778140+0000", 2024-09-17T21:30:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T21:30:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778145+0000", 2024-09-17T21:30:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7320000000000003e-06 2024-09-17T21:30:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778154+0000", 2024-09-17T21:30:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6969999999999999e-06 2024-09-17T21:30:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798049+0000", 2024-09-17T21:30:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019894603 2024-09-17T21:30:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798106+0000", 2024-09-17T21:30:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7207000000000003e-05 2024-09-17T21:30:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798313+0000", 2024-09-17T21:30:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206861 2024-09-17T21:30:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57817 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2555904~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.670013685000001, 2024-09-17T21:30:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020466314999999999, 2024-09-17T21:30:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57817 2024-09-17T21:30:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:30:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2399999999999999e-07 2024-09-17T21:30:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778163+0000", 2024-09-17T21:30:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.204e-06 2024-09-17T21:30:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778169+0000", 2024-09-17T21:30:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:30:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778173+0000", 2024-09-17T21:30:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:30:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798342+0000", 2024-09-17T21:30:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020168416000000002 2024-09-17T21:30:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798388+0000", 2024-09-17T21:30:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6171000000000002e-05 2024-09-17T21:30:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798627+0000", 2024-09-17T21:30:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000239058 2024-09-17T21:30:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57818 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2818048~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:30:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.669994987999999, 2024-09-17T21:30:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020767240999999999, 2024-09-17T21:30:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57818 2024-09-17T21:30:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:30:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:30:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778180+0000", 2024-09-17T21:30:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:30:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778182+0000", 2024-09-17T21:30:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7269999999999999e-06 2024-09-17T21:30:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778188+0000", 2024-09-17T21:30:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6080000000000001e-06 2024-09-17T21:30:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778202+0000", 2024-09-17T21:30:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3369e-05 2024-09-17T21:30:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798660+0000", 2024-09-17T21:30:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020458171000000001 2024-09-17T21:30:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798704+0000", 2024-09-17T21:30:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4273000000000001e-05 2024-09-17T21:30:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798947+0000", 2024-09-17T21:30:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242651 2024-09-17T21:30:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57819 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:30:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.669964118999999, 2024-09-17T21:30:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020892340999999998, 2024-09-17T21:30:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57819 2024-09-17T21:30:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:30:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:30:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778211+0000", 2024-09-17T21:30:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:30:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778217+0000", 2024-09-17T21:30:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T21:30:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778224+0000", 2024-09-17T21:30:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3520000000000003e-06 2024-09-17T21:30:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778228+0000", 2024-09-17T21:30:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7480000000000001e-06 2024-09-17T21:30:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798977+0000", 2024-09-17T21:30:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020748571 2024-09-17T21:30:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799035+0000", 2024-09-17T21:30:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7908999999999997e-05 2024-09-17T21:30:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799103+0000", 2024-09-17T21:30:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7794999999999997e-05 2024-09-17T21:30:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57820 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.669938926999997, 2024-09-17T21:30:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021016658000000001, 2024-09-17T21:30:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57820 2024-09-17T21:30:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:30:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:30:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778238+0000", 2024-09-17T21:30:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.945e-06 2024-09-17T21:30:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778244+0000", 2024-09-17T21:30:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6520000000000003e-06 2024-09-17T21:30:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778248+0000", 2024-09-17T21:30:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5920000000000002e-06 2024-09-17T21:30:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799131+0000", 2024-09-17T21:30:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020883288999999999 2024-09-17T21:30:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799185+0000", 2024-09-17T21:30:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3223000000000002e-05 2024-09-17T21:30:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799252+0000", 2024-09-17T21:30:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7576999999999994e-05 2024-09-17T21:30:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57821 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.669920582000003, 2024-09-17T21:30:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021153528000000001, 2024-09-17T21:30:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57821 2024-09-17T21:30:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:30:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-07 2024-09-17T21:30:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778256+0000", 2024-09-17T21:30:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.604e-06 2024-09-17T21:30:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778262+0000", 2024-09-17T21:30:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7629999999999997e-06 2024-09-17T21:30:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778266+0000", 2024-09-17T21:30:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:30:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799284+0000", 2024-09-17T21:30:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021017312 2024-09-17T21:30:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799326+0000", 2024-09-17T21:30:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2639000000000003e-05 2024-09-17T21:30:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799407+0000", 2024-09-17T21:30:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1077999999999999e-05 2024-09-17T21:30:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57822 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:30:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.669902272000002, 2024-09-17T21:30:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021268966, 2024-09-17T21:30:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57822 2024-09-17T21:30:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:30:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:30:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778273+0000", 2024-09-17T21:30:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:30:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778274+0000", 2024-09-17T21:30:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.598e-06 2024-09-17T21:30:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778283+0000", 2024-09-17T21:30:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8279999999999992e-06 2024-09-17T21:30:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778289+0000", 2024-09-17T21:30:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5609999999999998e-06 2024-09-17T21:30:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799432+0000", 2024-09-17T21:30:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021143862999999999 2024-09-17T21:30:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799473+0000", 2024-09-17T21:30:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1075999999999998e-05 2024-09-17T21:30:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799541+0000", 2024-09-17T21:30:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7657999999999995e-05 2024-09-17T21:30:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57823 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:30:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.669879311000003, 2024-09-17T21:30:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021379770999999999, 2024-09-17T21:30:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57823 2024-09-17T21:30:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:30:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:30:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778297+0000", 2024-09-17T21:30:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:30:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778303+0000", 2024-09-17T21:30:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4559999999999999e-06 2024-09-17T21:30:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778307+0000", 2024-09-17T21:30:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6229999999999996e-06 2024-09-17T21:30:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799560+0000", 2024-09-17T21:30:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021252931999999999 2024-09-17T21:30:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799609+0000", 2024-09-17T21:30:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8301999999999999e-05 2024-09-17T21:30:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799675+0000", 2024-09-17T21:30:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6205999999999996e-05 2024-09-17T21:30:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.539 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:59.541 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:30:59.541 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:59.539+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:59.641 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:30:59.639+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:30:59.645 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:59.645 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:59.645 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:30:59.645 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:59.654 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:30:59.742 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:30:59.740+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:30:59.798 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:59.798 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:30:59.798 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:30:59.798 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:30:59.799 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:30:59.807 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:30:59.843 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:59.841+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:59.923 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:30:59.944 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:30:59.942+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:30:59.961 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:30:59.961 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:30:59.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:30:59.961 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:30:59.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.962 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:49361 3.19 e46/25 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:head v 46'135, mlcod=45'133)", 2024-09-17T21:30:59.962 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:59.962 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 52.204927646999998, 2024-09-17T21:30:59.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043477744999999998, 2024-09-17T21:30:59.962 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.962 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.962 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:30:59.962 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:59.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:59.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:30:59.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-07 2024-09-17T21:30:59.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769703+0000", 2024-09-17T21:30:59.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016134381 2024-09-17T21:30:59.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769724+0000", 2024-09-17T21:30:59.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0781999999999999e-05 2024-09-17T21:30:59.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769734+0000", 2024-09-17T21:30:59.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6870000000000006e-06 2024-09-17T21:30:59.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769797+0000", 2024-09-17T21:30:59.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3418999999999997e-05 2024-09-17T21:30:59.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769822+0000", 2024-09-17T21:30:59.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4652999999999999e-05 2024-09-17T21:30:59.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:30:59.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.796982+0000", 2024-09-17T21:30:59.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027160205 2024-09-17T21:30:59.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.797047+0000", 2024-09-17T21:30:59.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4455999999999994e-05 2024-09-17T21:30:59.965 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.965 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.966 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.966 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50805 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 17825792~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:59.966 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:30:59.966 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.463344153000001, 2024-09-17T21:30:59.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035591579999999998, 2024-09-17T21:30:59.966 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.966 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.966 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.966 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.966 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.966 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.967 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50805 2024-09-17T21:30:59.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.967 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:30:59.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:30:59.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495153+0000", 2024-09-17T21:30:59.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:30:59.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495155+0000", 2024-09-17T21:30:59.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1849999999999998e-06 2024-09-17T21:30:59.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495162+0000", 2024-09-17T21:30:59.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.64e-06 2024-09-17T21:30:59.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495166+0000", 2024-09-17T21:30:59.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:30:59.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530020+0000", 2024-09-17T21:30:59.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.034853824999999998 2024-09-17T21:30:59.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530076+0000", 2024-09-17T21:30:59.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5235999999999998e-05 2024-09-17T21:30:59.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530744+0000", 2024-09-17T21:30:59.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066835500000000003 2024-09-17T21:30:59.970 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.970 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.970 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.970 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50806 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19398656~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:59.970 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:59.970 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.463323684999999, 2024-09-17T21:30:59.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036408459999999997, 2024-09-17T21:30:59.970 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.970 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.970 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.970 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.971 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.971 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.971 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50806 2024-09-17T21:30:59.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.971 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:59.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:59.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:30:59.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.72e-07 2024-09-17T21:30:59.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495175+0000", 2024-09-17T21:30:59.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:30:59.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495182+0000", 2024-09-17T21:30:59.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8079999999999999e-06 2024-09-17T21:30:59.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495186+0000", 2024-09-17T21:30:59.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5549999999999996e-06 2024-09-17T21:30:59.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530791+0000", 2024-09-17T21:30:59.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035604963000000003 2024-09-17T21:30:59.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530855+0000", 2024-09-17T21:30:59.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3286000000000004e-05 2024-09-17T21:30:59.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531581+0000", 2024-09-17T21:30:59.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00072655699999999996 2024-09-17T21:30:59.974 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.974 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.974 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.974 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50807 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23068672~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:59.974 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:59.974 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.463304800000003, 2024-09-17T21:30:59.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037183314000000002, 2024-09-17T21:30:59.975 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.975 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.975 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.975 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.975 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.975 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.975 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50807 2024-09-17T21:30:59.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.975 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:59.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:59.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:30:59.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9999999999999999e-07 2024-09-17T21:30:59.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495194+0000", 2024-09-17T21:30:59.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.779e-06 2024-09-17T21:30:59.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495199+0000", 2024-09-17T21:30:59.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.468e-06 2024-09-17T21:30:59.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495203+0000", 2024-09-17T21:30:59.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0010000000000003e-06 2024-09-17T21:30:59.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531636+0000", 2024-09-17T21:30:59.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036432665000000003 2024-09-17T21:30:59.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531727+0000", 2024-09-17T21:30:59.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1607999999999994e-05 2024-09-17T21:30:59.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532375+0000", 2024-09-17T21:30:59.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00064749299999999996 2024-09-17T21:30:59.979 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.979 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.979 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.979 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50808 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24641536~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:59.979 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:30:59.979 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.463288057, 2024-09-17T21:30:59.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037870190999999997, 2024-09-17T21:30:59.979 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.979 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.979 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.979 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.979 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.980 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.980 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50808 2024-09-17T21:30:59.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.980 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:30:59.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:30:59.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495209+0000", 2024-09-17T21:30:59.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:30:59.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495215+0000", 2024-09-17T21:30:59.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1630000000000001e-06 2024-09-17T21:30:59.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495221+0000", 2024-09-17T21:30:59.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-17T21:30:59.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495226+0000", 2024-09-17T21:30:59.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8189999999999999e-06 2024-09-17T21:30:59.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532425+0000", 2024-09-17T21:30:59.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037199135000000001 2024-09-17T21:30:59.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532531+0000", 2024-09-17T21:30:59.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106314 2024-09-17T21:30:59.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533079+0000", 2024-09-17T21:30:59.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00054739700000000003 2024-09-17T21:30:59.983 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.983 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.983 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.983 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50809 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28311552~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:59.983 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:30:59.983 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.463265090999997, 2024-09-17T21:30:59.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038597605, 2024-09-17T21:30:59.984 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.984 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.984 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.984 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.984 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.984 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.984 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50809 2024-09-17T21:30:59.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.984 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:30:59.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:30:59.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495232+0000", 2024-09-17T21:30:59.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.39e-07 2024-09-17T21:30:59.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495234+0000", 2024-09-17T21:30:59.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T21:30:59.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495239+0000", 2024-09-17T21:30:59.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7969999999999997e-06 2024-09-17T21:30:59.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495244+0000", 2024-09-17T21:30:59.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999996e-06 2024-09-17T21:30:59.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533125+0000", 2024-09-17T21:30:59.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037881523 2024-09-17T21:30:59.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533237+0000", 2024-09-17T21:30:59.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000111879 2024-09-17T21:30:59.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533829+0000", 2024-09-17T21:30:59.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000591933 2024-09-17T21:30:59.987 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.987 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.987 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.988 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50810 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 29884416~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:59.988 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:30:59.988 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.463247172999999, 2024-09-17T21:30:59.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039149809000000001, 2024-09-17T21:30:59.988 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.988 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.988 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.988 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.988 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.988 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.988 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50810 2024-09-17T21:30:59.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.989 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:30:59.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:30:59.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495250+0000", 2024-09-17T21:30:59.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:30:59.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495251+0000", 2024-09-17T21:30:59.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.81e-06 2024-09-17T21:30:59.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495257+0000", 2024-09-17T21:30:59.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1399999999999999e-06 2024-09-17T21:30:59.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495261+0000", 2024-09-17T21:30:59.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4229999999999998e-06 2024-09-17T21:30:59.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533879+0000", 2024-09-17T21:30:59.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038618292999999998 2024-09-17T21:30:59.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533939+0000", 2024-09-17T21:30:59.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.012e-05 2024-09-17T21:30:59.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534399+0000", 2024-09-17T21:30:59.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00045968099999999998 2024-09-17T21:30:59.992 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.992 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.992 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.992 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50850 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 2359296~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:59.992 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:59.992 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.463228798000003, 2024-09-17T21:30:59.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039731491000000001, 2024-09-17T21:30:59.993 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.993 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.993 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.993 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.993 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.993 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.993 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50850 2024-09-17T21:30:59.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.993 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:59.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:59.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:30:59.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3200000000000001e-07 2024-09-17T21:30:59.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495270+0000", 2024-09-17T21:30:59.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.694e-06 2024-09-17T21:30:59.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495275+0000", 2024-09-17T21:30:59.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5890000000000002e-06 2024-09-17T21:30:59.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495279+0000", 2024-09-17T21:30:59.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-17T21:30:59.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:30:59.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534440+0000", 2024-09-17T21:30:59.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039160805999999999 2024-09-17T21:30:59.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:30:59.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534526+0000", 2024-09-17T21:30:59.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5327999999999993e-05 2024-09-17T21:30:59.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:30:59.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534999+0000", 2024-09-17T21:30:59.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00047353400000000001 2024-09-17T21:30:59.996 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.996 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:30:59.996 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:30:59.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.997 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50851 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 3932160~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:30:59.997 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:30:59.997 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.463211186000002, 2024-09-17T21:30:59.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040211991000000002, 2024-09-17T21:30:59.997 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:30:59.997 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:30:59.997 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:30:59.997 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:30:59.997 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:30:59.997 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:30:59.997 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50851 2024-09-17T21:30:59.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.998 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:30:59.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:30:59.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:30:59.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:30:59.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:30:59.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:30:59.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:30:59.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495286+0000", 2024-09-17T21:30:59.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:30:59.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:30:59.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495287+0000", 2024-09-17T21:30:59.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6640000000000001e-06 2024-09-17T21:30:59.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:30:59.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495293+0000", 2024-09-17T21:30:59.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6710000000000004e-06 2024-09-17T21:30:59.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:30:59.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:30:59.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:30:59.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495298+0000", 2024-09-17T21:31:00.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6700000000000002e-06 2024-09-17T21:31:00.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:00.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535017+0000", 2024-09-17T21:31:00.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039719059000000001 2024-09-17T21:31:00.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:00.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535067+0000", 2024-09-17T21:31:00.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0046000000000002e-05 2024-09-17T21:31:00.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:00.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535497+0000", 2024-09-17T21:31:00.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043050299999999999 2024-09-17T21:31:00.001 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.001 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:00.001 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.001 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50852 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 7602176~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:00.001 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:00.001 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.463192268999997, 2024-09-17T21:31:00.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041166337999999997, 2024-09-17T21:31:00.001 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:00.001 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:00.001 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:00.001 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:00.002 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:00.002 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:00.002 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50852 2024-09-17T21:31:00.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.002 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:00.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:00.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:00.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:00.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:00.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:00.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495305+0000", 2024-09-17T21:31:00.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1600000000000002e-07 2024-09-17T21:31:00.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:00.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495307+0000", 2024-09-17T21:31:00.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0980000000000001e-06 2024-09-17T21:31:00.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:00.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495312+0000", 2024-09-17T21:31:00.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6210000000000001e-06 2024-09-17T21:31:00.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:00.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495317+0000", 2024-09-17T21:31:00.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T21:31:00.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:00.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535531+0000", 2024-09-17T21:31:00.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040214580999999999 2024-09-17T21:31:00.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:00.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535618+0000", 2024-09-17T21:31:00.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6648000000000001e-05 2024-09-17T21:31:00.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:00.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536471+0000", 2024-09-17T21:31:00.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00085251599999999999 2024-09-17T21:31:00.005 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.005 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:00.005 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.006 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50853 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 9175040~204800] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:00.006 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:00.006 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.463173937999997, 2024-09-17T21:31:00.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041673496999999997, 2024-09-17T21:31:00.006 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:00.006 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:00.006 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:00.006 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:00.006 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:00.006 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:00.006 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50853 2024-09-17T21:31:00.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.006 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:00.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:00.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:00.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:00.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:00.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:00.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:00.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:31:00.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:00.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495334+0000", 2024-09-17T21:31:00.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.076e-05 2024-09-17T21:31:00.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:00.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495340+0000", 2024-09-17T21:31:00.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:31:00.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:00.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495345+0000", 2024-09-17T21:31:00.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:31:00.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:00.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536514+0000", 2024-09-17T21:31:00.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041168534999999999 2024-09-17T21:31:00.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:00.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536572+0000", 2024-09-17T21:31:00.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8742000000000001e-05 2024-09-17T21:31:00.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:00.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536996+0000", 2024-09-17T21:31:00.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042368899999999999 2024-09-17T21:31:00.010 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.010 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:00.010 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.010 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50854 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 12845056~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:00.010 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:00.010 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.463144878999998, 2024-09-17T21:31:00.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042777397000000002, 2024-09-17T21:31:00.010 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:00.010 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:00.010 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:00.010 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:00.010 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:00.011 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:00.011 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50854 2024-09-17T21:31:00.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.011 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:00.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:00.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:00.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:00.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:00.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:00.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:00.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.82e-07 2024-09-17T21:31:00.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:00.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495354+0000", 2024-09-17T21:31:00.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.091e-06 2024-09-17T21:31:00.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:00.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495360+0000", 2024-09-17T21:31:00.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.716e-06 2024-09-17T21:31:00.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:00.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495365+0000", 2024-09-17T21:31:00.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2939999999999996e-06 2024-09-17T21:31:00.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:00.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537020+0000", 2024-09-17T21:31:00.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041654419999999998 2024-09-17T21:31:00.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:00.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537105+0000", 2024-09-17T21:31:00.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5857000000000001e-05 2024-09-17T21:31:00.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:00.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538129+0000", 2024-09-17T21:31:00.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010235369999999999 2024-09-17T21:31:00.014 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.014 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:00.014 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.014 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50855 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 14417920~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:00.014 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:00.014 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.463124913999998, 2024-09-17T21:31:00.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043254325000000003, 2024-09-17T21:31:00.015 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:00.015 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:00.015 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:00.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:00.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:00.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:00.015 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50855 2024-09-17T21:31:00.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.015 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:00.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:00.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:00.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:00.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:00.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:00.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:00.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:31:00.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:00.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495374+0000", 2024-09-17T21:31:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.922e-06 2024-09-17T21:31:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495379+0000", 2024-09-17T21:31:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-17T21:31:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495384+0000", 2024-09-17T21:31:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-17T21:31:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538167+0000", 2024-09-17T21:31:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042782845 2024-09-17T21:31:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538218+0000", 2024-09-17T21:31:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1335000000000002e-05 2024-09-17T21:31:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538626+0000", 2024-09-17T21:31:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000407656 2024-09-17T21:31:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50856 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 18087936~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.463106459000002, 2024-09-17T21:31:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043911444000000001, 2024-09-17T21:31:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50856 2024-09-17T21:31:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0499999999999999e-07 2024-09-17T21:31:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495392+0000", 2024-09-17T21:31:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7430000000000001e-06 2024-09-17T21:31:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495398+0000", 2024-09-17T21:31:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3040000000000001e-06 2024-09-17T21:31:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495403+0000", 2024-09-17T21:31:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5600000000000004e-06 2024-09-17T21:31:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538660+0000", 2024-09-17T21:31:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043256832000000002 2024-09-17T21:31:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538746+0000", 2024-09-17T21:31:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6448999999999999e-05 2024-09-17T21:31:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539301+0000", 2024-09-17T21:31:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055525099999999999 2024-09-17T21:31:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50857 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19660800~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.463087053999999, 2024-09-17T21:31:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044408156999999997, 2024-09-17T21:31:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50857 2024-09-17T21:31:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495410+0000", 2024-09-17T21:31:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:31:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495412+0000", 2024-09-17T21:31:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9580000000000001e-06 2024-09-17T21:31:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495418+0000", 2024-09-17T21:31:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1909999999999997e-06 2024-09-17T21:31:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495423+0000", 2024-09-17T21:31:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2429999999999996e-06 2024-09-17T21:31:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539338+0000", 2024-09-17T21:31:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043914478 2024-09-17T21:31:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539393+0000", 2024-09-17T21:31:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5226999999999998e-05 2024-09-17T21:31:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539818+0000", 2024-09-17T21:31:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042469300000000002 2024-09-17T21:31:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50858 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23330816~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.463065956999998, 2024-09-17T21:31:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045010798999999997, 2024-09-17T21:31:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50858 2024-09-17T21:31:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:31:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495433+0000", 2024-09-17T21:31:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.787e-06 2024-09-17T21:31:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495438+0000", 2024-09-17T21:31:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:31:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495442+0000", 2024-09-17T21:31:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2640000000000002e-06 2024-09-17T21:31:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539852+0000", 2024-09-17T21:31:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044409592999999997 2024-09-17T21:31:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539940+0000", 2024-09-17T21:31:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7696999999999997e-05 2024-09-17T21:31:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540441+0000", 2024-09-17T21:31:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050164299999999997 2024-09-17T21:31:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50859 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24903680~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.463048278999999, 2024-09-17T21:31:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045571569999999999, 2024-09-17T21:31:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50859 2024-09-17T21:31:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495449+0000", 2024-09-17T21:31:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495450+0000", 2024-09-17T21:31:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7889999999999999e-06 2024-09-17T21:31:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495456+0000", 2024-09-17T21:31:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5679999999999999e-06 2024-09-17T21:31:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495460+0000", 2024-09-17T21:31:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4989999999999996e-06 2024-09-17T21:31:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540472+0000", 2024-09-17T21:31:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045011585999999999 2024-09-17T21:31:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540554+0000", 2024-09-17T21:31:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2443999999999999e-05 2024-09-17T21:31:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541020+0000", 2024-09-17T21:31:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00046534300000000001 2024-09-17T21:31:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50860 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28573696~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.463029925999997, 2024-09-17T21:31:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046167899999999998, 2024-09-17T21:31:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50860 2024-09-17T21:31:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:31:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495469+0000", 2024-09-17T21:31:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.99e-06 2024-09-17T21:31:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495475+0000", 2024-09-17T21:31:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:31:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495480+0000", 2024-09-17T21:31:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:31:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541050+0000", 2024-09-17T21:31:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045569853 2024-09-17T21:31:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541138+0000", 2024-09-17T21:31:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7997000000000005e-05 2024-09-17T21:31:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541634+0000", 2024-09-17T21:31:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00049685500000000004 2024-09-17T21:31:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50861 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 30146560~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.463010748999999, 2024-09-17T21:31:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046690553000000003, 2024-09-17T21:31:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50861 2024-09-17T21:31:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:31:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495488+0000", 2024-09-17T21:31:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:31:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495495+0000", 2024-09-17T21:31:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:31:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495500+0000", 2024-09-17T21:31:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8620000000000002e-06 2024-09-17T21:31:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541687+0000", 2024-09-17T21:31:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046186434999999998 2024-09-17T21:31:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541741+0000", 2024-09-17T21:31:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4228999999999999e-05 2024-09-17T21:31:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.542176+0000", 2024-09-17T21:31:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043531100000000002 2024-09-17T21:31:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97804 3.7 3:e725aa05:::rbd_data.1120c318df8c.0000000000000003:24 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.997064648, 2024-09-17T21:31:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035514416, 2024-09-17T21:31:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97804 2024-09-17T21:31:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:31:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961434+0000", 2024-09-17T21:31:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8959999999999999e-06 2024-09-17T21:31:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961439+0000", 2024-09-17T21:31:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T21:31:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961444+0000", 2024-09-17T21:31:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4970000000000003e-06 2024-09-17T21:31:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996466+0000", 2024-09-17T21:31:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035022438000000003 2024-09-17T21:31:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996556+0000", 2024-09-17T21:31:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9561000000000005e-05 2024-09-17T21:31:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996946+0000", 2024-09-17T21:31:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039028700000000001 2024-09-17T21:31:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.051 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:00.051 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:00.051 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:00.052 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:31:00.053 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:00.049+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:00.060 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:00.060 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:00.077 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:00.077 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:00.093 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:00.093 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:00.109 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:00.109 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:00.125 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:00.125 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:00.143 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:00.143 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:00.153 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:00.151+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:00.159 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:00.159 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:00.163 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:00.163 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:00.172 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:31:00.175 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:00.176 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:00.254 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:00.252+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:00.275 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:00.275 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:00.284 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:31:00.356 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:00.353+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:00.387 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:00.387 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:00.396 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:31:00.457 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:00.455+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:00.499 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:00.499 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:00.508 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:31:00.558 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:00.556+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:00.611 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:00.611 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:00.619 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:31:00.660 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:00.658+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:00.760 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:00.757+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:00.763 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:00.763 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:00.771 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:31:00.861 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:00.859+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:00.876 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:00.876 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:00.884 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:31:00.962 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:00.959+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:00.999 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:00.999 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:01.006 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:31:01.008 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:31:01.063 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:01.060+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:01.114 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:01.114 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:01.123 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:31:01.164 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:01.161+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:01.225 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:01.226 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:01.234 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:31:01.278 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:01.276+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:01.337 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:01.338 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:01.346 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:31:01.387 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:01.385+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:01.468 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:01.466+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:01.490 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:01.491 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:01.499 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:31:01.569 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:01.567+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:01.603 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:01.603 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:01.611 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:31:01.695 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:01.693+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:01.755 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:01.755 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:01.763 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:31:01.771 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:01.769+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:01.871 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:01.869+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:01.906 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:01.907 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:01.915 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:31:01.946 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is unset 2024-09-17T21:31:01.972 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:01.970+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:02.038 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:02.039 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:02.041 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-17T21:31:02.041 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:31:02.041 INFO:tasks.thrashosds.thrasher:Setting full ratio to .001 2024-09-17T21:31:02.041 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .001 2024-09-17T21:31:02.048 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:31:02.089 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:02.087+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:02.175 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:02.173+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:02.187 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:31:02.210 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:02.210 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:02.219 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:31:02.276 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:02.274+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:02.312 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:02.312 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:02.333 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:02.333 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:02.349 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:02.349 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:02.367 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:02.367 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:02.370 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:02.370 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:02.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:02.370 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:02.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.370 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:37817 3.17 3:eab85a8d:::rbd_header.1120e62eea89:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:31:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:31:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.671404948999999, 2024-09-17T21:31:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86635970399999995, 2024-09-17T21:31:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37817 2024-09-17T21:31:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:31:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:31:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695457+0000", 2024-09-17T21:31:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5799999999999998e-07 2024-09-17T21:31:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695461+0000", 2024-09-17T21:31:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1110000000000001e-06 2024-09-17T21:31:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695471+0000", 2024-09-17T21:31:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0441e-05 2024-09-17T21:31:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695477+0000", 2024-09-17T21:31:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7599999999999999e-06 2024-09-17T21:31:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695533+0000", 2024-09-17T21:31:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6570999999999998e-05 2024-09-17T21:31:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:31:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695543+0000", 2024-09-17T21:31:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9979999999999998e-06 2024-09-17T21:31:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561472+0000", 2024-09-17T21:31:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86592841899999995 2024-09-17T21:31:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561610+0000", 2024-09-17T21:31:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013854699999999999 2024-09-17T21:31:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561816+0000", 2024-09-17T21:31:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020509900000000001 2024-09-17T21:31:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.376 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50202 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.376 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:31:02.376 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.259234159000002, 2024-09-17T21:31:02.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.076620261999999995, 2024-09-17T21:31:02.376 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.376 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.376 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:02.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.376 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50202 2024-09-17T21:31:02.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.377 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:31:02.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:31:02.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107629+0000", 2024-09-17T21:31:02.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3709999999999998e-06 2024-09-17T21:31:02.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119765+0000", 2024-09-17T21:31:02.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012135683 2024-09-17T21:31:02.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119787+0000", 2024-09-17T21:31:02.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2748999999999999e-05 2024-09-17T21:31:02.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119797+0000", 2024-09-17T21:31:02.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8169999999999994e-06 2024-09-17T21:31:02.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119872+0000", 2024-09-17T21:31:02.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270999999999994e-05 2024-09-17T21:31:02.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119987+0000", 2024-09-17T21:31:02.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011557699999999999 2024-09-17T21:31:02.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:02.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.121766+0000", 2024-09-17T21:31:02.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00177853 2024-09-17T21:31:02.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:02.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178307+0000", 2024-09-17T21:31:02.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056541559999999998 2024-09-17T21:31:02.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:02.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184097+0000", 2024-09-17T21:31:02.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057899029999999999 2024-09-17T21:31:02.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:02.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184172+0000", 2024-09-17T21:31:02.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5226000000000001e-05 2024-09-17T21:31:02.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184247+0000", 2024-09-17T21:31:02.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4574999999999997e-05 2024-09-17T21:31:02.381 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.381 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.381 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.381 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50203 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [truncate 20971520] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.381 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:31:02.382 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.247034253999999, 2024-09-17T21:31:02.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065009779000000004, 2024-09-17T21:31:02.382 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.382 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.382 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:02.382 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.382 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.382 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.382 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50203 2024-09-17T21:31:02.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.382 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:31:02.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:31:02.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119828+0000", 2024-09-17T21:31:02.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6390000000000001e-06 2024-09-17T21:31:02.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119832+0000", 2024-09-17T21:31:02.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7699999999999999e-06 2024-09-17T21:31:02.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119845+0000", 2024-09-17T21:31:02.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3093e-05 2024-09-17T21:31:02.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119852+0000", 2024-09-17T21:31:02.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6950000000000003e-06 2024-09-17T21:31:02.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.160945+0000", 2024-09-17T21:31:02.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041093076999999999 2024-09-17T21:31:02.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161071+0000", 2024-09-17T21:31:02.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012569799999999999 2024-09-17T21:31:02.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:02.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161283+0000", 2024-09-17T21:31:02.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021201999999999999 2024-09-17T21:31:02.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:02.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.179360+0000", 2024-09-17T21:31:02.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018077784999999999 2024-09-17T21:31:02.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:02.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184726+0000", 2024-09-17T21:31:02.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053652960000000003 2024-09-17T21:31:02.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:02.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184787+0000", 2024-09-17T21:31:02.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1402000000000006e-05 2024-09-17T21:31:02.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184836+0000", 2024-09-17T21:31:02.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9304000000000001e-05 2024-09-17T21:31:02.387 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.387 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.387 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.387 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50206 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.388 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:31:02.388 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.209952714000003, 2024-09-17T21:31:02.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.090361968000000001, 2024-09-17T21:31:02.388 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.388 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.388 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:02.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.388 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50206 2024-09-17T21:31:02.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.388 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:31:02.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:31:02.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156911+0000", 2024-09-17T21:31:02.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-17T21:31:02.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173670+0000", 2024-09-17T21:31:02.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016758907 2024-09-17T21:31:02.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173696+0000", 2024-09-17T21:31:02.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6329000000000002e-05 2024-09-17T21:31:02.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173708+0000", 2024-09-17T21:31:02.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2512e-05 2024-09-17T21:31:02.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173788+0000", 2024-09-17T21:31:02.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9332999999999994e-05 2024-09-17T21:31:02.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173918+0000", 2024-09-17T21:31:02.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013063300000000001 2024-09-17T21:31:02.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:02.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.176228+0000", 2024-09-17T21:31:02.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002309757 2024-09-17T21:31:02.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:02.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.245164+0000", 2024-09-17T21:31:02.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068936301000000005 2024-09-17T21:31:02.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:02.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247176+0000", 2024-09-17T21:31:02.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020117590000000001 2024-09-17T21:31:02.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:02.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247215+0000", 2024-09-17T21:31:02.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9094000000000002e-05 2024-09-17T21:31:02.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247270+0000", 2024-09-17T21:31:02.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4793999999999997e-05 2024-09-17T21:31:02.393 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.393 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.393 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.393 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50207 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [truncate 21556327] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.393 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:31:02.394 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.193103733999997, 2024-09-17T21:31:02.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.074055363999999999, 2024-09-17T21:31:02.394 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.394 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.394 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:02.394 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.394 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.394 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.394 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50207 2024-09-17T21:31:02.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.394 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:31:02.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:31:02.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173759+0000", 2024-09-17T21:31:02.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8160000000000001e-06 2024-09-17T21:31:02.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173763+0000", 2024-09-17T21:31:02.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5360000000000003e-06 2024-09-17T21:31:02.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173771+0000", 2024-09-17T21:31:02.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8849999999999999e-06 2024-09-17T21:31:02.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173777+0000", 2024-09-17T21:31:02.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7799999999999997e-06 2024-09-17T21:31:02.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221363+0000", 2024-09-17T21:31:02.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047585493 2024-09-17T21:31:02.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221430+0000", 2024-09-17T21:31:02.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7129000000000001e-05 2024-09-17T21:31:02.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:02.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221550+0000", 2024-09-17T21:31:02.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120658 2024-09-17T21:31:02.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:02.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246113+0000", 2024-09-17T21:31:02.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02456262 2024-09-17T21:31:02.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:02.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247744+0000", 2024-09-17T21:31:02.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001630692 2024-09-17T21:31:02.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:02.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247777+0000", 2024-09-17T21:31:02.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3015000000000002e-05 2024-09-17T21:31:02.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247812+0000", 2024-09-17T21:31:02.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.574e-05 2024-09-17T21:31:02.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.399 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.399 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50828 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.399 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:02.400 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.884991802000002, 2024-09-17T21:31:02.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047324185999999997, 2024-09-17T21:31:02.400 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.400 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.400 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.400 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50828 2024-09-17T21:31:02.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.400 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:02.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:02.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:02.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.46e-07 2024-09-17T21:31:02.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481871+0000", 2024-09-17T21:31:02.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T21:31:02.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481878+0000", 2024-09-17T21:31:02.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:31:02.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481882+0000", 2024-09-17T21:31:02.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:31:02.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529038+0000", 2024-09-17T21:31:02.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047156034999999999 2024-09-17T21:31:02.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529099+0000", 2024-09-17T21:31:02.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0316000000000003e-05 2024-09-17T21:31:02.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529193+0000", 2024-09-17T21:31:02.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4369000000000004e-05 2024-09-17T21:31:02.403 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.404 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.404 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.404 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50829 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 12845056~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.404 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:02.404 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.884973008000003, 2024-09-17T21:31:02.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047887596999999997, 2024-09-17T21:31:02.404 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.404 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.404 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.404 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.404 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.404 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.405 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50829 2024-09-17T21:31:02.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.405 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:02.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:02.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:02.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:31:02.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481890+0000", 2024-09-17T21:31:02.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.877e-06 2024-09-17T21:31:02.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481895+0000", 2024-09-17T21:31:02.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:31:02.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481900+0000", 2024-09-17T21:31:02.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4440000000000001e-06 2024-09-17T21:31:02.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529225+0000", 2024-09-17T21:31:02.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047325117999999999 2024-09-17T21:31:02.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529292+0000", 2024-09-17T21:31:02.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7100000000000005e-05 2024-09-17T21:31:02.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529775+0000", 2024-09-17T21:31:02.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048322799999999999 2024-09-17T21:31:02.408 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.408 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.408 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.408 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50830 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.408 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:31:02.408 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.884954606000001, 2024-09-17T21:31:02.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048062123999999998, 2024-09-17T21:31:02.409 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.409 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.409 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.409 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.409 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.409 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.409 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50830 2024-09-17T21:31:02.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.409 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:31:02.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:31:02.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481907+0000", 2024-09-17T21:31:02.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:31:02.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481909+0000", 2024-09-17T21:31:02.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T21:31:02.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481915+0000", 2024-09-17T21:31:02.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.02e-06 2024-09-17T21:31:02.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481920+0000", 2024-09-17T21:31:02.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6799999999999998e-06 2024-09-17T21:31:02.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529793+0000", 2024-09-17T21:31:02.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047873042999999997 2024-09-17T21:31:02.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529870+0000", 2024-09-17T21:31:02.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6967999999999999e-05 2024-09-17T21:31:02.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529968+0000", 2024-09-17T21:31:02.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8085999999999996e-05 2024-09-17T21:31:02.413 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.413 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.413 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.413 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50831 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 18087936~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.413 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:02.413 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.884934260000001, 2024-09-17T21:31:02.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048663304999999997, 2024-09-17T21:31:02.413 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.413 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.413 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.413 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.414 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50831 2024-09-17T21:31:02.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.414 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:02.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:02.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:02.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:31:02.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481929+0000", 2024-09-17T21:31:02.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2189999999999998e-06 2024-09-17T21:31:02.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481935+0000", 2024-09-17T21:31:02.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.383e-06 2024-09-17T21:31:02.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481939+0000", 2024-09-17T21:31:02.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2479999999999998e-06 2024-09-17T21:31:02.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530095+0000", 2024-09-17T21:31:02.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048156617999999998 2024-09-17T21:31:02.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530159+0000", 2024-09-17T21:31:02.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3394999999999999e-05 2024-09-17T21:31:02.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530590+0000", 2024-09-17T21:31:02.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043101199999999999 2024-09-17T21:31:02.417 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.417 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.417 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.417 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50832 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 19660800~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.417 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:31:02.418 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.884915612999997, 2024-09-17T21:31:02.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049339466999999998, 2024-09-17T21:31:02.418 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.418 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.418 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.418 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.418 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.418 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.418 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50832 2024-09-17T21:31:02.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.418 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:31:02.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:31:02.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481946+0000", 2024-09-17T21:31:02.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:02.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481951+0000", 2024-09-17T21:31:02.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1479999999999997e-06 2024-09-17T21:31:02.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481956+0000", 2024-09-17T21:31:02.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7339999999999996e-06 2024-09-17T21:31:02.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481961+0000", 2024-09-17T21:31:02.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4889999999999997e-06 2024-09-17T21:31:02.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530630+0000", 2024-09-17T21:31:02.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048668596000000001 2024-09-17T21:31:02.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530720+0000", 2024-09-17T21:31:02.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0163999999999999e-05 2024-09-17T21:31:02.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531285+0000", 2024-09-17T21:31:02.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056493199999999998 2024-09-17T21:31:02.421 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.421 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.421 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.422 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50833 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.422 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:31:02.422 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.884893468999998, 2024-09-17T21:31:02.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049908069999999999, 2024-09-17T21:31:02.422 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.422 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.422 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.422 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.422 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.422 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.422 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50833 2024-09-17T21:31:02.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.423 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:31:02.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:31:02.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481968+0000", 2024-09-17T21:31:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5600000000000001e-07 2024-09-17T21:31:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481970+0000", 2024-09-17T21:31:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:31:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481975+0000", 2024-09-17T21:31:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8499999999999999e-06 2024-09-17T21:31:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481980+0000", 2024-09-17T21:31:02.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-17T21:31:02.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531337+0000", 2024-09-17T21:31:02.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049356874000000002 2024-09-17T21:31:02.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531392+0000", 2024-09-17T21:31:02.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4826e-05 2024-09-17T21:31:02.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531875+0000", 2024-09-17T21:31:02.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048345099999999998 2024-09-17T21:31:02.426 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.426 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.426 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.426 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50834 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 24903680~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.426 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:31:02.426 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.884868701000002, 2024-09-17T21:31:02.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050534015000000002, 2024-09-17T21:31:02.426 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.427 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.427 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.427 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.427 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.427 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.427 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50834 2024-09-17T21:31:02.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.427 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:31:02.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:31:02.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481993+0000", 2024-09-17T21:31:02.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:31:02.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481995+0000", 2024-09-17T21:31:02.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.199e-06 2024-09-17T21:31:02.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482001+0000", 2024-09-17T21:31:02.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.772e-06 2024-09-17T21:31:02.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482006+0000", 2024-09-17T21:31:02.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0660000000000003e-06 2024-09-17T21:31:02.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531923+0000", 2024-09-17T21:31:02.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049917200000000002 2024-09-17T21:31:02.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532004+0000", 2024-09-17T21:31:02.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0758000000000002e-05 2024-09-17T21:31:02.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532526+0000", 2024-09-17T21:31:02.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00052258899999999995 2024-09-17T21:31:02.430 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.430 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.430 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.431 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50835 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 28573696~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.431 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:31:02.431 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.884848718000001, 2024-09-17T21:31:02.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051204701999999998, 2024-09-17T21:31:02.431 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.431 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.431 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.431 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.431 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.431 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.431 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50835 2024-09-17T21:31:02.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.431 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:31:02.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:31:02.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482013+0000", 2024-09-17T21:31:02.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:31:02.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482015+0000", 2024-09-17T21:31:02.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.272e-06 2024-09-17T21:31:02.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482020+0000", 2024-09-17T21:31:02.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3040000000000004e-06 2024-09-17T21:31:02.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482025+0000", 2024-09-17T21:31:02.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5650000000000003e-06 2024-09-17T21:31:02.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532569+0000", 2024-09-17T21:31:02.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050544661999999997 2024-09-17T21:31:02.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532649+0000", 2024-09-17T21:31:02.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9640000000000003e-05 2024-09-17T21:31:02.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533217+0000", 2024-09-17T21:31:02.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056784100000000003 2024-09-17T21:31:02.435 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.435 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.435 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.435 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50836 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 30146560~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.435 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:02.435 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.884829101000001, 2024-09-17T21:31:02.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051875589, 2024-09-17T21:31:02.435 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.435 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.435 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.435 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.436 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.436 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.436 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50836 2024-09-17T21:31:02.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.436 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:02.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:02.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:02.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3700000000000001e-07 2024-09-17T21:31:02.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482034+0000", 2024-09-17T21:31:02.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0059999999999999e-06 2024-09-17T21:31:02.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482040+0000", 2024-09-17T21:31:02.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:31:02.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482044+0000", 2024-09-17T21:31:02.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:31:02.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533260+0000", 2024-09-17T21:31:02.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051215416 2024-09-17T21:31:02.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533348+0000", 2024-09-17T21:31:02.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8695000000000004e-05 2024-09-17T21:31:02.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533907+0000", 2024-09-17T21:31:02.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055902199999999995 2024-09-17T21:31:02.439 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.439 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.439 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.440 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50844 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.440 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:31:02.440 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.884665675000001, 2024-09-17T21:31:02.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0475644, 2024-09-17T21:31:02.440 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.440 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.440 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.440 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.440 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.440 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.440 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50844 2024-09-17T21:31:02.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.440 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:31:02.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:31:02.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482196+0000", 2024-09-17T21:31:02.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.39e-07 2024-09-17T21:31:02.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482198+0000", 2024-09-17T21:31:02.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0099999999999998e-06 2024-09-17T21:31:02.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482204+0000", 2024-09-17T21:31:02.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.93e-06 2024-09-17T21:31:02.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482208+0000", 2024-09-17T21:31:02.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7439999999999998e-06 2024-09-17T21:31:02.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529318+0000", 2024-09-17T21:31:02.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047110051999999999 2024-09-17T21:31:02.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529396+0000", 2024-09-17T21:31:02.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7712000000000004e-05 2024-09-17T21:31:02.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529760+0000", 2024-09-17T21:31:02.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00036351299999999999 2024-09-17T21:31:02.444 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.444 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.444 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.444 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50845 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 21757952~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.444 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:31:02.444 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.884646572000001, 2024-09-17T21:31:02.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048084625999999998, 2024-09-17T21:31:02.444 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.444 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.444 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.444 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.444 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.445 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.445 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50845 2024-09-17T21:31:02.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.445 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:31:02.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:31:02.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482215+0000", 2024-09-17T21:31:02.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:31:02.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482216+0000", 2024-09-17T21:31:02.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.823e-06 2024-09-17T21:31:02.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482222+0000", 2024-09-17T21:31:02.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.507e-06 2024-09-17T21:31:02.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482227+0000", 2024-09-17T21:31:02.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6609999999999999e-06 2024-09-17T21:31:02.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529792+0000", 2024-09-17T21:31:02.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047565075999999998 2024-09-17T21:31:02.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529871+0000", 2024-09-17T21:31:02.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8855000000000003e-05 2024-09-17T21:31:02.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530299+0000", 2024-09-17T21:31:02.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000428307 2024-09-17T21:31:02.448 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.448 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.448 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.448 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50846 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 25427968~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.448 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:02.448 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.884624185, 2024-09-17T21:31:02.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048735142000000002, 2024-09-17T21:31:02.448 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.449 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.449 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.449 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.449 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.449 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.449 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50846 2024-09-17T21:31:02.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.449 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:02.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:02.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:02.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:31:02.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482239+0000", 2024-09-17T21:31:02.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.871e-06 2024-09-17T21:31:02.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482244+0000", 2024-09-17T21:31:02.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.49e-06 2024-09-17T21:31:02.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482249+0000", 2024-09-17T21:31:02.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6269999999999999e-06 2024-09-17T21:31:02.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530347+0000", 2024-09-17T21:31:02.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048097721000000003 2024-09-17T21:31:02.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530437+0000", 2024-09-17T21:31:02.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0604000000000002e-05 2024-09-17T21:31:02.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530972+0000", 2024-09-17T21:31:02.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053445000000000001 2024-09-17T21:31:02.452 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.452 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.452 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.453 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50847 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 27000832~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.453 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:02.453 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.884606288999997, 2024-09-17T21:31:02.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049308668, 2024-09-17T21:31:02.453 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.453 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.453 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.453 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.453 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.453 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.453 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50847 2024-09-17T21:31:02.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.453 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:02.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:02.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:02.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:31:02.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482257+0000", 2024-09-17T21:31:02.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.846e-06 2024-09-17T21:31:02.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482264+0000", 2024-09-17T21:31:02.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8170000000000001e-06 2024-09-17T21:31:02.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482283+0000", 2024-09-17T21:31:02.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9718e-05 2024-09-17T21:31:02.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531018+0000", 2024-09-17T21:31:02.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048734497000000002 2024-09-17T21:31:02.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531080+0000", 2024-09-17T21:31:02.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1795e-05 2024-09-17T21:31:02.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531563+0000", 2024-09-17T21:31:02.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048366499999999998 2024-09-17T21:31:02.457 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.457 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.457 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.457 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50848 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 30670848~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.457 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:31:02.457 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.884569747, 2024-09-17T21:31:02.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049893337000000003, 2024-09-17T21:31:02.458 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.458 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.458 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.458 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.458 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.458 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.458 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50848 2024-09-17T21:31:02.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.458 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:31:02.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:31:02.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482292+0000", 2024-09-17T21:31:02.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T21:31:02.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482294+0000", 2024-09-17T21:31:02.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3209999999999999e-06 2024-09-17T21:31:02.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482300+0000", 2024-09-17T21:31:02.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7189999999999998e-06 2024-09-17T21:31:02.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482305+0000", 2024-09-17T21:31:02.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T21:31:02.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531606+0000", 2024-09-17T21:31:02.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049301273999999999 2024-09-17T21:31:02.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531696+0000", 2024-09-17T21:31:02.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0161999999999995e-05 2024-09-17T21:31:02.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532184+0000", 2024-09-17T21:31:02.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048806400000000002 2024-09-17T21:31:02.461 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.461 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.462 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.462 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50849 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 32243712~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:02.462 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:31:02.462 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.884550306000001, 2024-09-17T21:31:02.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050444607000000002, 2024-09-17T21:31:02.462 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.462 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.462 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.462 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.462 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.462 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.462 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50849 2024-09-17T21:31:02.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.463 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:31:02.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:31:02.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482311+0000", 2024-09-17T21:31:02.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:02.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482316+0000", 2024-09-17T21:31:02.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1209999999999998e-06 2024-09-17T21:31:02.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482321+0000", 2024-09-17T21:31:02.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4779999999999999e-06 2024-09-17T21:31:02.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482326+0000", 2024-09-17T21:31:02.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7099999999999998e-06 2024-09-17T21:31:02.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532221+0000", 2024-09-17T21:31:02.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049894936000000001 2024-09-17T21:31:02.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532287+0000", 2024-09-17T21:31:02.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5914000000000006e-05 2024-09-17T21:31:02.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532755+0000", 2024-09-17T21:31:02.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000468107 2024-09-17T21:31:02.466 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.466 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.466 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.466 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.466 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.466 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:02.468 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:31:02.468 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:02.468 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:02.469 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:02.469 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:02.469 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:02.470 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:02.470 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:02.470 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:02.471 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:02.465+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:02.568 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:02.566+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:02.611 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:02.611 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:02.611 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:02.611 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:02.619 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:31:02.703 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:02.701+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:02.761 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:02.762 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:02.770 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:31:02.771 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:02.768+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:02.871 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:02.869+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:02.916 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:02.916 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:02.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:02.916 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:02.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.916 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "pg_update_log_missing(3.5 epoch 45/37 rep_tid 844 entries 45'73 (0'0) error 3:a65c40ae:::rbd_data.1120b21c3309.0000000000000126:head by client.4384.0:48900 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 45'70)", 2024-09-17T21:31:02.917 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:31:02.917 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.866176154000001, 2024-09-17T21:31:02.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019554365000000001, 2024-09-17T21:31:02.917 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.917 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.917 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T21:31:02.917 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:31:02.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:31:02.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047112+0000", 2024-09-17T21:31:02.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:31:02.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047121+0000", 2024-09-17T21:31:02.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4379999999999995e-06 2024-09-17T21:31:02.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047139+0000", 2024-09-17T21:31:02.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8235000000000001e-05 2024-09-17T21:31:02.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047303+0000", 2024-09-17T21:31:02.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016389400000000001 2024-09-17T21:31:02.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047396+0000", 2024-09-17T21:31:02.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3355000000000003e-05 2024-09-17T21:31:02.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066666+0000", 2024-09-17T21:31:02.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019269649999999999 2024-09-17T21:31:02.920 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.920 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.920 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.920 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48906 3.4 e45/25 3:2503fe27:::rbd_data.1120b21c3309.0000000000000127:head v 45'148, mlcod=45'146)", 2024-09-17T21:31:02.920 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:02.921 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.864049975, 2024-09-17T21:31:02.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018169739000000001, 2024-09-17T21:31:02.921 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.921 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.921 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:02.921 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:02.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:02.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:02.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:31:02.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049244+0000", 2024-09-17T21:31:02.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8390000000000003e-06 2024-09-17T21:31:02.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066411+0000", 2024-09-17T21:31:02.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017166818 2024-09-17T21:31:02.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066426+0000", 2024-09-17T21:31:02.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4888999999999999e-05 2024-09-17T21:31:02.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066611+0000", 2024-09-17T21:31:02.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018464499999999999 2024-09-17T21:31:02.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066630+0000", 2024-09-17T21:31:02.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9201000000000001e-05 2024-09-17T21:31:02.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:02.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067373+0000", 2024-09-17T21:31:02.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00074322600000000004 2024-09-17T21:31:02.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067407+0000", 2024-09-17T21:31:02.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4357999999999999e-05 2024-09-17T21:31:02.926 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.926 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.926 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.926 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50206 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'301, mlcod=45'299)", 2024-09-17T21:31:02.926 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:31:02.926 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.734573130999998, 2024-09-17T21:31:02.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068285810000000002, 2024-09-17T21:31:02.926 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.926 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.926 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:02.927 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:31:02.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:31:02.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178716+0000", 2024-09-17T21:31:02.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.438e-06 2024-09-17T21:31:02.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192044+0000", 2024-09-17T21:31:02.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013328370000000001 2024-09-17T21:31:02.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192069+0000", 2024-09-17T21:31:02.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4496000000000001e-05 2024-09-17T21:31:02.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192085+0000", 2024-09-17T21:31:02.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5758999999999999e-05 2024-09-17T21:31:02.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192167+0000", 2024-09-17T21:31:02.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2842000000000004e-05 2024-09-17T21:31:02.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192201+0000", 2024-09-17T21:31:02.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3127999999999999e-05 2024-09-17T21:31:02.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:02.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246938+0000", 2024-09-17T21:31:02.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054736982000000003 2024-09-17T21:31:02.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247000+0000", 2024-09-17T21:31:02.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2794999999999998e-05 2024-09-17T21:31:02.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.930 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.931 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.931 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50207 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'303, mlcod=45'299)", 2024-09-17T21:31:02.931 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:31:02.931 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.691626081999999, 2024-09-17T21:31:02.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026003433999999999, 2024-09-17T21:31:02.931 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.931 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.931 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:02.931 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:31:02.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:31:02.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221663+0000", 2024-09-17T21:31:02.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.271e-06 2024-09-17T21:31:02.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221672+0000", 2024-09-17T21:31:02.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6810000000000004e-06 2024-09-17T21:31:02.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221684+0000", 2024-09-17T21:31:02.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:31:02.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221691+0000", 2024-09-17T21:31:02.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:31:02.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221743+0000", 2024-09-17T21:31:02.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2868999999999998e-05 2024-09-17T21:31:02.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221765+0000", 2024-09-17T21:31:02.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1316999999999999e-05 2024-09-17T21:31:02.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:02.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247616+0000", 2024-09-17T21:31:02.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025851504000000001 2024-09-17T21:31:02.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247665+0000", 2024-09-17T21:31:02.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8739000000000002e-05 2024-09-17T21:31:02.936 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.936 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.936 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.936 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57808 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 196608~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:02.936 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:02.936 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.135327042, 2024-09-17T21:31:02.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017751475999999999, 2024-09-17T21:31:02.937 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.937 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.937 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.937 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57808 2024-09-17T21:31:02.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.937 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:02.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:02.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:02.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:31:02.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777968+0000", 2024-09-17T21:31:02.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8530000000000003e-06 2024-09-17T21:31:02.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777975+0000", 2024-09-17T21:31:02.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4410000000000004e-06 2024-09-17T21:31:02.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777980+0000", 2024-09-17T21:31:02.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7210000000000002e-06 2024-09-17T21:31:02.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795323+0000", 2024-09-17T21:31:02.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017343319999999999 2024-09-17T21:31:02.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795410+0000", 2024-09-17T21:31:02.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6567e-05 2024-09-17T21:31:02.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795712+0000", 2024-09-17T21:31:02.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030219200000000001 2024-09-17T21:31:02.941 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.941 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.941 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.941 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57809 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 458752~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:02.941 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:31:02.942 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.13530119, 2024-09-17T21:31:02.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018092869000000001, 2024-09-17T21:31:02.942 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.942 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.942 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.942 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57809 2024-09-17T21:31:02.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.942 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:31:02.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:31:02.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777987+0000", 2024-09-17T21:31:02.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T21:31:02.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777989+0000", 2024-09-17T21:31:02.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8649999999999999e-06 2024-09-17T21:31:02.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777994+0000", 2024-09-17T21:31:02.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4199999999999998e-06 2024-09-17T21:31:02.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778004+0000", 2024-09-17T21:31:02.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6900000000000004e-06 2024-09-17T21:31:02.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795745+0000", 2024-09-17T21:31:02.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017740859000000001 2024-09-17T21:31:02.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795815+0000", 2024-09-17T21:31:02.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0219e-05 2024-09-17T21:31:02.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796079+0000", 2024-09-17T21:31:02.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000264446 2024-09-17T21:31:02.946 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.946 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.946 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.946 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57810 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 720896~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:02.946 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:31:02.946 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.135277463000001, 2024-09-17T21:31:02.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018361253000000001, 2024-09-17T21:31:02.947 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.947 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.947 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.947 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.947 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.947 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.947 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57810 2024-09-17T21:31:02.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.947 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:31:02.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:31:02.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778011+0000", 2024-09-17T21:31:02.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:31:02.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778012+0000", 2024-09-17T21:31:02.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.872e-06 2024-09-17T21:31:02.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778018+0000", 2024-09-17T21:31:02.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8150000000000002e-06 2024-09-17T21:31:02.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778023+0000", 2024-09-17T21:31:02.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3939999999999998e-06 2024-09-17T21:31:02.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796106+0000", 2024-09-17T21:31:02.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018083446999999999 2024-09-17T21:31:02.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796159+0000", 2024-09-17T21:31:02.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3029999999999999e-05 2024-09-17T21:31:02.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796371+0000", 2024-09-17T21:31:02.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021231700000000001 2024-09-17T21:31:02.950 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.950 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.950 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.951 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57811 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:02.951 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:02.951 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.135259042999998, 2024-09-17T21:31:02.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018666035000000001, 2024-09-17T21:31:02.951 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.951 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.951 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.951 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57811 2024-09-17T21:31:02.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.951 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:02.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:02.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:02.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:02.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778031+0000", 2024-09-17T21:31:02.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.796e-06 2024-09-17T21:31:02.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778036+0000", 2024-09-17T21:31:02.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3290000000000001e-06 2024-09-17T21:31:02.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778041+0000", 2024-09-17T21:31:02.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5839999999999996e-06 2024-09-17T21:31:02.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796409+0000", 2024-09-17T21:31:02.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018368703 2024-09-17T21:31:02.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796483+0000", 2024-09-17T21:31:02.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3751999999999995e-05 2024-09-17T21:31:02.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796695+0000", 2024-09-17T21:31:02.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021153 2024-09-17T21:31:02.955 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.955 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.955 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.955 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57812 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1245184~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:02.955 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:02.955 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.135240146000001, 2024-09-17T21:31:02.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019000064000000001, 2024-09-17T21:31:02.955 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.955 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.955 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57812 2024-09-17T21:31:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:02.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:31:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778050+0000", 2024-09-17T21:31:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7e-06 2024-09-17T21:31:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778055+0000", 2024-09-17T21:31:02.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4049999999999999e-06 2024-09-17T21:31:02.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778059+0000", 2024-09-17T21:31:02.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1479999999999999e-06 2024-09-17T21:31:02.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796734+0000", 2024-09-17T21:31:02.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018674399000000001 2024-09-17T21:31:02.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796786+0000", 2024-09-17T21:31:02.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2879e-05 2024-09-17T21:31:02.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797048+0000", 2024-09-17T21:31:02.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026118400000000003 2024-09-17T21:31:02.959 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.959 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.959 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.960 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57813 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1507328~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:02.960 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:31:02.960 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.135222253999999, 2024-09-17T21:31:02.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019279392999999999, 2024-09-17T21:31:02.960 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.960 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.960 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.960 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.960 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.960 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.960 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57813 2024-09-17T21:31:02.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.960 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:31:02.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:31:02.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778066+0000", 2024-09-17T21:31:02.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6199999999999999e-07 2024-09-17T21:31:02.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778068+0000", 2024-09-17T21:31:02.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7990000000000001e-06 2024-09-17T21:31:02.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778074+0000", 2024-09-17T21:31:02.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:31:02.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778078+0000", 2024-09-17T21:31:02.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2579999999999997e-06 2024-09-17T21:31:02.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797077+0000", 2024-09-17T21:31:02.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018999180000000001 2024-09-17T21:31:02.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797123+0000", 2024-09-17T21:31:02.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5834999999999997e-05 2024-09-17T21:31:02.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797345+0000", 2024-09-17T21:31:02.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022191100000000001 2024-09-17T21:31:02.964 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.964 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.964 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.964 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57814 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:02.964 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:31:02.964 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.135203382, 2024-09-17T21:31:02.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019604947000000001, 2024-09-17T21:31:02.965 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.965 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.965 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.965 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.965 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.965 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.965 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57814 2024-09-17T21:31:02.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.965 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:31:02.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:31:02.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778085+0000", 2024-09-17T21:31:02.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:31:02.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778086+0000", 2024-09-17T21:31:02.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7850000000000001e-06 2024-09-17T21:31:02.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778093+0000", 2024-09-17T21:31:02.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6449999999999999e-06 2024-09-17T21:31:02.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778098+0000", 2024-09-17T21:31:02.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:31:02.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797383+0000", 2024-09-17T21:31:02.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019284879000000001 2024-09-17T21:31:02.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797431+0000", 2024-09-17T21:31:02.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8856000000000001e-05 2024-09-17T21:31:02.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797689+0000", 2024-09-17T21:31:02.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025785800000000002 2024-09-17T21:31:02.969 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.969 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.969 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.969 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57815 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2031616~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:02.969 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:02.970 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.135182714000003, 2024-09-17T21:31:02.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019910231, 2024-09-17T21:31:02.970 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.970 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.970 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.970 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.971 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.971 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.972 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57815 2024-09-17T21:31:02.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.972 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:02.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:02.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:02.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:31:02.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778118+0000", 2024-09-17T21:31:02.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2967e-05 2024-09-17T21:31:02.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778125+0000", 2024-09-17T21:31:02.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.601e-06 2024-09-17T21:31:02.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778130+0000", 2024-09-17T21:31:02.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1839999999999998e-06 2024-09-17T21:31:02.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797720+0000", 2024-09-17T21:31:02.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019589766000000002 2024-09-17T21:31:02.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797765+0000", 2024-09-17T21:31:02.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5281999999999998e-05 2024-09-17T21:31:02.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798015+0000", 2024-09-17T21:31:02.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025006200000000001 2024-09-17T21:31:02.975 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.975 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.975 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.975 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57816 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2293760~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:02.976 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:02.976 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.135150119999999, 2024-09-17T21:31:02.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020175322999999998, 2024-09-17T21:31:02.976 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.976 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.976 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.976 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.976 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.977 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57816 2024-09-17T21:31:02.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.977 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:02.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:02.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:02.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:31:02.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778140+0000", 2024-09-17T21:31:02.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T21:31:02.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778145+0000", 2024-09-17T21:31:02.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7320000000000003e-06 2024-09-17T21:31:02.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778154+0000", 2024-09-17T21:31:02.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6969999999999999e-06 2024-09-17T21:31:02.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798049+0000", 2024-09-17T21:31:02.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019894603 2024-09-17T21:31:02.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798106+0000", 2024-09-17T21:31:02.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7207000000000003e-05 2024-09-17T21:31:02.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798313+0000", 2024-09-17T21:31:02.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206861 2024-09-17T21:31:02.980 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.981 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.981 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57817 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2555904~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:02.981 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:02.981 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.135126874000001, 2024-09-17T21:31:02.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020466314999999999, 2024-09-17T21:31:02.981 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.981 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.982 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.982 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57817 2024-09-17T21:31:02.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.982 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:02.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:02.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:02.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2399999999999999e-07 2024-09-17T21:31:02.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778163+0000", 2024-09-17T21:31:02.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.204e-06 2024-09-17T21:31:02.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778169+0000", 2024-09-17T21:31:02.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:31:02.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778173+0000", 2024-09-17T21:31:02.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:31:02.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798342+0000", 2024-09-17T21:31:02.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020168416000000002 2024-09-17T21:31:02.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798388+0000", 2024-09-17T21:31:02.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6171000000000002e-05 2024-09-17T21:31:02.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798627+0000", 2024-09-17T21:31:02.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000239058 2024-09-17T21:31:02.986 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.986 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.986 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.986 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57818 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2818048~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:02.986 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:31:02.986 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.135108176999999, 2024-09-17T21:31:02.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020767240999999999, 2024-09-17T21:31:02.986 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.987 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.987 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.987 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.987 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.987 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.987 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57818 2024-09-17T21:31:02.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.987 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:31:02.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:31:02.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778180+0000", 2024-09-17T21:31:02.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:31:02.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778182+0000", 2024-09-17T21:31:02.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7269999999999999e-06 2024-09-17T21:31:02.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778188+0000", 2024-09-17T21:31:02.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6080000000000001e-06 2024-09-17T21:31:02.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778202+0000", 2024-09-17T21:31:02.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3369e-05 2024-09-17T21:31:02.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798660+0000", 2024-09-17T21:31:02.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020458171000000001 2024-09-17T21:31:02.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798704+0000", 2024-09-17T21:31:02.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4273000000000001e-05 2024-09-17T21:31:02.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798947+0000", 2024-09-17T21:31:02.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242651 2024-09-17T21:31:02.991 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.991 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.991 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.991 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57819 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:02.991 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:31:02.991 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.135077308, 2024-09-17T21:31:02.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020892340999999998, 2024-09-17T21:31:02.991 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.991 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.991 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.992 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.992 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.992 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57819 2024-09-17T21:31:02.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.992 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:31:02.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:31:02.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778211+0000", 2024-09-17T21:31:02.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:31:02.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778217+0000", 2024-09-17T21:31:02.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T21:31:02.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778224+0000", 2024-09-17T21:31:02.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3520000000000003e-06 2024-09-17T21:31:02.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778228+0000", 2024-09-17T21:31:02.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7480000000000001e-06 2024-09-17T21:31:02.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:02.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798977+0000", 2024-09-17T21:31:02.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020748571 2024-09-17T21:31:02.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:02.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799035+0000", 2024-09-17T21:31:02.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7908999999999997e-05 2024-09-17T21:31:02.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:02.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799103+0000", 2024-09-17T21:31:02.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7794999999999997e-05 2024-09-17T21:31:02.996 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.996 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:02.996 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:02.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.996 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57820 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:02.996 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:02.996 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.135052115999997, 2024-09-17T21:31:02.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021016658000000001, 2024-09-17T21:31:02.996 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:02.996 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:02.996 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:02.996 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:02.997 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:02.997 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:02.997 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57820 2024-09-17T21:31:02.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.997 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:02.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:02.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:02.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:02.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:02.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:02.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:02.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:02.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:31:02.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:02.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778238+0000", 2024-09-17T21:31:02.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.945e-06 2024-09-17T21:31:02.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:02.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778244+0000", 2024-09-17T21:31:02.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6520000000000003e-06 2024-09-17T21:31:02.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:02.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:02.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778248+0000", 2024-09-17T21:31:02.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5920000000000002e-06 2024-09-17T21:31:02.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:02.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799131+0000", 2024-09-17T21:31:03.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020883288999999999 2024-09-17T21:31:03.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799185+0000", 2024-09-17T21:31:03.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3223000000000002e-05 2024-09-17T21:31:03.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799252+0000", 2024-09-17T21:31:03.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7576999999999994e-05 2024-09-17T21:31:03.001 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.001 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.001 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.001 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57821 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:03.001 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:03.001 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.135033771000003, 2024-09-17T21:31:03.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021153528000000001, 2024-09-17T21:31:03.001 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.001 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.001 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.002 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.002 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.002 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.002 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57821 2024-09-17T21:31:03.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.002 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:03.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:03.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:03.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-07 2024-09-17T21:31:03.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778256+0000", 2024-09-17T21:31:03.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.604e-06 2024-09-17T21:31:03.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778262+0000", 2024-09-17T21:31:03.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7629999999999997e-06 2024-09-17T21:31:03.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778266+0000", 2024-09-17T21:31:03.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:31:03.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799284+0000", 2024-09-17T21:31:03.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021017312 2024-09-17T21:31:03.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799326+0000", 2024-09-17T21:31:03.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2639000000000003e-05 2024-09-17T21:31:03.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799407+0000", 2024-09-17T21:31:03.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1077999999999999e-05 2024-09-17T21:31:03.006 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.006 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.006 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.006 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57822 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:03.006 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:31:03.006 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.135015461000002, 2024-09-17T21:31:03.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021268966, 2024-09-17T21:31:03.006 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.006 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.006 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.006 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.006 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.007 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.007 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57822 2024-09-17T21:31:03.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.007 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:31:03.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:31:03.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778273+0000", 2024-09-17T21:31:03.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:31:03.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778274+0000", 2024-09-17T21:31:03.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.598e-06 2024-09-17T21:31:03.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778283+0000", 2024-09-17T21:31:03.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8279999999999992e-06 2024-09-17T21:31:03.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778289+0000", 2024-09-17T21:31:03.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5609999999999998e-06 2024-09-17T21:31:03.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799432+0000", 2024-09-17T21:31:03.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021143862999999999 2024-09-17T21:31:03.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799473+0000", 2024-09-17T21:31:03.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1075999999999998e-05 2024-09-17T21:31:03.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799541+0000", 2024-09-17T21:31:03.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7657999999999995e-05 2024-09-17T21:31:03.010 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.010 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.010 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.011 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57823 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:03.011 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:03.011 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.134992500000003, 2024-09-17T21:31:03.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021379770999999999, 2024-09-17T21:31:03.011 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.011 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.011 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.011 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.011 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.011 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.012 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57823 2024-09-17T21:31:03.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.012 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:03.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:03.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:03.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:31:03.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778297+0000", 2024-09-17T21:31:03.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:31:03.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778303+0000", 2024-09-17T21:31:03.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4559999999999999e-06 2024-09-17T21:31:03.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778307+0000", 2024-09-17T21:31:03.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6229999999999996e-06 2024-09-17T21:31:03.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799560+0000", 2024-09-17T21:31:03.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021252931999999999 2024-09-17T21:31:03.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799609+0000", 2024-09-17T21:31:03.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8301999999999999e-05 2024-09-17T21:31:03.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799675+0000", 2024-09-17T21:31:03.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6205999999999996e-05 2024-09-17T21:31:03.016 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.016 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.016 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.016 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.016 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.016 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:03.018 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:31:03.018 INFO:tasks.ceph.mon.a.smithi129.stderr:2024-09-17T21:31:02.944+0000 7f70139b4640 -1 log_channel(cluster) log [ERR] : Health check failed: full ratio(s) out of order (OSD_OUT_OF_ORDER_FULL) 2024-09-17T21:31:03.019 INFO:teuthology.orchestra.run.smithi129.stderr:osd set-full-ratio 0.001 2024-09-17T21:31:03.019 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:03.015+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:03.118 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:03.116+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:03.164 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:03.164 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:03.164 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:03.164 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:03.172 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:31:03.219 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:03.217+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:03.315 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:03.316 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:03.320 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:03.317+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:03.324 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:31:03.421 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:03.418+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:03.470 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:03.470 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:03.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:03.470 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:03.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.470 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:49361 3.19 e46/25 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:head v 46'135, mlcod=45'133)", 2024-09-17T21:31:03.470 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:03.471 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.713839243000002, 2024-09-17T21:31:03.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043477744999999998, 2024-09-17T21:31:03.471 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.471 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.471 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:03.471 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:03.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:03.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:03.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-07 2024-09-17T21:31:03.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769703+0000", 2024-09-17T21:31:03.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016134381 2024-09-17T21:31:03.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769724+0000", 2024-09-17T21:31:03.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0781999999999999e-05 2024-09-17T21:31:03.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769734+0000", 2024-09-17T21:31:03.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6870000000000006e-06 2024-09-17T21:31:03.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769797+0000", 2024-09-17T21:31:03.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3418999999999997e-05 2024-09-17T21:31:03.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769822+0000", 2024-09-17T21:31:03.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4652999999999999e-05 2024-09-17T21:31:03.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:03.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.796982+0000", 2024-09-17T21:31:03.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027160205 2024-09-17T21:31:03.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.797047+0000", 2024-09-17T21:31:03.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4455999999999994e-05 2024-09-17T21:31:03.475 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.475 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.475 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.475 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50805 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 17825792~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:03.476 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:31:03.476 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.972255748999999, 2024-09-17T21:31:03.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035591579999999998, 2024-09-17T21:31:03.476 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.476 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.476 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.476 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.476 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.476 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.476 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50805 2024-09-17T21:31:03.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.476 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:31:03.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:31:03.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495153+0000", 2024-09-17T21:31:03.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:31:03.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495155+0000", 2024-09-17T21:31:03.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1849999999999998e-06 2024-09-17T21:31:03.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495162+0000", 2024-09-17T21:31:03.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.64e-06 2024-09-17T21:31:03.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495166+0000", 2024-09-17T21:31:03.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:31:03.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530020+0000", 2024-09-17T21:31:03.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.034853824999999998 2024-09-17T21:31:03.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530076+0000", 2024-09-17T21:31:03.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5235999999999998e-05 2024-09-17T21:31:03.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530744+0000", 2024-09-17T21:31:03.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066835500000000003 2024-09-17T21:31:03.480 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.480 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.480 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.480 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50806 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19398656~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:03.480 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:03.480 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.972235281000003, 2024-09-17T21:31:03.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036408459999999997, 2024-09-17T21:31:03.480 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.480 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.480 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.481 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50806 2024-09-17T21:31:03.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.481 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:03.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:03.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:03.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.72e-07 2024-09-17T21:31:03.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495175+0000", 2024-09-17T21:31:03.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:31:03.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495182+0000", 2024-09-17T21:31:03.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8079999999999999e-06 2024-09-17T21:31:03.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495186+0000", 2024-09-17T21:31:03.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5549999999999996e-06 2024-09-17T21:31:03.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530791+0000", 2024-09-17T21:31:03.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035604963000000003 2024-09-17T21:31:03.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530855+0000", 2024-09-17T21:31:03.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3286000000000004e-05 2024-09-17T21:31:03.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531581+0000", 2024-09-17T21:31:03.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00072655699999999996 2024-09-17T21:31:03.484 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.484 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.484 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.485 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50807 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23068672~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:03.485 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:03.485 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.972216396, 2024-09-17T21:31:03.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037183314000000002, 2024-09-17T21:31:03.485 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.485 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.485 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.485 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.485 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.485 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.485 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50807 2024-09-17T21:31:03.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.485 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:03.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:03.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:03.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9999999999999999e-07 2024-09-17T21:31:03.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495194+0000", 2024-09-17T21:31:03.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.779e-06 2024-09-17T21:31:03.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495199+0000", 2024-09-17T21:31:03.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.468e-06 2024-09-17T21:31:03.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495203+0000", 2024-09-17T21:31:03.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0010000000000003e-06 2024-09-17T21:31:03.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531636+0000", 2024-09-17T21:31:03.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036432665000000003 2024-09-17T21:31:03.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531727+0000", 2024-09-17T21:31:03.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1607999999999994e-05 2024-09-17T21:31:03.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532375+0000", 2024-09-17T21:31:03.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00064749299999999996 2024-09-17T21:31:03.489 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.489 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.489 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.490 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50808 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24641536~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:03.490 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:31:03.490 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.972199652999997, 2024-09-17T21:31:03.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037870190999999997, 2024-09-17T21:31:03.490 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.490 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.490 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.490 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.490 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.490 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.490 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50808 2024-09-17T21:31:03.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.490 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:31:03.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:31:03.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495209+0000", 2024-09-17T21:31:03.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:31:03.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495215+0000", 2024-09-17T21:31:03.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1630000000000001e-06 2024-09-17T21:31:03.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495221+0000", 2024-09-17T21:31:03.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-17T21:31:03.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495226+0000", 2024-09-17T21:31:03.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8189999999999999e-06 2024-09-17T21:31:03.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532425+0000", 2024-09-17T21:31:03.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037199135000000001 2024-09-17T21:31:03.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532531+0000", 2024-09-17T21:31:03.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106314 2024-09-17T21:31:03.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533079+0000", 2024-09-17T21:31:03.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00054739700000000003 2024-09-17T21:31:03.494 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.494 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.494 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.494 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50809 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28311552~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:03.494 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:31:03.494 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.972176687000001, 2024-09-17T21:31:03.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038597605, 2024-09-17T21:31:03.494 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.494 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.495 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.495 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50809 2024-09-17T21:31:03.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.495 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:31:03.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:31:03.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495232+0000", 2024-09-17T21:31:03.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.39e-07 2024-09-17T21:31:03.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495234+0000", 2024-09-17T21:31:03.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T21:31:03.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495239+0000", 2024-09-17T21:31:03.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7969999999999997e-06 2024-09-17T21:31:03.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495244+0000", 2024-09-17T21:31:03.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999996e-06 2024-09-17T21:31:03.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533125+0000", 2024-09-17T21:31:03.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037881523 2024-09-17T21:31:03.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533237+0000", 2024-09-17T21:31:03.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000111879 2024-09-17T21:31:03.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533829+0000", 2024-09-17T21:31:03.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000591933 2024-09-17T21:31:03.498 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.498 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.499 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.499 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50810 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 29884416~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:03.499 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:31:03.499 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.972158769000004, 2024-09-17T21:31:03.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039149809000000001, 2024-09-17T21:31:03.499 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.499 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.499 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.499 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50810 2024-09-17T21:31:03.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.500 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:31:03.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:31:03.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495250+0000", 2024-09-17T21:31:03.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:31:03.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495251+0000", 2024-09-17T21:31:03.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.81e-06 2024-09-17T21:31:03.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495257+0000", 2024-09-17T21:31:03.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1399999999999999e-06 2024-09-17T21:31:03.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495261+0000", 2024-09-17T21:31:03.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4229999999999998e-06 2024-09-17T21:31:03.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533879+0000", 2024-09-17T21:31:03.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038618292999999998 2024-09-17T21:31:03.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533939+0000", 2024-09-17T21:31:03.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.012e-05 2024-09-17T21:31:03.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534399+0000", 2024-09-17T21:31:03.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00045968099999999998 2024-09-17T21:31:03.503 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.503 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.503 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.503 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50850 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 2359296~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:03.503 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:03.504 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.972140394, 2024-09-17T21:31:03.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039731491000000001, 2024-09-17T21:31:03.504 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.504 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.504 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.504 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.504 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.504 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.504 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50850 2024-09-17T21:31:03.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.504 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:03.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:03.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:03.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3200000000000001e-07 2024-09-17T21:31:03.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495270+0000", 2024-09-17T21:31:03.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.694e-06 2024-09-17T21:31:03.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495275+0000", 2024-09-17T21:31:03.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5890000000000002e-06 2024-09-17T21:31:03.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495279+0000", 2024-09-17T21:31:03.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-17T21:31:03.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534440+0000", 2024-09-17T21:31:03.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039160805999999999 2024-09-17T21:31:03.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534526+0000", 2024-09-17T21:31:03.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5327999999999993e-05 2024-09-17T21:31:03.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534999+0000", 2024-09-17T21:31:03.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00047353400000000001 2024-09-17T21:31:03.508 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.508 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.508 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.508 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50851 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 3932160~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:03.508 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:31:03.508 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.972122782, 2024-09-17T21:31:03.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040211991000000002, 2024-09-17T21:31:03.508 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.508 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.508 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.509 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50851 2024-09-17T21:31:03.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.509 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:31:03.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:31:03.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495286+0000", 2024-09-17T21:31:03.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:31:03.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495287+0000", 2024-09-17T21:31:03.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6640000000000001e-06 2024-09-17T21:31:03.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495293+0000", 2024-09-17T21:31:03.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6710000000000004e-06 2024-09-17T21:31:03.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495298+0000", 2024-09-17T21:31:03.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6700000000000002e-06 2024-09-17T21:31:03.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535017+0000", 2024-09-17T21:31:03.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039719059000000001 2024-09-17T21:31:03.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535067+0000", 2024-09-17T21:31:03.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0046000000000002e-05 2024-09-17T21:31:03.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535497+0000", 2024-09-17T21:31:03.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043050299999999999 2024-09-17T21:31:03.512 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.512 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.512 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.512 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50852 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 7602176~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:03.513 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:03.513 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.972103865000001, 2024-09-17T21:31:03.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041166337999999997, 2024-09-17T21:31:03.513 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.513 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.513 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.513 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.513 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.513 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.513 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50852 2024-09-17T21:31:03.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.513 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:03.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:03.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495305+0000", 2024-09-17T21:31:03.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1600000000000002e-07 2024-09-17T21:31:03.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495307+0000", 2024-09-17T21:31:03.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0980000000000001e-06 2024-09-17T21:31:03.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495312+0000", 2024-09-17T21:31:03.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6210000000000001e-06 2024-09-17T21:31:03.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495317+0000", 2024-09-17T21:31:03.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T21:31:03.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535531+0000", 2024-09-17T21:31:03.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040214580999999999 2024-09-17T21:31:03.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535618+0000", 2024-09-17T21:31:03.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6648000000000001e-05 2024-09-17T21:31:03.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536471+0000", 2024-09-17T21:31:03.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00085251599999999999 2024-09-17T21:31:03.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.517 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.517 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50853 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 9175040~204800] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:03.517 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:03.517 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.972085534000001, 2024-09-17T21:31:03.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041673496999999997, 2024-09-17T21:31:03.518 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.518 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.518 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.518 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50853 2024-09-17T21:31:03.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.518 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:03.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:03.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:03.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:31:03.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495334+0000", 2024-09-17T21:31:03.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.076e-05 2024-09-17T21:31:03.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495340+0000", 2024-09-17T21:31:03.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:31:03.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495345+0000", 2024-09-17T21:31:03.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:31:03.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536514+0000", 2024-09-17T21:31:03.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041168534999999999 2024-09-17T21:31:03.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536572+0000", 2024-09-17T21:31:03.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8742000000000001e-05 2024-09-17T21:31:03.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536996+0000", 2024-09-17T21:31:03.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042368899999999999 2024-09-17T21:31:03.522 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.522 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.522 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.522 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50854 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 12845056~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:03.522 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:03.522 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.972056475000002, 2024-09-17T21:31:03.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042777397000000002, 2024-09-17T21:31:03.522 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.522 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.523 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.523 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.523 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.523 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.523 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50854 2024-09-17T21:31:03.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.523 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:03.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:03.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:03.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.82e-07 2024-09-17T21:31:03.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495354+0000", 2024-09-17T21:31:03.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.091e-06 2024-09-17T21:31:03.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495360+0000", 2024-09-17T21:31:03.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.716e-06 2024-09-17T21:31:03.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495365+0000", 2024-09-17T21:31:03.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2939999999999996e-06 2024-09-17T21:31:03.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537020+0000", 2024-09-17T21:31:03.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041654419999999998 2024-09-17T21:31:03.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537105+0000", 2024-09-17T21:31:03.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5857000000000001e-05 2024-09-17T21:31:03.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538129+0000", 2024-09-17T21:31:03.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010235369999999999 2024-09-17T21:31:03.526 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.526 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.526 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.527 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50855 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 14417920~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:03.527 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:03.527 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.972036510000002, 2024-09-17T21:31:03.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043254325000000003, 2024-09-17T21:31:03.527 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.527 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.527 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.527 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50855 2024-09-17T21:31:03.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.528 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:03.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:03.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:03.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:31:03.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495374+0000", 2024-09-17T21:31:03.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.922e-06 2024-09-17T21:31:03.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495379+0000", 2024-09-17T21:31:03.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-17T21:31:03.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495384+0000", 2024-09-17T21:31:03.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-17T21:31:03.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538167+0000", 2024-09-17T21:31:03.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042782845 2024-09-17T21:31:03.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538218+0000", 2024-09-17T21:31:03.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1335000000000002e-05 2024-09-17T21:31:03.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538626+0000", 2024-09-17T21:31:03.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000407656 2024-09-17T21:31:03.531 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.531 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.531 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.531 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50856 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 18087936~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:03.531 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:03.531 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.972018054999999, 2024-09-17T21:31:03.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043911444000000001, 2024-09-17T21:31:03.532 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.532 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.532 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.532 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.532 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.532 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.532 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50856 2024-09-17T21:31:03.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.532 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:03.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:03.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:03.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0499999999999999e-07 2024-09-17T21:31:03.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495392+0000", 2024-09-17T21:31:03.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7430000000000001e-06 2024-09-17T21:31:03.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495398+0000", 2024-09-17T21:31:03.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3040000000000001e-06 2024-09-17T21:31:03.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495403+0000", 2024-09-17T21:31:03.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5600000000000004e-06 2024-09-17T21:31:03.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538660+0000", 2024-09-17T21:31:03.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043256832000000002 2024-09-17T21:31:03.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538746+0000", 2024-09-17T21:31:03.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6448999999999999e-05 2024-09-17T21:31:03.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539301+0000", 2024-09-17T21:31:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055525099999999999 2024-09-17T21:31:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50857 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19660800~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.971998650000003, 2024-09-17T21:31:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044408156999999997, 2024-09-17T21:31:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.537 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.537 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.537 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50857 2024-09-17T21:31:03.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.537 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:03.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:03.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495410+0000", 2024-09-17T21:31:03.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:31:03.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495412+0000", 2024-09-17T21:31:03.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9580000000000001e-06 2024-09-17T21:31:03.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495418+0000", 2024-09-17T21:31:03.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1909999999999997e-06 2024-09-17T21:31:03.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495423+0000", 2024-09-17T21:31:03.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2429999999999996e-06 2024-09-17T21:31:03.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539338+0000", 2024-09-17T21:31:03.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043914478 2024-09-17T21:31:03.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539393+0000", 2024-09-17T21:31:03.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5226999999999998e-05 2024-09-17T21:31:03.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539818+0000", 2024-09-17T21:31:03.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042469300000000002 2024-09-17T21:31:03.540 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.540 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.540 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.540 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50858 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23330816~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:03.540 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:03.541 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.971977553000002, 2024-09-17T21:31:03.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045010798999999997, 2024-09-17T21:31:03.541 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.541 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.541 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.541 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.541 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.541 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.541 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50858 2024-09-17T21:31:03.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.541 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:03.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:03.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:03.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:31:03.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495433+0000", 2024-09-17T21:31:03.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.787e-06 2024-09-17T21:31:03.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495438+0000", 2024-09-17T21:31:03.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:31:03.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495442+0000", 2024-09-17T21:31:03.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2640000000000002e-06 2024-09-17T21:31:03.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539852+0000", 2024-09-17T21:31:03.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044409592999999997 2024-09-17T21:31:03.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539940+0000", 2024-09-17T21:31:03.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7696999999999997e-05 2024-09-17T21:31:03.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540441+0000", 2024-09-17T21:31:03.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050164299999999997 2024-09-17T21:31:03.545 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.545 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.545 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.545 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50859 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24903680~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:03.545 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:03.545 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.971959875000003, 2024-09-17T21:31:03.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045571569999999999, 2024-09-17T21:31:03.545 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.545 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.545 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.546 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.546 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.546 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50859 2024-09-17T21:31:03.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.546 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:03.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:03.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495449+0000", 2024-09-17T21:31:03.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:03.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495450+0000", 2024-09-17T21:31:03.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7889999999999999e-06 2024-09-17T21:31:03.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495456+0000", 2024-09-17T21:31:03.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5679999999999999e-06 2024-09-17T21:31:03.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495460+0000", 2024-09-17T21:31:03.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4989999999999996e-06 2024-09-17T21:31:03.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540472+0000", 2024-09-17T21:31:03.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045011585999999999 2024-09-17T21:31:03.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540554+0000", 2024-09-17T21:31:03.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2443999999999999e-05 2024-09-17T21:31:03.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541020+0000", 2024-09-17T21:31:03.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00046534300000000001 2024-09-17T21:31:03.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.549 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.550 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50860 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28573696~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:03.550 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:03.550 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.971941522000002, 2024-09-17T21:31:03.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046167899999999998, 2024-09-17T21:31:03.550 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.550 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.550 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.550 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50860 2024-09-17T21:31:03.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.550 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:03.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:03.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:03.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:31:03.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495469+0000", 2024-09-17T21:31:03.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.99e-06 2024-09-17T21:31:03.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495475+0000", 2024-09-17T21:31:03.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:31:03.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495480+0000", 2024-09-17T21:31:03.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:31:03.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541050+0000", 2024-09-17T21:31:03.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045569853 2024-09-17T21:31:03.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541138+0000", 2024-09-17T21:31:03.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7997000000000005e-05 2024-09-17T21:31:03.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541634+0000", 2024-09-17T21:31:03.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00049685500000000004 2024-09-17T21:31:03.554 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.554 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.554 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.554 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50861 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 30146560~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:03.554 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:03.554 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 51.971922345000003, 2024-09-17T21:31:03.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046690553000000003, 2024-09-17T21:31:03.555 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.555 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.555 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.555 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.555 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.555 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.555 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50861 2024-09-17T21:31:03.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.555 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:03.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:03.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:03.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:31:03.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495488+0000", 2024-09-17T21:31:03.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:31:03.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495495+0000", 2024-09-17T21:31:03.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:31:03.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495500+0000", 2024-09-17T21:31:03.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8620000000000002e-06 2024-09-17T21:31:03.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541687+0000", 2024-09-17T21:31:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046186434999999998 2024-09-17T21:31:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541741+0000", 2024-09-17T21:31:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4228999999999999e-05 2024-09-17T21:31:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.542176+0000", 2024-09-17T21:31:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043531100000000002 2024-09-17T21:31:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.558 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.559 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.559 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97804 3.7 3:e725aa05:::rbd_data.1120c318df8c.0000000000000003:24 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:03.559 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:03.559 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.505976243999999, 2024-09-17T21:31:03.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035514416, 2024-09-17T21:31:03.559 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:03.559 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:03.559 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:03.559 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:03.559 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:03.559 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.559 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97804 2024-09-17T21:31:03.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.560 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:03.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:03.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:03.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:03.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:03.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:03.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:03.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:03.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:31:03.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:03.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961434+0000", 2024-09-17T21:31:03.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8959999999999999e-06 2024-09-17T21:31:03.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:03.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961439+0000", 2024-09-17T21:31:03.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T21:31:03.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961444+0000", 2024-09-17T21:31:03.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4970000000000003e-06 2024-09-17T21:31:03.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:03.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996466+0000", 2024-09-17T21:31:03.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035022438000000003 2024-09-17T21:31:03.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:03.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996556+0000", 2024-09-17T21:31:03.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9561000000000005e-05 2024-09-17T21:31:03.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:03.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:03.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:03.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996946+0000", 2024-09-17T21:31:03.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039028700000000001 2024-09-17T21:31:03.563 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.563 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.563 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.563 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:03.563 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:03.564 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:03.565 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:31:03.566 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:03.563+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:03.666 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:03.664+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:03.671 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:03.671 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:03.671 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:03.671 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:03.679 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:31:03.767 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:03.765+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:03.782 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:03.782 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:03.782 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:03.782 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:03.782 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:03.790 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:31:03.868 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:03.866+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:03.896 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:03.896 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:03.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:03.896 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:03.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.896 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21342 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2082816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:31:03.896 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:03.896 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 74.216362275999998, 2024-09-17T21:31:03.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022083523000000001, 2024-09-17T21:31:03.896 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.896 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.896 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:03.897 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:03.897 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:03.897 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.897 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21342 2024-09-17T21:31:03.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.897 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:03.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:03.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:03.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:31:03.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676659+0000", 2024-09-17T21:31:03.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0950000000000001e-06 2024-09-17T21:31:03.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676662+0000", 2024-09-17T21:31:03.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.971e-06 2024-09-17T21:31:03.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676665+0000", 2024-09-17T21:31:03.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6189999999999998e-06 2024-09-17T21:31:03.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698568+0000", 2024-09-17T21:31:03.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021903549000000001 2024-09-17T21:31:03.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698668+0000", 2024-09-17T21:31:03.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9778999999999995e-05 2024-09-17T21:31:03.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698741+0000", 2024-09-17T21:31:03.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000006e-05 2024-09-17T21:31:03.900 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.900 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.900 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.900 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21343 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2095104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:31:03.900 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:03.900 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 74.216352337999993, 2024-09-17T21:31:03.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022504024000000001, 2024-09-17T21:31:03.901 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.901 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.901 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:03.901 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:03.901 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:03.901 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.901 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21343 2024-09-17T21:31:03.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.901 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:03.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:03.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:03.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:31:03.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676669+0000", 2024-09-17T21:31:03.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.003e-06 2024-09-17T21:31:03.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676672+0000", 2024-09-17T21:31:03.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9280000000000002e-06 2024-09-17T21:31:03.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676674+0000", 2024-09-17T21:31:03.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5469999999999999e-06 2024-09-17T21:31:03.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699015+0000", 2024-09-17T21:31:03.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022340563000000001 2024-09-17T21:31:03.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699102+0000", 2024-09-17T21:31:03.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7425999999999999e-05 2024-09-17T21:31:03.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699172+0000", 2024-09-17T21:31:03.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9346999999999999e-05 2024-09-17T21:31:03.904 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.904 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.904 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.905 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45576 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:03.905 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:03.905 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.188118537999998, 2024-09-17T21:31:03.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022024660000000001, 2024-09-17T21:31:03.905 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.905 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.905 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:03.905 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:03.905 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:03.905 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.905 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45576 2024-09-17T21:31:03.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.905 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:03.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:03.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704902+0000", 2024-09-17T21:31:03.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9100000000000004e-07 2024-09-17T21:31:03.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704905+0000", 2024-09-17T21:31:03.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6139999999999999e-06 2024-09-17T21:31:03.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704911+0000", 2024-09-17T21:31:03.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8619999999999997e-06 2024-09-17T21:31:03.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704917+0000", 2024-09-17T21:31:03.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:31:03.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726681+0000", 2024-09-17T21:31:03.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021764569000000001 2024-09-17T21:31:03.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726726+0000", 2024-09-17T21:31:03.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4218999999999998e-05 2024-09-17T21:31:03.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726926+0000", 2024-09-17T21:31:03.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000200466 2024-09-17T21:31:03.908 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.908 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.908 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.909 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45577 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:03.909 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:03.909 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.188094882000001, 2024-09-17T21:31:03.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022269694999999999, 2024-09-17T21:31:03.909 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.909 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.909 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:03.909 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:03.909 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:03.909 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.909 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45577 2024-09-17T21:31:03.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.909 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:03.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:03.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704926+0000", 2024-09-17T21:31:03.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:31:03.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704928+0000", 2024-09-17T21:31:03.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9989999999999998e-06 2024-09-17T21:31:03.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704934+0000", 2024-09-17T21:31:03.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:31:03.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704939+0000", 2024-09-17T21:31:03.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5909999999999998e-06 2024-09-17T21:31:03.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726954+0000", 2024-09-17T21:31:03.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022014885000000001 2024-09-17T21:31:03.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726995+0000", 2024-09-17T21:31:03.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1474000000000003e-05 2024-09-17T21:31:03.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727195+0000", 2024-09-17T21:31:03.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019968300000000001 2024-09-17T21:31:03.913 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.913 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.913 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.913 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45578 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:03.913 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:03.913 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.188073637999999, 2024-09-17T21:31:03.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022484006000000001, 2024-09-17T21:31:03.914 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.914 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.914 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:03.914 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:03.914 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:03.914 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.914 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45578 2024-09-17T21:31:03.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.914 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:03.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:03.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704947+0000", 2024-09-17T21:31:03.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1399999999999997e-07 2024-09-17T21:31:03.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704949+0000", 2024-09-17T21:31:03.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0959999999999999e-06 2024-09-17T21:31:03.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704956+0000", 2024-09-17T21:31:03.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9330000000000004e-06 2024-09-17T21:31:03.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704961+0000", 2024-09-17T21:31:03.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6759999999999998e-06 2024-09-17T21:31:03.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727213+0000", 2024-09-17T21:31:03.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022252568 2024-09-17T21:31:03.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727259+0000", 2024-09-17T21:31:03.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5238000000000003e-05 2024-09-17T21:31:03.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727430+0000", 2024-09-17T21:31:03.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000171881 2024-09-17T21:31:03.917 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.917 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.917 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.918 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45579 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:03.918 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:03.918 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.188051710000003, 2024-09-17T21:31:03.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022749080000000001, 2024-09-17T21:31:03.918 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.918 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.918 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:03.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:03.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:03.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.918 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45579 2024-09-17T21:31:03.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.919 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:03.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:03.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704969+0000", 2024-09-17T21:31:03.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:31:03.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704971+0000", 2024-09-17T21:31:03.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.097e-06 2024-09-17T21:31:03.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704977+0000", 2024-09-17T21:31:03.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6329999999999999e-06 2024-09-17T21:31:03.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704983+0000", 2024-09-17T21:31:03.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:31:03.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727457+0000", 2024-09-17T21:31:03.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022474521000000001 2024-09-17T21:31:03.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727498+0000", 2024-09-17T21:31:03.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0558e-05 2024-09-17T21:31:03.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727717+0000", 2024-09-17T21:31:03.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219805 2024-09-17T21:31:03.922 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.922 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.922 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.922 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45652 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:03.922 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:03.922 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.186905572000001, 2024-09-17T21:31:03.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022100465999999999, 2024-09-17T21:31:03.922 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.923 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.923 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:03.923 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:03.923 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:03.923 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.923 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45652 2024-09-17T21:31:03.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.923 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:03.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:03.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706115+0000", 2024-09-17T21:31:03.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:31:03.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706117+0000", 2024-09-17T21:31:03.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-06 2024-09-17T21:31:03.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706122+0000", 2024-09-17T21:31:03.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.288e-06 2024-09-17T21:31:03.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706136+0000", 2024-09-17T21:31:03.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3409e-05 2024-09-17T21:31:03.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728112+0000", 2024-09-17T21:31:03.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021976724999999999 2024-09-17T21:31:03.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728151+0000", 2024-09-17T21:31:03.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8318999999999997e-05 2024-09-17T21:31:03.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728215+0000", 2024-09-17T21:31:03.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4172999999999997e-05 2024-09-17T21:31:03.926 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.926 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.926 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.927 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45653 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:03.927 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:03.927 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.186876362, 2024-09-17T21:31:03.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022249517999999999, 2024-09-17T21:31:03.927 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.927 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.927 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:03.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:03.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:03.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.927 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45653 2024-09-17T21:31:03.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.928 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:03.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:03.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:03.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T21:31:03.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706147+0000", 2024-09-17T21:31:03.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4430000000000002e-06 2024-09-17T21:31:03.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706152+0000", 2024-09-17T21:31:03.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5350000000000004e-06 2024-09-17T21:31:03.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706157+0000", 2024-09-17T21:31:03.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8420000000000004e-06 2024-09-17T21:31:03.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728246+0000", 2024-09-17T21:31:03.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022089237000000001 2024-09-17T21:31:03.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728309+0000", 2024-09-17T21:31:03.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2356999999999999e-05 2024-09-17T21:31:03.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:31:03.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4596000000000001e-05 2024-09-17T21:31:03.931 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.931 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.931 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.931 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45657 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 36864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:03.931 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:03.931 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.186845097999999, 2024-09-17T21:31:03.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022218252000000001, 2024-09-17T21:31:03.931 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.932 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.932 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:03.932 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:03.932 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:03.932 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.932 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45657 2024-09-17T21:31:03.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.932 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:03.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:03.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:03.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8299999999999997e-07 2024-09-17T21:31:03.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706178+0000", 2024-09-17T21:31:03.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3700000000000002e-06 2024-09-17T21:31:03.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706184+0000", 2024-09-17T21:31:03.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.144e-06 2024-09-17T21:31:03.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706190+0000", 2024-09-17T21:31:03.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8590000000000001e-06 2024-09-17T21:31:03.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728240+0000", 2024-09-17T21:31:03.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022049804999999999 2024-09-17T21:31:03.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728307+0000", 2024-09-17T21:31:03.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7477999999999993e-05 2024-09-17T21:31:03.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:31:03.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6112999999999994e-05 2024-09-17T21:31:03.935 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.935 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.935 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.935 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45658 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:03.935 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:03.935 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.186821977999998, 2024-09-17T21:31:03.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022376455, 2024-09-17T21:31:03.936 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.936 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.936 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:03.936 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:03.936 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:03.936 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.936 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45658 2024-09-17T21:31:03.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.936 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:03.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:03.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:03.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:31:03.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706200+0000", 2024-09-17T21:31:03.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:31:03.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706206+0000", 2024-09-17T21:31:03.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7039999999999999e-06 2024-09-17T21:31:03.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706211+0000", 2024-09-17T21:31:03.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9429999999999999e-06 2024-09-17T21:31:03.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728418+0000", 2024-09-17T21:31:03.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022206760999999998 2024-09-17T21:31:03.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728497+0000", 2024-09-17T21:31:03.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9493999999999995e-05 2024-09-17T21:31:03.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728574+0000", 2024-09-17T21:31:03.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7180000000000003e-05 2024-09-17T21:31:03.939 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.939 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.940 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.940 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45707 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 212992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:03.940 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:03.940 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.185117912000003, 2024-09-17T21:31:03.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022235550999999999, 2024-09-17T21:31:03.940 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.940 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.940 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:03.940 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:03.940 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:03.940 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.940 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45707 2024-09-17T21:31:03.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.941 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:03.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:03.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:03.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:31:03.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707905+0000", 2024-09-17T21:31:03.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2120000000000002e-06 2024-09-17T21:31:03.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707911+0000", 2024-09-17T21:31:03.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1929999999999998e-06 2024-09-17T21:31:03.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707915+0000", 2024-09-17T21:31:03.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:31:03.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729897+0000", 2024-09-17T21:31:03.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021981186 2024-09-17T21:31:03.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729944+0000", 2024-09-17T21:31:03.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571000000000002e-05 2024-09-17T21:31:03.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730138+0000", 2024-09-17T21:31:03.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000193451 2024-09-17T21:31:03.944 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.944 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.944 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.944 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45708 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:03.944 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:03.944 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.18509778, 2024-09-17T21:31:03.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022533742999999998, 2024-09-17T21:31:03.944 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.945 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.945 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:03.945 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:03.945 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:03.945 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.945 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45708 2024-09-17T21:31:03.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.945 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:03.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:03.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707923+0000", 2024-09-17T21:31:03.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:31:03.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707932+0000", 2024-09-17T21:31:03.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6430000000000007e-06 2024-09-17T21:31:03.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707939+0000", 2024-09-17T21:31:03.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3539999999999996e-06 2024-09-17T21:31:03.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707944+0000", 2024-09-17T21:31:03.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3249999999999998e-06 2024-09-17T21:31:03.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730176+0000", 2024-09-17T21:31:03.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022232393 2024-09-17T21:31:03.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730263+0000", 2024-09-17T21:31:03.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6638999999999995e-05 2024-09-17T21:31:03.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730456+0000", 2024-09-17T21:31:03.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019304899999999999 2024-09-17T21:31:03.948 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.948 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.948 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.949 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45752 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:03.949 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:03.949 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.18484256, 2024-09-17T21:31:03.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022426878000000001, 2024-09-17T21:31:03.949 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.949 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.949 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:03.949 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:03.949 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:03.949 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.949 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45752 2024-09-17T21:31:03.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.949 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:03.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:03.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708178+0000", 2024-09-17T21:31:03.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:31:03.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708181+0000", 2024-09-17T21:31:03.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.909e-06 2024-09-17T21:31:03.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708187+0000", 2024-09-17T21:31:03.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4999999999999996e-06 2024-09-17T21:31:03.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708193+0000", 2024-09-17T21:31:03.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:31:03.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730484+0000", 2024-09-17T21:31:03.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022291765000000002 2024-09-17T21:31:03.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730541+0000", 2024-09-17T21:31:03.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6483000000000001e-05 2024-09-17T21:31:03.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730604+0000", 2024-09-17T21:31:03.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3429000000000005e-05 2024-09-17T21:31:03.953 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.953 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.953 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.953 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45761 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:03.953 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:03.953 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.18481465, 2024-09-17T21:31:03.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022614408999999999, 2024-09-17T21:31:03.953 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.953 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.953 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:03.953 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:03.954 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:03.954 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.954 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45761 2024-09-17T21:31:03.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.954 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:03.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:03.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708206+0000", 2024-09-17T21:31:03.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4700000000000001e-07 2024-09-17T21:31:03.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708208+0000", 2024-09-17T21:31:03.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1509999999999998e-06 2024-09-17T21:31:03.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708214+0000", 2024-09-17T21:31:03.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.49e-06 2024-09-17T21:31:03.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708219+0000", 2024-09-17T21:31:03.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8119999999999998e-06 2024-09-17T21:31:03.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730641+0000", 2024-09-17T21:31:03.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022422765000000001 2024-09-17T21:31:03.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730731+0000", 2024-09-17T21:31:03.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9598999999999994e-05 2024-09-17T21:31:03.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730820+0000", 2024-09-17T21:31:03.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8844999999999994e-05 2024-09-17T21:31:03.957 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.957 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.957 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.958 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45785 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:03.958 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:03.958 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.183592863000001, 2024-09-17T21:31:03.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022044113000000001, 2024-09-17T21:31:03.958 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.958 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.958 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:03.958 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:03.958 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:03.958 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.958 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45785 2024-09-17T21:31:03.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.958 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:03.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:03.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709428+0000", 2024-09-17T21:31:03.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:31:03.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709430+0000", 2024-09-17T21:31:03.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:31:03.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709451+0000", 2024-09-17T21:31:03.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1372000000000001e-05 2024-09-17T21:31:03.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709472+0000", 2024-09-17T21:31:03.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1007999999999999e-05 2024-09-17T21:31:03.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731293+0000", 2024-09-17T21:31:03.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021821320000000002 2024-09-17T21:31:03.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731378+0000", 2024-09-17T21:31:03.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5059e-05 2024-09-17T21:31:03.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731471+0000", 2024-09-17T21:31:03.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2867000000000004e-05 2024-09-17T21:31:03.962 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.962 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.962 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.962 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45786 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:03.962 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:03.962 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.183534066, 2024-09-17T21:31:03.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022181821000000001, 2024-09-17T21:31:03.962 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.962 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.962 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:03.963 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:03.963 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:03.963 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.963 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45786 2024-09-17T21:31:03.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.963 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:03.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:03.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709487+0000", 2024-09-17T21:31:03.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1299999999999999e-07 2024-09-17T21:31:03.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709491+0000", 2024-09-17T21:31:03.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2089999999999999e-06 2024-09-17T21:31:03.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709498+0000", 2024-09-17T21:31:03.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3980000000000002e-06 2024-09-17T21:31:03.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709504+0000", 2024-09-17T21:31:03.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7219999999999996e-06 2024-09-17T21:31:03.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731499+0000", 2024-09-17T21:31:03.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021995297 2024-09-17T21:31:03.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731591+0000", 2024-09-17T21:31:03.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1373999999999996e-05 2024-09-17T21:31:03.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731668+0000", 2024-09-17T21:31:03.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7107999999999994e-05 2024-09-17T21:31:03.966 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.966 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.966 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.966 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50202 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'139, mlcod=45'137)", 2024-09-17T21:31:03.967 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:03.967 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 52.768798197000002, 2024-09-17T21:31:03.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059659441000000001, 2024-09-17T21:31:03.967 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.967 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.967 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:03.967 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:03.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:03.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124224+0000", 2024-09-17T21:31:03.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5000000000000002e-06 2024-09-17T21:31:03.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147385+0000", 2024-09-17T21:31:03.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023160544000000002 2024-09-17T21:31:03.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147411+0000", 2024-09-17T21:31:03.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6095999999999999e-05 2024-09-17T21:31:03.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147438+0000", 2024-09-17T21:31:03.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6982999999999999e-05 2024-09-17T21:31:03.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147504+0000", 2024-09-17T21:31:03.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6156999999999997e-05 2024-09-17T21:31:03.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147531+0000", 2024-09-17T21:31:03.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6741999999999999e-05 2024-09-17T21:31:03.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:03.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183820+0000", 2024-09-17T21:31:03.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036289090000000003 2024-09-17T21:31:03.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183881+0000", 2024-09-17T21:31:03.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1328999999999995e-05 2024-09-17T21:31:03.970 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.970 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.970 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.971 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50203 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'141, mlcod=45'137)", 2024-09-17T21:31:03.971 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:03.971 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 52.731535454000003, 2024-09-17T21:31:03.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023138823999999999, 2024-09-17T21:31:03.971 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.971 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.971 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:03.971 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:03.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:03.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161486+0000", 2024-09-17T21:31:03.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5290000000000001e-06 2024-09-17T21:31:03.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161496+0000", 2024-09-17T21:31:03.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0063e-05 2024-09-17T21:31:03.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161514+0000", 2024-09-17T21:31:03.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7728e-05 2024-09-17T21:31:03.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161524+0000", 2024-09-17T21:31:03.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0339e-05 2024-09-17T21:31:03.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170872+0000", 2024-09-17T21:31:03.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093473640000000004 2024-09-17T21:31:03.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170899+0000", 2024-09-17T21:31:03.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7321000000000002e-05 2024-09-17T21:31:03.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:03.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184580+0000", 2024-09-17T21:31:03.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013680988999999999 2024-09-17T21:31:03.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184623+0000", 2024-09-17T21:31:03.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3491000000000001e-05 2024-09-17T21:31:03.975 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.975 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.975 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.975 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:88193 3.9 3:97498da5:::rbd_data.1120ea662e44.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-17T21:31:03.975 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:03.975 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.569354543999999, 2024-09-17T21:31:03.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026278261000000001, 2024-09-17T21:31:03.975 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.976 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.976 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:03.976 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:03.976 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:03.976 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.976 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 88193 2024-09-17T21:31:03.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.976 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:03.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323666+0000", 2024-09-17T21:31:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:31:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323670+0000", 2024-09-17T21:31:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.771e-06 2024-09-17T21:31:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323680+0000", 2024-09-17T21:31:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9469999999999998e-06 2024-09-17T21:31:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323685+0000", 2024-09-17T21:31:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9919999999999998e-06 2024-09-17T21:31:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323728+0000", 2024-09-17T21:31:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2811000000000001e-05 2024-09-17T21:31:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323776+0000", 2024-09-17T21:31:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8310000000000003e-05 2024-09-17T21:31:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.328583+0000", 2024-09-17T21:31:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048072899999999997 2024-09-17T21:31:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349861+0000", 2024-09-17T21:31:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021277695999999999 2024-09-17T21:31:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349877+0000", 2024-09-17T21:31:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:31:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349911+0000", 2024-09-17T21:31:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3844999999999999e-05 2024-09-17T21:31:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349944+0000", 2024-09-17T21:31:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2663000000000002e-05 2024-09-17T21:31:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:101384 3.9 3:90f2a32a:::rbd_data.11206b8c0882.0000000000000178:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T21:31:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.566484944999999, 2024-09-17T21:31:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025286194000000001, 2024-09-17T21:31:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 101384 2024-09-17T21:31:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326536+0000", 2024-09-17T21:31:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4799999999999997e-07 2024-09-17T21:31:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326541+0000", 2024-09-17T21:31:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8489999999999997e-06 2024-09-17T21:31:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326552+0000", 2024-09-17T21:31:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0912e-05 2024-09-17T21:31:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326557+0000", 2024-09-17T21:31:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0359999999999997e-06 2024-09-17T21:31:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326603+0000", 2024-09-17T21:31:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6428999999999999e-05 2024-09-17T21:31:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326686+0000", 2024-09-17T21:31:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2521999999999994e-05 2024-09-17T21:31:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.332698+0000", 2024-09-17T21:31:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060124599999999999 2024-09-17T21:31:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351744+0000", 2024-09-17T21:31:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019045838999999998 2024-09-17T21:31:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351754+0000", 2024-09-17T21:31:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0380999999999999e-05 2024-09-17T21:31:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351791+0000", 2024-09-17T21:31:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6678999999999999e-05 2024-09-17T21:31:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:03.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351821+0000", 2024-09-17T21:31:03.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0139000000000001e-05 2024-09-17T21:31:03.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.987 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:03.987 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:03.987 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:03.987 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:31:03.989 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:31:03.992 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:03.986+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:04.020 INFO:tasks.thrashosds.thrasher:Setting full ratio back to .95 2024-09-17T21:31:04.020 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .95 2024-09-17T21:31:04.089 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:04.087+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:04.138 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:04.138 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:04.138 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:04.138 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:04.147 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:31:04.190 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:04.187+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:04.250 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:04.250 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:04.250 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:04.250 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:04.250 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:04.259 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:31:04.291 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:04.289+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:04.363 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:04.363 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:04.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:04.363 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:04.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.363 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10799 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.363 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:04.363 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.294407613000004, 2024-09-17T21:31:04.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039122745, 2024-09-17T21:31:04.363 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.364 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.364 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:04.364 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.364 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.364 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.364 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10799 2024-09-17T21:31:04.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.364 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:04.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:04.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065469+0000", 2024-09-17T21:31:04.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:31:04.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065477+0000", 2024-09-17T21:31:04.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1319999999999994e-06 2024-09-17T21:31:04.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065483+0000", 2024-09-17T21:31:04.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2040000000000002e-06 2024-09-17T21:31:04.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065487+0000", 2024-09-17T21:31:04.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.42e-06 2024-09-17T21:31:04.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099984+0000", 2024-09-17T21:31:04.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034496484000000001 2024-09-17T21:31:04.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100065+0000", 2024-09-17T21:31:04.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1200999999999997e-05 2024-09-17T21:31:04.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:04.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100422+0000", 2024-09-17T21:31:04.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035657700000000001 2024-09-17T21:31:04.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:04.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104214+0000", 2024-09-17T21:31:04.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003792045 2024-09-17T21:31:04.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:04.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104505+0000", 2024-09-17T21:31:04.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029141399999999999 2024-09-17T21:31:04.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:04.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104556+0000", 2024-09-17T21:31:04.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1388999999999999e-05 2024-09-17T21:31:04.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104591+0000", 2024-09-17T21:31:04.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4455999999999997e-05 2024-09-17T21:31:04.369 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.369 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.369 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.369 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12187 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.370 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:04.370 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.220314881999997, 2024-09-17T21:31:04.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038825750999999999, 2024-09-17T21:31:04.370 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.370 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.370 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:04.370 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.370 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.370 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.370 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12187 2024-09-17T21:31:04.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.370 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:04.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:04.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:04.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:31:04.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139563+0000", 2024-09-17T21:31:04.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.624e-06 2024-09-17T21:31:04.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139569+0000", 2024-09-17T21:31:04.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1090000000000003e-06 2024-09-17T21:31:04.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139573+0000", 2024-09-17T21:31:04.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4429999999999996e-06 2024-09-17T21:31:04.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178206+0000", 2024-09-17T21:31:04.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038632376000000003 2024-09-17T21:31:04.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178282+0000", 2024-09-17T21:31:04.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6076999999999997e-05 2024-09-17T21:31:04.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178387+0000", 2024-09-17T21:31:04.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104682 2024-09-17T21:31:04.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.374 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.374 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12188 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.374 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:04.374 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.220296700999995, 2024-09-17T21:31:04.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039053826999999999, 2024-09-17T21:31:04.374 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.374 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.374 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:04.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.375 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12188 2024-09-17T21:31:04.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.375 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:04.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:04.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139746+0000", 2024-09-17T21:31:04.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000167281 2024-09-17T21:31:04.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139749+0000", 2024-09-17T21:31:04.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9660000000000001e-06 2024-09-17T21:31:04.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139757+0000", 2024-09-17T21:31:04.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8029999999999997e-06 2024-09-17T21:31:04.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139771+0000", 2024-09-17T21:31:04.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3541e-05 2024-09-17T21:31:04.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178415+0000", 2024-09-17T21:31:04.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038644803999999998 2024-09-17T21:31:04.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178517+0000", 2024-09-17T21:31:04.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101298 2024-09-17T21:31:04.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178633+0000", 2024-09-17T21:31:04.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116134 2024-09-17T21:31:04.378 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.378 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.378 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.379 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12189 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.379 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:04.379 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.220096224000002, 2024-09-17T21:31:04.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039135991000000002, 2024-09-17T21:31:04.379 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.379 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.379 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:04.379 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.379 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.379 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.379 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12189 2024-09-17T21:31:04.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.380 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:04.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:04.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139780+0000", 2024-09-17T21:31:04.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:31:04.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139782+0000", 2024-09-17T21:31:04.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1950000000000002e-06 2024-09-17T21:31:04.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139788+0000", 2024-09-17T21:31:04.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0639999999999999e-06 2024-09-17T21:31:04.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139792+0000", 2024-09-17T21:31:04.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1960000000000001e-06 2024-09-17T21:31:04.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178671+0000", 2024-09-17T21:31:04.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038878176 2024-09-17T21:31:04.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178778+0000", 2024-09-17T21:31:04.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107278 2024-09-17T21:31:04.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178915+0000", 2024-09-17T21:31:04.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013757900000000001 2024-09-17T21:31:04.383 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.383 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.383 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.383 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12190 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.383 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:04.383 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.220077040999996, 2024-09-17T21:31:04.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039343781000000001, 2024-09-17T21:31:04.384 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.384 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.384 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:04.384 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.384 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.384 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.384 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12190 2024-09-17T21:31:04.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.384 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:04.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:04.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:04.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:04.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139801+0000", 2024-09-17T21:31:04.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.916e-06 2024-09-17T21:31:04.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139807+0000", 2024-09-17T21:31:04.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4540000000000004e-06 2024-09-17T21:31:04.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139812+0000", 2024-09-17T21:31:04.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9729999999999997e-06 2024-09-17T21:31:04.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178960+0000", 2024-09-17T21:31:04.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039147567000000001 2024-09-17T21:31:04.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179051+0000", 2024-09-17T21:31:04.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0754000000000006e-05 2024-09-17T21:31:04.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179142+0000", 2024-09-17T21:31:04.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1712999999999994e-05 2024-09-17T21:31:04.387 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.388 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.388 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.388 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12191 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.388 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:04.388 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.220057182999994, 2024-09-17T21:31:04.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039464336000000003, 2024-09-17T21:31:04.388 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.388 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.388 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:04.388 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.388 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.389 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12191 2024-09-17T21:31:04.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.389 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:04.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:04.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:04.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:31:04.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139821+0000", 2024-09-17T21:31:04.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1260000000000001e-06 2024-09-17T21:31:04.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139827+0000", 2024-09-17T21:31:04.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T21:31:04.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139840+0000", 2024-09-17T21:31:04.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2607e-05 2024-09-17T21:31:04.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179162+0000", 2024-09-17T21:31:04.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039322863999999999 2024-09-17T21:31:04.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179204+0000", 2024-09-17T21:31:04.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1938999999999999e-05 2024-09-17T21:31:04.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179283+0000", 2024-09-17T21:31:04.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8547000000000005e-05 2024-09-17T21:31:04.392 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.392 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.392 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.393 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12192 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.393 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:04.393 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.220029471000004, 2024-09-17T21:31:04.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039652318999999998, 2024-09-17T21:31:04.393 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.393 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.393 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:04.393 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.393 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.393 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.393 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12192 2024-09-17T21:31:04.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.393 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:04.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:04.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141458+0000", 2024-09-17T21:31:04.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001612098 2024-09-17T21:31:04.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141469+0000", 2024-09-17T21:31:04.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-17T21:31:04.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141478+0000", 2024-09-17T21:31:04.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T21:31:04.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141485+0000", 2024-09-17T21:31:04.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4760000000000003e-06 2024-09-17T21:31:04.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179317+0000", 2024-09-17T21:31:04.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037831832000000003 2024-09-17T21:31:04.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179395+0000", 2024-09-17T21:31:04.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8349000000000005e-05 2024-09-17T21:31:04.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179499+0000", 2024-09-17T21:31:04.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103628 2024-09-17T21:31:04.397 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.397 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.397 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.397 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12257 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.397 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:04.397 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.217840663000004, 2024-09-17T21:31:04.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038862751000000001, 2024-09-17T21:31:04.397 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.398 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.398 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:04.398 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.398 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.398 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.398 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12257 2024-09-17T21:31:04.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.398 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:04.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:04.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:04.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T21:31:04.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142037+0000", 2024-09-17T21:31:04.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.03e-06 2024-09-17T21:31:04.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142043+0000", 2024-09-17T21:31:04.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6319999999999996e-06 2024-09-17T21:31:04.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142052+0000", 2024-09-17T21:31:04.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2709999999999998e-06 2024-09-17T21:31:04.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180794+0000", 2024-09-17T21:31:04.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038741758000000001 2024-09-17T21:31:04.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180843+0000", 2024-09-17T21:31:04.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9021999999999999e-05 2024-09-17T21:31:04.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180898+0000", 2024-09-17T21:31:04.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673000000000003e-05 2024-09-17T21:31:04.402 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.402 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.402 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.402 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12258 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.403 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:04.403 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.217817146000002, 2024-09-17T21:31:04.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038950717000000003, 2024-09-17T21:31:04.403 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.403 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.403 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:04.403 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.403 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.403 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.403 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12258 2024-09-17T21:31:04.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.403 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:04.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:04.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:04.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199999999999999e-07 2024-09-17T21:31:04.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142061+0000", 2024-09-17T21:31:04.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8279999999999999e-06 2024-09-17T21:31:04.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142067+0000", 2024-09-17T21:31:04.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5600000000000001e-06 2024-09-17T21:31:04.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142071+0000", 2024-09-17T21:31:04.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6240000000000001e-06 2024-09-17T21:31:04.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180921+0000", 2024-09-17T21:31:04.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038849366000000003 2024-09-17T21:31:04.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180961+0000", 2024-09-17T21:31:04.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0219000000000002e-05 2024-09-17T21:31:04.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181009+0000", 2024-09-17T21:31:04.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8437999999999999e-05 2024-09-17T21:31:04.407 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.407 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.407 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.407 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12259 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.407 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:04.407 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.217789898999996, 2024-09-17T21:31:04.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039116832999999997, 2024-09-17T21:31:04.408 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.408 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.408 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:04.408 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.408 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.408 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.408 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12259 2024-09-17T21:31:04.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.408 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:04.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:04.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:04.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2200000000000004e-07 2024-09-17T21:31:04.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142088+0000", 2024-09-17T21:31:04.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0329999999999998e-06 2024-09-17T21:31:04.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142095+0000", 2024-09-17T21:31:04.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4320000000000004e-06 2024-09-17T21:31:04.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142100+0000", 2024-09-17T21:31:04.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:31:04.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181037+0000", 2024-09-17T21:31:04.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038937601000000002 2024-09-17T21:31:04.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181114+0000", 2024-09-17T21:31:04.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6857e-05 2024-09-17T21:31:04.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181203+0000", 2024-09-17T21:31:04.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8437999999999996e-05 2024-09-17T21:31:04.411 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.411 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.412 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.412 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12260 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.412 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:04.412 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.217769611999998, 2024-09-17T21:31:04.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039204184000000003, 2024-09-17T21:31:04.412 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.412 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.412 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:04.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.413 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12260 2024-09-17T21:31:04.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.413 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:04.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:04.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142107+0000", 2024-09-17T21:31:04.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:31:04.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142108+0000", 2024-09-17T21:31:04.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.753e-06 2024-09-17T21:31:04.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142114+0000", 2024-09-17T21:31:04.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7660000000000004e-06 2024-09-17T21:31:04.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142119+0000", 2024-09-17T21:31:04.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:31:04.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181216+0000", 2024-09-17T21:31:04.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039097191000000003 2024-09-17T21:31:04.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181260+0000", 2024-09-17T21:31:04.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4731000000000003e-05 2024-09-17T21:31:04.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181310+0000", 2024-09-17T21:31:04.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9818000000000003e-05 2024-09-17T21:31:04.416 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.416 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.416 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.416 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12261 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.416 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:04.416 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.217751288000002, 2024-09-17T21:31:04.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039292200999999999, 2024-09-17T21:31:04.417 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.417 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.417 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:04.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.417 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12261 2024-09-17T21:31:04.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.417 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:04.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:04.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142125+0000", 2024-09-17T21:31:04.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:31:04.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142127+0000", 2024-09-17T21:31:04.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.792e-06 2024-09-17T21:31:04.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142132+0000", 2024-09-17T21:31:04.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3979999999999998e-06 2024-09-17T21:31:04.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142136+0000", 2024-09-17T21:31:04.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.211e-06 2024-09-17T21:31:04.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181324+0000", 2024-09-17T21:31:04.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039187560000000003 2024-09-17T21:31:04.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181363+0000", 2024-09-17T21:31:04.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9375000000000002e-05 2024-09-17T21:31:04.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181417+0000", 2024-09-17T21:31:04.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3372000000000003e-05 2024-09-17T21:31:04.420 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.420 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.421 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.421 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12262 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.421 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:04.421 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.217733495000004, 2024-09-17T21:31:04.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039384875, 2024-09-17T21:31:04.421 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.421 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.421 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:04.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.422 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12262 2024-09-17T21:31:04.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.422 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:04.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:04.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142143+0000", 2024-09-17T21:31:04.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:31:04.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142148+0000", 2024-09-17T21:31:04.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4360000000000001e-06 2024-09-17T21:31:04.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142154+0000", 2024-09-17T21:31:04.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8719999999999999e-06 2024-09-17T21:31:04.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142158+0000", 2024-09-17T21:31:04.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.476e-06 2024-09-17T21:31:04.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181436+0000", 2024-09-17T21:31:04.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039277919000000001 2024-09-17T21:31:04.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181480+0000", 2024-09-17T21:31:04.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3649999999999997e-05 2024-09-17T21:31:04.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181527+0000", 2024-09-17T21:31:04.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7131e-05 2024-09-17T21:31:04.425 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.425 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.425 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.426 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12263 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1079296~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.426 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:04.426 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.217711059999999, 2024-09-17T21:31:04.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039473339000000003, 2024-09-17T21:31:04.426 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.426 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.426 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:04.426 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.426 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.426 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.426 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12263 2024-09-17T21:31:04.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.427 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:04.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:04.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:04.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:31:04.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142167+0000", 2024-09-17T21:31:04.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.81e-06 2024-09-17T21:31:04.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142172+0000", 2024-09-17T21:31:04.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:31:04.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142177+0000", 2024-09-17T21:31:04.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T21:31:04.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181553+0000", 2024-09-17T21:31:04.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039376130000000002 2024-09-17T21:31:04.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181592+0000", 2024-09-17T21:31:04.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9153999999999998e-05 2024-09-17T21:31:04.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181638+0000", 2024-09-17T21:31:04.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5864e-05 2024-09-17T21:31:04.430 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.430 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.430 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.430 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12278 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1095680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.430 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:04.430 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.217693084999993, 2024-09-17T21:31:04.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039454222999999997, 2024-09-17T21:31:04.431 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.431 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.431 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:04.431 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.431 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.431 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.431 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12278 2024-09-17T21:31:04.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.431 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:04.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:04.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:04.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:31:04.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142185+0000", 2024-09-17T21:31:04.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.697e-06 2024-09-17T21:31:04.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142191+0000", 2024-09-17T21:31:04.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:31:04.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142196+0000", 2024-09-17T21:31:04.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4829999999999999e-06 2024-09-17T21:31:04.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181548+0000", 2024-09-17T21:31:04.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039351200000000003 2024-09-17T21:31:04.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181591+0000", 2024-09-17T21:31:04.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3492000000000003e-05 2024-09-17T21:31:04.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181637+0000", 2024-09-17T21:31:04.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5847000000000003e-05 2024-09-17T21:31:04.435 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.435 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.435 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.435 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12279 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.435 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:04.435 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.217669795999996, 2024-09-17T21:31:04.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039539801999999999, 2024-09-17T21:31:04.435 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.435 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.435 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:04.435 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.436 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.436 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.436 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12279 2024-09-17T21:31:04.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.436 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:04.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:04.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:04.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.89e-07 2024-09-17T21:31:04.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142208+0000", 2024-09-17T21:31:04.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7579999999999999e-06 2024-09-17T21:31:04.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142214+0000", 2024-09-17T21:31:04.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:31:04.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142218+0000", 2024-09-17T21:31:04.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3429999999999998e-06 2024-09-17T21:31:04.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181653+0000", 2024-09-17T21:31:04.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039435322000000002 2024-09-17T21:31:04.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181698+0000", 2024-09-17T21:31:04.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4978999999999997e-05 2024-09-17T21:31:04.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181746+0000", 2024-09-17T21:31:04.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571999999999998e-05 2024-09-17T21:31:04.439 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.439 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.439 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.440 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12280 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1124352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.440 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:04.440 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.217652276999999, 2024-09-17T21:31:04.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039704358000000002, 2024-09-17T21:31:04.440 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.440 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.440 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:04.440 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.440 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.440 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.440 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12280 2024-09-17T21:31:04.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.440 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:04.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:04.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142224+0000", 2024-09-17T21:31:04.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:31:04.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142226+0000", 2024-09-17T21:31:04.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:31:04.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142231+0000", 2024-09-17T21:31:04.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:31:04.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142240+0000", 2024-09-17T21:31:04.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4590000000000004e-06 2024-09-17T21:31:04.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181774+0000", 2024-09-17T21:31:04.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039533833999999997 2024-09-17T21:31:04.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181859+0000", 2024-09-17T21:31:04.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4328999999999998e-05 2024-09-17T21:31:04.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181928+0000", 2024-09-17T21:31:04.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9139000000000003e-05 2024-09-17T21:31:04.444 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.444 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.444 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.444 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12281 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1136640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.444 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:04.444 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.217629371000001, 2024-09-17T21:31:04.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039787799999999998, 2024-09-17T21:31:04.444 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.445 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.445 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:04.445 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.445 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.445 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.445 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12281 2024-09-17T21:31:04.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.445 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:04.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:04.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142309+0000", 2024-09-17T21:31:04.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2323999999999995e-05 2024-09-17T21:31:04.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142317+0000", 2024-09-17T21:31:04.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1859999999999992e-06 2024-09-17T21:31:04.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142324+0000", 2024-09-17T21:31:04.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4699999999999996e-06 2024-09-17T21:31:04.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142330+0000", 2024-09-17T21:31:04.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4650000000000002e-06 2024-09-17T21:31:04.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181948+0000", 2024-09-17T21:31:04.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039618056999999998 2024-09-17T21:31:04.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181986+0000", 2024-09-17T21:31:04.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8442999999999998e-05 2024-09-17T21:31:04.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182034+0000", 2024-09-17T21:31:04.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7855000000000002e-05 2024-09-17T21:31:04.449 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.449 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.449 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.449 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12282 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1150976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.449 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:04.449 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.217537141999998, 2024-09-17T21:31:04.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039871919999999998, 2024-09-17T21:31:04.449 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.449 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.449 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:04.450 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.450 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.450 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.450 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12282 2024-09-17T21:31:04.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.450 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:04.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:04.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:04.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:31:04.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142341+0000", 2024-09-17T21:31:04.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.232e-06 2024-09-17T21:31:04.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142348+0000", 2024-09-17T21:31:04.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7320000000000001e-06 2024-09-17T21:31:04.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142353+0000", 2024-09-17T21:31:04.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:31:04.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182062+0000", 2024-09-17T21:31:04.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039709739000000001 2024-09-17T21:31:04.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:04.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182128+0000", 2024-09-17T21:31:04.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5739000000000002e-05 2024-09-17T21:31:04.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182210+0000", 2024-09-17T21:31:04.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:31:04.453 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.453 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.453 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.454 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12283 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:04.454 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:04.454 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.217516731000003, 2024-09-17T21:31:04.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039896434000000001, 2024-09-17T21:31:04.454 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:04.454 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:04.454 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:31:04.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:04.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:04.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:04.454 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12283 2024-09-17T21:31:04.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.455 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:04.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:04.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:04.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:04.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:04.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:04.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:04.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144852+0000", 2024-09-17T21:31:04.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024934079999999999 2024-09-17T21:31:04.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:04.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144856+0000", 2024-09-17T21:31:04.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:31:04.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:04.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144864+0000", 2024-09-17T21:31:04.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0160000000000008e-06 2024-09-17T21:31:04.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:04.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144869+0000", 2024-09-17T21:31:04.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:31:04.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:04.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182246+0000", 2024-09-17T21:31:04.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037377207000000003 2024-09-17T21:31:04.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:04.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:04.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:04.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182255+0000", 2024-09-17T21:31:04.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1840000000000005e-06 2024-09-17T21:31:04.458 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.458 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.458 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.458 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:04.458 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:04.458 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:04.459 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:31:04.460 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:04.457+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:04.471 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:31:04.560 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:04.558+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:04.592 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:04.592 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:04.606 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:04.607 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:04.607 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:04.607 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:04.609 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:04.609 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:04.616 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:31:04.625 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:04.626 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:04.641 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:04.642 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:04.658 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:04.658 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:04.661 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:04.658+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:04.674 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:04.674 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:04.691 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:04.691 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:04.708 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:04.708 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:04.760 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:04.760 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:04.761 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:04.759+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:04.769 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:31:04.862 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:04.860+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:04.913 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:04.914 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:04.923 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:31:04.958 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is set 2024-09-17T21:31:04.958 INFO:teuthology.orchestra.run.smithi129.stderr:osd set-full-ratio 0.95 2024-09-17T21:31:04.965 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:04.963+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:05.064 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:05.062+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:05.082 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:05.082 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:05.092 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:31:05.165 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:05.163+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:05.236 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:05.236 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:05.245 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:31:05.290 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:05.288+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:05.349 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:05.349 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:05.358 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:31:05.401 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:05.400+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:05.465 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:05.465 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:05.467 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:05.465+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:05.473 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:31:05.568 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:05.566+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:05.616 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:05.617 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:05.625 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:31:05.669 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:05.667+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:05.729 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:05.729 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:05.738 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:31:05.772 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:05.770+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:05.842 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:05.842 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:05.851 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:31:05.872 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:05.869+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:05.954 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:05.954 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:05.963 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:31:05.972 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:05.970+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:06.073 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:06.071+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:06.077 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:06.077 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:06.086 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:31:06.173 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:06.172+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:06.231 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:06.231 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:06.239 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:31:06.280 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:06.279+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:06.376 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:06.373+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:06.383 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:06.383 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:06.392 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:31:06.476 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:06.474+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:06.494 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:06.494 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:06.503 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:31:06.576 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:06.575+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:06.607 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:06.607 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:06.615 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:31:06.699 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:06.698+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:06.719 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:31:06.761 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:06.762 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:06.771 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:31:06.778 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:06.776+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:06.841 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:06.842 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:06.858 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:06.858 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:06.874 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:06.874 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:06.879 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:06.877+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:06.891 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:06.891 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:06.908 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:06.909 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:06.919 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:06.919 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:06.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:06.919 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:06.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.920 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:49361 3.19 e46/25 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:head v 46'135, mlcod=45'133)", 2024-09-17T21:31:06.920 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:06.920 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.162993223000001, 2024-09-17T21:31:06.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043477744999999998, 2024-09-17T21:31:06.920 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.920 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:06.920 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:06.920 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:06.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:06.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:06.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:06.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:06.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:06.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:06.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-07 2024-09-17T21:31:06.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:06.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769703+0000", 2024-09-17T21:31:06.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016134381 2024-09-17T21:31:06.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:06.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769724+0000", 2024-09-17T21:31:06.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0781999999999999e-05 2024-09-17T21:31:06.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:06.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769734+0000", 2024-09-17T21:31:06.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6870000000000006e-06 2024-09-17T21:31:06.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:06.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769797+0000", 2024-09-17T21:31:06.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3418999999999997e-05 2024-09-17T21:31:06.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:06.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769822+0000", 2024-09-17T21:31:06.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4652999999999999e-05 2024-09-17T21:31:06.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:06.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.796982+0000", 2024-09-17T21:31:06.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027160205 2024-09-17T21:31:06.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:06.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.797047+0000", 2024-09-17T21:31:06.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4455999999999994e-05 2024-09-17T21:31:06.924 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.924 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:06.924 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.924 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50805 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 17825792~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:06.924 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:31:06.924 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.421409728999997, 2024-09-17T21:31:06.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035591579999999998, 2024-09-17T21:31:06.924 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.924 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:06.925 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:06.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:06.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:06.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:06.925 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50805 2024-09-17T21:31:06.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.925 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:06.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:06.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:31:06.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:06.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:31:06.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:06.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495153+0000", 2024-09-17T21:31:06.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:31:06.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:06.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495155+0000", 2024-09-17T21:31:06.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1849999999999998e-06 2024-09-17T21:31:06.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:06.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495162+0000", 2024-09-17T21:31:06.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.64e-06 2024-09-17T21:31:06.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:06.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495166+0000", 2024-09-17T21:31:06.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:31:06.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:06.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530020+0000", 2024-09-17T21:31:06.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.034853824999999998 2024-09-17T21:31:06.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:06.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530076+0000", 2024-09-17T21:31:06.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5235999999999998e-05 2024-09-17T21:31:06.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:06.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530744+0000", 2024-09-17T21:31:06.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066835500000000003 2024-09-17T21:31:06.928 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.928 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:06.928 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.929 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50806 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19398656~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:06.929 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:06.929 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.421389261000002, 2024-09-17T21:31:06.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036408459999999997, 2024-09-17T21:31:06.929 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.929 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:06.929 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:06.929 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:06.929 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:06.929 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:06.929 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50806 2024-09-17T21:31:06.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.929 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:06.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:06.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:06.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:06.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:06.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:06.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:06.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.72e-07 2024-09-17T21:31:06.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:06.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495175+0000", 2024-09-17T21:31:06.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:31:06.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:06.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495182+0000", 2024-09-17T21:31:06.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8079999999999999e-06 2024-09-17T21:31:06.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:06.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495186+0000", 2024-09-17T21:31:06.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5549999999999996e-06 2024-09-17T21:31:06.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:06.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530791+0000", 2024-09-17T21:31:06.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035604963000000003 2024-09-17T21:31:06.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:06.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530855+0000", 2024-09-17T21:31:06.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3286000000000004e-05 2024-09-17T21:31:06.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:06.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531581+0000", 2024-09-17T21:31:06.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00072655699999999996 2024-09-17T21:31:06.933 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.933 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:06.933 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.933 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50807 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23068672~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:06.933 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:06.933 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.421370375999999, 2024-09-17T21:31:06.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037183314000000002, 2024-09-17T21:31:06.933 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.933 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:06.934 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:06.934 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:06.934 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:06.934 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:06.934 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50807 2024-09-17T21:31:06.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.934 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:06.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:06.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:06.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:06.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:06.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:06.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:06.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9999999999999999e-07 2024-09-17T21:31:06.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:06.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495194+0000", 2024-09-17T21:31:06.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.779e-06 2024-09-17T21:31:06.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:06.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495199+0000", 2024-09-17T21:31:06.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.468e-06 2024-09-17T21:31:06.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:06.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495203+0000", 2024-09-17T21:31:06.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0010000000000003e-06 2024-09-17T21:31:06.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:06.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531636+0000", 2024-09-17T21:31:06.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036432665000000003 2024-09-17T21:31:06.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:06.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531727+0000", 2024-09-17T21:31:06.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1607999999999994e-05 2024-09-17T21:31:06.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:06.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532375+0000", 2024-09-17T21:31:06.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00064749299999999996 2024-09-17T21:31:06.937 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.937 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:06.937 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.937 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50808 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24641536~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:06.937 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:31:06.938 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.421353633000003, 2024-09-17T21:31:06.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037870190999999997, 2024-09-17T21:31:06.938 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.938 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:06.938 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:06.938 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:06.938 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:06.938 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:06.938 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50808 2024-09-17T21:31:06.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.938 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:06.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:06.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:31:06.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:06.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:31:06.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:06.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495209+0000", 2024-09-17T21:31:06.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:31:06.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:06.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495215+0000", 2024-09-17T21:31:06.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1630000000000001e-06 2024-09-17T21:31:06.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:06.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495221+0000", 2024-09-17T21:31:06.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-17T21:31:06.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:06.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495226+0000", 2024-09-17T21:31:06.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8189999999999999e-06 2024-09-17T21:31:06.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:06.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532425+0000", 2024-09-17T21:31:06.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037199135000000001 2024-09-17T21:31:06.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:06.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532531+0000", 2024-09-17T21:31:06.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106314 2024-09-17T21:31:06.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:06.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533079+0000", 2024-09-17T21:31:06.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00054739700000000003 2024-09-17T21:31:06.941 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.941 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:06.942 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.942 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50809 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28311552~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:06.942 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:31:06.942 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.421330666999999, 2024-09-17T21:31:06.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038597605, 2024-09-17T21:31:06.942 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.942 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:06.942 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:06.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:06.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:06.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:06.942 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50809 2024-09-17T21:31:06.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.943 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:06.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:06.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:31:06.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:06.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:31:06.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:06.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495232+0000", 2024-09-17T21:31:06.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.39e-07 2024-09-17T21:31:06.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:06.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495234+0000", 2024-09-17T21:31:06.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T21:31:06.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:06.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495239+0000", 2024-09-17T21:31:06.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7969999999999997e-06 2024-09-17T21:31:06.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:06.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495244+0000", 2024-09-17T21:31:06.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999996e-06 2024-09-17T21:31:06.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:06.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533125+0000", 2024-09-17T21:31:06.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037881523 2024-09-17T21:31:06.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:06.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533237+0000", 2024-09-17T21:31:06.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000111879 2024-09-17T21:31:06.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:06.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533829+0000", 2024-09-17T21:31:06.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000591933 2024-09-17T21:31:06.946 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.946 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:06.946 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.946 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50810 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 29884416~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:06.946 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:31:06.946 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.421312749000002, 2024-09-17T21:31:06.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039149809000000001, 2024-09-17T21:31:06.946 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.946 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:06.946 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:06.946 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:06.947 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:06.947 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:06.947 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50810 2024-09-17T21:31:06.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.947 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:06.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:06.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:31:06.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:06.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:31:06.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:06.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495250+0000", 2024-09-17T21:31:06.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:31:06.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:06.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495251+0000", 2024-09-17T21:31:06.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.81e-06 2024-09-17T21:31:06.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:06.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495257+0000", 2024-09-17T21:31:06.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1399999999999999e-06 2024-09-17T21:31:06.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:06.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495261+0000", 2024-09-17T21:31:06.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4229999999999998e-06 2024-09-17T21:31:06.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:06.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533879+0000", 2024-09-17T21:31:06.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038618292999999998 2024-09-17T21:31:06.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:06.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533939+0000", 2024-09-17T21:31:06.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.012e-05 2024-09-17T21:31:06.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:06.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534399+0000", 2024-09-17T21:31:06.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00045968099999999998 2024-09-17T21:31:06.950 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.950 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:06.950 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.950 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50850 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 2359296~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:06.950 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:06.950 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.421294373999999, 2024-09-17T21:31:06.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039731491000000001, 2024-09-17T21:31:06.951 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.951 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:06.951 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:06.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:06.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:06.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:06.951 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50850 2024-09-17T21:31:06.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.951 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:06.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:06.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:06.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:06.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:06.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:06.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:06.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3200000000000001e-07 2024-09-17T21:31:06.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:06.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495270+0000", 2024-09-17T21:31:06.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.694e-06 2024-09-17T21:31:06.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:06.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495275+0000", 2024-09-17T21:31:06.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5890000000000002e-06 2024-09-17T21:31:06.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:06.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495279+0000", 2024-09-17T21:31:06.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-17T21:31:06.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:06.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534440+0000", 2024-09-17T21:31:06.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039160805999999999 2024-09-17T21:31:06.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534526+0000", 2024-09-17T21:31:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5327999999999993e-05 2024-09-17T21:31:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534999+0000", 2024-09-17T21:31:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00047353400000000001 2024-09-17T21:31:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50851 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 3932160~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:31:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.421276761999998, 2024-09-17T21:31:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040211991000000002, 2024-09-17T21:31:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50851 2024-09-17T21:31:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:31:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:31:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495286+0000", 2024-09-17T21:31:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:31:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495287+0000", 2024-09-17T21:31:06.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6640000000000001e-06 2024-09-17T21:31:06.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:06.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495293+0000", 2024-09-17T21:31:06.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6710000000000004e-06 2024-09-17T21:31:06.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:06.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495298+0000", 2024-09-17T21:31:06.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6700000000000002e-06 2024-09-17T21:31:06.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:06.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535017+0000", 2024-09-17T21:31:06.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039719059000000001 2024-09-17T21:31:06.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:06.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535067+0000", 2024-09-17T21:31:06.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0046000000000002e-05 2024-09-17T21:31:06.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:06.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535497+0000", 2024-09-17T21:31:06.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043050299999999999 2024-09-17T21:31:06.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.958 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:06.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.959 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50852 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 7602176~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:06.959 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:06.959 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.421257845, 2024-09-17T21:31:06.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041166337999999997, 2024-09-17T21:31:06.959 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.959 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:06.959 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:06.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:06.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:06.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:06.959 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50852 2024-09-17T21:31:06.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.959 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:06.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:06.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:06.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:06.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:06.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:06.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495305+0000", 2024-09-17T21:31:06.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1600000000000002e-07 2024-09-17T21:31:06.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:06.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495307+0000", 2024-09-17T21:31:06.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0980000000000001e-06 2024-09-17T21:31:06.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:06.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495312+0000", 2024-09-17T21:31:06.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6210000000000001e-06 2024-09-17T21:31:06.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:06.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495317+0000", 2024-09-17T21:31:06.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T21:31:06.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:06.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535531+0000", 2024-09-17T21:31:06.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040214580999999999 2024-09-17T21:31:06.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:06.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535618+0000", 2024-09-17T21:31:06.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6648000000000001e-05 2024-09-17T21:31:06.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:06.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536471+0000", 2024-09-17T21:31:06.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00085251599999999999 2024-09-17T21:31:06.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.963 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:06.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.963 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50853 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 9175040~204800] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:06.963 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:06.963 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.421239514, 2024-09-17T21:31:06.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041673496999999997, 2024-09-17T21:31:06.963 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.964 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:06.964 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:06.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:06.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:06.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:06.964 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50853 2024-09-17T21:31:06.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.964 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:06.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:06.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:06.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:06.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:06.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:06.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:06.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:31:06.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:06.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495334+0000", 2024-09-17T21:31:06.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.076e-05 2024-09-17T21:31:06.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:06.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495340+0000", 2024-09-17T21:31:06.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:31:06.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:06.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495345+0000", 2024-09-17T21:31:06.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:31:06.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:06.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536514+0000", 2024-09-17T21:31:06.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041168534999999999 2024-09-17T21:31:06.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:06.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536572+0000", 2024-09-17T21:31:06.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8742000000000001e-05 2024-09-17T21:31:06.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:06.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536996+0000", 2024-09-17T21:31:06.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042368899999999999 2024-09-17T21:31:06.968 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.968 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:06.968 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.968 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50854 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 12845056~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:06.968 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:06.968 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.421210455000001, 2024-09-17T21:31:06.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042777397000000002, 2024-09-17T21:31:06.968 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.968 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:06.969 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:06.969 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:06.969 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:06.969 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:06.969 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50854 2024-09-17T21:31:06.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.969 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:06.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:06.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:06.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:06.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:06.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:06.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:06.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.82e-07 2024-09-17T21:31:06.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:06.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495354+0000", 2024-09-17T21:31:06.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.091e-06 2024-09-17T21:31:06.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:06.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495360+0000", 2024-09-17T21:31:06.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.716e-06 2024-09-17T21:31:06.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:06.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495365+0000", 2024-09-17T21:31:06.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2939999999999996e-06 2024-09-17T21:31:06.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:06.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537020+0000", 2024-09-17T21:31:06.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041654419999999998 2024-09-17T21:31:06.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:06.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537105+0000", 2024-09-17T21:31:06.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5857000000000001e-05 2024-09-17T21:31:06.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:06.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538129+0000", 2024-09-17T21:31:06.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010235369999999999 2024-09-17T21:31:06.972 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.973 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:06.973 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.973 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50855 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 14417920~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:06.973 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:06.973 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.421190490000001, 2024-09-17T21:31:06.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043254325000000003, 2024-09-17T21:31:06.973 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.973 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:06.973 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:06.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:06.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:06.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:06.974 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50855 2024-09-17T21:31:06.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.974 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:06.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:06.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:06.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:06.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:06.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:06.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:06.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:31:06.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:06.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495374+0000", 2024-09-17T21:31:06.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.922e-06 2024-09-17T21:31:06.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:06.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495379+0000", 2024-09-17T21:31:06.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-17T21:31:06.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:06.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495384+0000", 2024-09-17T21:31:06.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-17T21:31:06.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:06.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538167+0000", 2024-09-17T21:31:06.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042782845 2024-09-17T21:31:06.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:06.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538218+0000", 2024-09-17T21:31:06.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1335000000000002e-05 2024-09-17T21:31:06.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:06.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538626+0000", 2024-09-17T21:31:06.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000407656 2024-09-17T21:31:06.977 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.977 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:06.977 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.977 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50856 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 18087936~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:06.977 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:06.977 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.421172034999998, 2024-09-17T21:31:06.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043911444000000001, 2024-09-17T21:31:06.978 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.978 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:06.978 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:06.978 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:06.978 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:06.978 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:06.978 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50856 2024-09-17T21:31:06.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.978 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:06.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:06.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:06.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:06.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:06.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:06.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:06.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0499999999999999e-07 2024-09-17T21:31:06.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:06.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495392+0000", 2024-09-17T21:31:06.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7430000000000001e-06 2024-09-17T21:31:06.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:06.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495398+0000", 2024-09-17T21:31:06.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3040000000000001e-06 2024-09-17T21:31:06.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:06.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495403+0000", 2024-09-17T21:31:06.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5600000000000004e-06 2024-09-17T21:31:06.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:06.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538660+0000", 2024-09-17T21:31:06.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043256832000000002 2024-09-17T21:31:06.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:06.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538746+0000", 2024-09-17T21:31:06.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6448999999999999e-05 2024-09-17T21:31:06.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:06.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539301+0000", 2024-09-17T21:31:06.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055525099999999999 2024-09-17T21:31:06.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.981 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:06.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.982 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50857 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19660800~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:06.982 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:06.982 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.421152630000002, 2024-09-17T21:31:06.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044408156999999997, 2024-09-17T21:31:06.982 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.982 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:06.982 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:06.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:06.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:06.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:06.982 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50857 2024-09-17T21:31:06.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.983 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:06.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:06.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:06.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:06.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:06.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:06.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495410+0000", 2024-09-17T21:31:06.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:31:06.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:06.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495412+0000", 2024-09-17T21:31:06.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9580000000000001e-06 2024-09-17T21:31:06.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:06.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495418+0000", 2024-09-17T21:31:06.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1909999999999997e-06 2024-09-17T21:31:06.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:06.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495423+0000", 2024-09-17T21:31:06.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2429999999999996e-06 2024-09-17T21:31:06.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:06.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539338+0000", 2024-09-17T21:31:06.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043914478 2024-09-17T21:31:06.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:06.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539393+0000", 2024-09-17T21:31:06.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5226999999999998e-05 2024-09-17T21:31:06.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:06.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539818+0000", 2024-09-17T21:31:06.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042469300000000002 2024-09-17T21:31:06.986 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.986 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:06.986 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.986 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50858 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23330816~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:06.986 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:06.986 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.421131533000001, 2024-09-17T21:31:06.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045010798999999997, 2024-09-17T21:31:06.986 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.986 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:06.986 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:06.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:06.987 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:06.987 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:06.987 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50858 2024-09-17T21:31:06.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.987 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:06.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:06.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:06.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:06.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:06.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:06.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:06.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:31:06.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:06.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495433+0000", 2024-09-17T21:31:06.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.787e-06 2024-09-17T21:31:06.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:06.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495438+0000", 2024-09-17T21:31:06.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:31:06.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:06.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495442+0000", 2024-09-17T21:31:06.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2640000000000002e-06 2024-09-17T21:31:06.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:06.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539852+0000", 2024-09-17T21:31:06.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044409592999999997 2024-09-17T21:31:06.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:06.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539940+0000", 2024-09-17T21:31:06.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7696999999999997e-05 2024-09-17T21:31:06.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:06.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540441+0000", 2024-09-17T21:31:06.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050164299999999997 2024-09-17T21:31:06.990 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.990 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:06.990 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.990 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50859 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24903680~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:06.990 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:06.990 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.421113855000002, 2024-09-17T21:31:06.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045571569999999999, 2024-09-17T21:31:06.991 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.991 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:06.991 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:06.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:06.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:06.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:06.991 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50859 2024-09-17T21:31:06.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.991 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:06.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:06.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:06.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:06.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:06.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:06.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495449+0000", 2024-09-17T21:31:06.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:06.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:06.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495450+0000", 2024-09-17T21:31:06.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7889999999999999e-06 2024-09-17T21:31:06.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:06.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495456+0000", 2024-09-17T21:31:06.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5679999999999999e-06 2024-09-17T21:31:06.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:06.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495460+0000", 2024-09-17T21:31:06.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4989999999999996e-06 2024-09-17T21:31:06.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:06.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540472+0000", 2024-09-17T21:31:06.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045011585999999999 2024-09-17T21:31:06.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:06.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540554+0000", 2024-09-17T21:31:06.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2443999999999999e-05 2024-09-17T21:31:06.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:06.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541020+0000", 2024-09-17T21:31:06.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00046534300000000001 2024-09-17T21:31:06.994 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.994 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:06.994 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.994 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50860 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28573696~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:06.994 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:06.995 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.421095502, 2024-09-17T21:31:06.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046167899999999998, 2024-09-17T21:31:06.995 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.995 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:06.995 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:06.995 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:06.995 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:06.995 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:06.995 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50860 2024-09-17T21:31:06.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.995 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:06.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:06.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:06.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:06.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:06.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:06.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:06.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:06.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:31:06.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:06.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495469+0000", 2024-09-17T21:31:06.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.99e-06 2024-09-17T21:31:06.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:06.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495475+0000", 2024-09-17T21:31:06.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:31:06.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:06.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495480+0000", 2024-09-17T21:31:06.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:31:06.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:06.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541050+0000", 2024-09-17T21:31:06.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045569853 2024-09-17T21:31:06.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:06.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541138+0000", 2024-09-17T21:31:06.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7997000000000005e-05 2024-09-17T21:31:06.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:06.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541634+0000", 2024-09-17T21:31:06.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00049685500000000004 2024-09-17T21:31:06.999 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.999 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:06.999 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:06.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:06.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:06.999 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50861 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 30146560~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:06.999 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:06.999 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.421076325000001, 2024-09-17T21:31:06.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046690553000000003, 2024-09-17T21:31:06.999 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:06.999 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:07.000 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:07.000 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:07.000 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:07.000 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.000 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50861 2024-09-17T21:31:07.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.000 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:07.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:07.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:07.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:07.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:07.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:07.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:07.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:07.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:07.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:31:07.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:07.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495488+0000", 2024-09-17T21:31:07.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:31:07.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:07.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495495+0000", 2024-09-17T21:31:07.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:31:07.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495500+0000", 2024-09-17T21:31:07.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8620000000000002e-06 2024-09-17T21:31:07.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:07.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541687+0000", 2024-09-17T21:31:07.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046186434999999998 2024-09-17T21:31:07.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:07.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541741+0000", 2024-09-17T21:31:07.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4228999999999999e-05 2024-09-17T21:31:07.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:07.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.542176+0000", 2024-09-17T21:31:07.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043531100000000002 2024-09-17T21:31:07.003 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:07.003 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:07.003 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:07.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.003 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97804 3.7 3:e725aa05:::rbd_data.1120c318df8c.0000000000000003:24 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:07.003 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:07.004 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 21.955130224000001, 2024-09-17T21:31:07.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035514416, 2024-09-17T21:31:07.004 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:07.004 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:07.004 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:07.004 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:07.004 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:07.004 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.004 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97804 2024-09-17T21:31:07.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.004 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:07.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:07.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:07.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:07.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:07.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:07.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:07.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:07.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:07.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:31:07.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:07.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961434+0000", 2024-09-17T21:31:07.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8959999999999999e-06 2024-09-17T21:31:07.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:07.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961439+0000", 2024-09-17T21:31:07.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T21:31:07.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961444+0000", 2024-09-17T21:31:07.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4970000000000003e-06 2024-09-17T21:31:07.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:07.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996466+0000", 2024-09-17T21:31:07.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035022438000000003 2024-09-17T21:31:07.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:07.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996556+0000", 2024-09-17T21:31:07.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9561000000000005e-05 2024-09-17T21:31:07.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:07.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:07.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:07.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996946+0000", 2024-09-17T21:31:07.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039028700000000001 2024-09-17T21:31:07.007 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:07.007 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:07.007 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:07.007 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:07.007 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:07.008 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:07.008 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:31:07.009 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:07.009 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:07.006+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:07.009 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:07.010 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:31:07.010 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:07.011 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:07.011 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:07.012 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:07.109 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:07.107+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:07.130 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:07.130 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:07.130 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:07.130 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:07.139 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:31:07.224 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:07.222+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:07.284 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:07.284 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:07.285 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:07.285 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:07.285 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:07.293 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:31:07.334 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:07.332+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:07.403 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is set 2024-09-17T21:31:07.412 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:07.410+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:07.451 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21342 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2082816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:31:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.771653710999999, 2024-09-17T21:31:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022083523000000001, 2024-09-17T21:31:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21342 2024-09-17T21:31:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:31:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676659+0000", 2024-09-17T21:31:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0950000000000001e-06 2024-09-17T21:31:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676662+0000", 2024-09-17T21:31:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.971e-06 2024-09-17T21:31:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676665+0000", 2024-09-17T21:31:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6189999999999998e-06 2024-09-17T21:31:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698568+0000", 2024-09-17T21:31:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021903549000000001 2024-09-17T21:31:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698668+0000", 2024-09-17T21:31:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9778999999999995e-05 2024-09-17T21:31:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698741+0000", 2024-09-17T21:31:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000006e-05 2024-09-17T21:31:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21343 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2095104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:31:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.771643772999994, 2024-09-17T21:31:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022504024000000001, 2024-09-17T21:31:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21343 2024-09-17T21:31:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:31:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676669+0000", 2024-09-17T21:31:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.003e-06 2024-09-17T21:31:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676672+0000", 2024-09-17T21:31:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9280000000000002e-06 2024-09-17T21:31:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676674+0000", 2024-09-17T21:31:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5469999999999999e-06 2024-09-17T21:31:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699015+0000", 2024-09-17T21:31:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022340563000000001 2024-09-17T21:31:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699102+0000", 2024-09-17T21:31:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7425999999999999e-05 2024-09-17T21:31:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699172+0000", 2024-09-17T21:31:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9346999999999999e-05 2024-09-17T21:31:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45576 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.743409972999999, 2024-09-17T21:31:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022024660000000001, 2024-09-17T21:31:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45576 2024-09-17T21:31:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704902+0000", 2024-09-17T21:31:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9100000000000004e-07 2024-09-17T21:31:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704905+0000", 2024-09-17T21:31:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6139999999999999e-06 2024-09-17T21:31:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704911+0000", 2024-09-17T21:31:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8619999999999997e-06 2024-09-17T21:31:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704917+0000", 2024-09-17T21:31:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:31:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726681+0000", 2024-09-17T21:31:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021764569000000001 2024-09-17T21:31:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726726+0000", 2024-09-17T21:31:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4218999999999998e-05 2024-09-17T21:31:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726926+0000", 2024-09-17T21:31:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000200466 2024-09-17T21:31:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45577 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.743386317000002, 2024-09-17T21:31:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022269694999999999, 2024-09-17T21:31:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45577 2024-09-17T21:31:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704926+0000", 2024-09-17T21:31:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:31:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704928+0000", 2024-09-17T21:31:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9989999999999998e-06 2024-09-17T21:31:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704934+0000", 2024-09-17T21:31:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:31:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704939+0000", 2024-09-17T21:31:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5909999999999998e-06 2024-09-17T21:31:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726954+0000", 2024-09-17T21:31:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022014885000000001 2024-09-17T21:31:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726995+0000", 2024-09-17T21:31:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1474000000000003e-05 2024-09-17T21:31:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727195+0000", 2024-09-17T21:31:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019968300000000001 2024-09-17T21:31:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45578 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.743365073, 2024-09-17T21:31:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022484006000000001, 2024-09-17T21:31:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45578 2024-09-17T21:31:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704947+0000", 2024-09-17T21:31:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1399999999999997e-07 2024-09-17T21:31:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704949+0000", 2024-09-17T21:31:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0959999999999999e-06 2024-09-17T21:31:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704956+0000", 2024-09-17T21:31:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9330000000000004e-06 2024-09-17T21:31:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704961+0000", 2024-09-17T21:31:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6759999999999998e-06 2024-09-17T21:31:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727213+0000", 2024-09-17T21:31:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022252568 2024-09-17T21:31:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727259+0000", 2024-09-17T21:31:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5238000000000003e-05 2024-09-17T21:31:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727430+0000", 2024-09-17T21:31:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000171881 2024-09-17T21:31:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45579 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.743343144999997, 2024-09-17T21:31:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022749080000000001, 2024-09-17T21:31:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45579 2024-09-17T21:31:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704969+0000", 2024-09-17T21:31:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:31:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704971+0000", 2024-09-17T21:31:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.097e-06 2024-09-17T21:31:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704977+0000", 2024-09-17T21:31:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6329999999999999e-06 2024-09-17T21:31:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704983+0000", 2024-09-17T21:31:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:31:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727457+0000", 2024-09-17T21:31:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022474521000000001 2024-09-17T21:31:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727498+0000", 2024-09-17T21:31:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0558e-05 2024-09-17T21:31:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727717+0000", 2024-09-17T21:31:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219805 2024-09-17T21:31:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45652 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.742197007000001, 2024-09-17T21:31:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022100465999999999, 2024-09-17T21:31:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45652 2024-09-17T21:31:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706115+0000", 2024-09-17T21:31:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:31:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706117+0000", 2024-09-17T21:31:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-06 2024-09-17T21:31:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706122+0000", 2024-09-17T21:31:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.288e-06 2024-09-17T21:31:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706136+0000", 2024-09-17T21:31:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3409e-05 2024-09-17T21:31:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728112+0000", 2024-09-17T21:31:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021976724999999999 2024-09-17T21:31:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728151+0000", 2024-09-17T21:31:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8318999999999997e-05 2024-09-17T21:31:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728215+0000", 2024-09-17T21:31:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4172999999999997e-05 2024-09-17T21:31:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45653 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.742167797, 2024-09-17T21:31:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022249517999999999, 2024-09-17T21:31:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45653 2024-09-17T21:31:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T21:31:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706147+0000", 2024-09-17T21:31:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4430000000000002e-06 2024-09-17T21:31:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706152+0000", 2024-09-17T21:31:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5350000000000004e-06 2024-09-17T21:31:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706157+0000", 2024-09-17T21:31:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8420000000000004e-06 2024-09-17T21:31:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728246+0000", 2024-09-17T21:31:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022089237000000001 2024-09-17T21:31:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728309+0000", 2024-09-17T21:31:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2356999999999999e-05 2024-09-17T21:31:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:31:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4596000000000001e-05 2024-09-17T21:31:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45657 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 36864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.742136533, 2024-09-17T21:31:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022218252000000001, 2024-09-17T21:31:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45657 2024-09-17T21:31:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8299999999999997e-07 2024-09-17T21:31:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706178+0000", 2024-09-17T21:31:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3700000000000002e-06 2024-09-17T21:31:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706184+0000", 2024-09-17T21:31:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.144e-06 2024-09-17T21:31:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706190+0000", 2024-09-17T21:31:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8590000000000001e-06 2024-09-17T21:31:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728240+0000", 2024-09-17T21:31:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022049804999999999 2024-09-17T21:31:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728307+0000", 2024-09-17T21:31:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7477999999999993e-05 2024-09-17T21:31:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:31:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6112999999999994e-05 2024-09-17T21:31:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45658 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.742113412999998, 2024-09-17T21:31:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022376455, 2024-09-17T21:31:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45658 2024-09-17T21:31:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:31:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706200+0000", 2024-09-17T21:31:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:31:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706206+0000", 2024-09-17T21:31:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7039999999999999e-06 2024-09-17T21:31:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706211+0000", 2024-09-17T21:31:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9429999999999999e-06 2024-09-17T21:31:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728418+0000", 2024-09-17T21:31:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022206760999999998 2024-09-17T21:31:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728497+0000", 2024-09-17T21:31:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9493999999999995e-05 2024-09-17T21:31:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728574+0000", 2024-09-17T21:31:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7180000000000003e-05 2024-09-17T21:31:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45707 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 212992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.740409347000003, 2024-09-17T21:31:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022235550999999999, 2024-09-17T21:31:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45707 2024-09-17T21:31:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:31:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707905+0000", 2024-09-17T21:31:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2120000000000002e-06 2024-09-17T21:31:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707911+0000", 2024-09-17T21:31:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1929999999999998e-06 2024-09-17T21:31:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707915+0000", 2024-09-17T21:31:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:31:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729897+0000", 2024-09-17T21:31:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021981186 2024-09-17T21:31:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729944+0000", 2024-09-17T21:31:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571000000000002e-05 2024-09-17T21:31:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730138+0000", 2024-09-17T21:31:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000193451 2024-09-17T21:31:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45708 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.740389215, 2024-09-17T21:31:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022533742999999998, 2024-09-17T21:31:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45708 2024-09-17T21:31:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707923+0000", 2024-09-17T21:31:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:31:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707932+0000", 2024-09-17T21:31:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6430000000000007e-06 2024-09-17T21:31:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707939+0000", 2024-09-17T21:31:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3539999999999996e-06 2024-09-17T21:31:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707944+0000", 2024-09-17T21:31:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3249999999999998e-06 2024-09-17T21:31:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730176+0000", 2024-09-17T21:31:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022232393 2024-09-17T21:31:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730263+0000", 2024-09-17T21:31:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6638999999999995e-05 2024-09-17T21:31:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730456+0000", 2024-09-17T21:31:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019304899999999999 2024-09-17T21:31:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45752 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.740133995000001, 2024-09-17T21:31:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022426878000000001, 2024-09-17T21:31:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45752 2024-09-17T21:31:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708178+0000", 2024-09-17T21:31:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:31:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708181+0000", 2024-09-17T21:31:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.909e-06 2024-09-17T21:31:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708187+0000", 2024-09-17T21:31:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4999999999999996e-06 2024-09-17T21:31:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708193+0000", 2024-09-17T21:31:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:31:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730484+0000", 2024-09-17T21:31:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022291765000000002 2024-09-17T21:31:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730541+0000", 2024-09-17T21:31:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6483000000000001e-05 2024-09-17T21:31:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730604+0000", 2024-09-17T21:31:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3429000000000005e-05 2024-09-17T21:31:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45761 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.740106085000001, 2024-09-17T21:31:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022614408999999999, 2024-09-17T21:31:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45761 2024-09-17T21:31:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708206+0000", 2024-09-17T21:31:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4700000000000001e-07 2024-09-17T21:31:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708208+0000", 2024-09-17T21:31:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1509999999999998e-06 2024-09-17T21:31:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708214+0000", 2024-09-17T21:31:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.49e-06 2024-09-17T21:31:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708219+0000", 2024-09-17T21:31:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8119999999999998e-06 2024-09-17T21:31:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730641+0000", 2024-09-17T21:31:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022422765000000001 2024-09-17T21:31:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730731+0000", 2024-09-17T21:31:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9598999999999994e-05 2024-09-17T21:31:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730820+0000", 2024-09-17T21:31:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8844999999999994e-05 2024-09-17T21:31:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45785 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.738884298000002, 2024-09-17T21:31:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022044113000000001, 2024-09-17T21:31:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45785 2024-09-17T21:31:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709428+0000", 2024-09-17T21:31:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:31:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709430+0000", 2024-09-17T21:31:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:31:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709451+0000", 2024-09-17T21:31:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1372000000000001e-05 2024-09-17T21:31:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709472+0000", 2024-09-17T21:31:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1007999999999999e-05 2024-09-17T21:31:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731293+0000", 2024-09-17T21:31:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021821320000000002 2024-09-17T21:31:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731378+0000", 2024-09-17T21:31:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5059e-05 2024-09-17T21:31:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731471+0000", 2024-09-17T21:31:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2867000000000004e-05 2024-09-17T21:31:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45786 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.738825501000001, 2024-09-17T21:31:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022181821000000001, 2024-09-17T21:31:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45786 2024-09-17T21:31:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709487+0000", 2024-09-17T21:31:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1299999999999999e-07 2024-09-17T21:31:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709491+0000", 2024-09-17T21:31:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2089999999999999e-06 2024-09-17T21:31:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709498+0000", 2024-09-17T21:31:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3980000000000002e-06 2024-09-17T21:31:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709504+0000", 2024-09-17T21:31:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7219999999999996e-06 2024-09-17T21:31:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731499+0000", 2024-09-17T21:31:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021995297 2024-09-17T21:31:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731591+0000", 2024-09-17T21:31:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1373999999999996e-05 2024-09-17T21:31:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731668+0000", 2024-09-17T21:31:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7107999999999994e-05 2024-09-17T21:31:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50202 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'139, mlcod=45'137)", 2024-09-17T21:31:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.324089632000003, 2024-09-17T21:31:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059659441000000001, 2024-09-17T21:31:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124224+0000", 2024-09-17T21:31:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5000000000000002e-06 2024-09-17T21:31:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147385+0000", 2024-09-17T21:31:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023160544000000002 2024-09-17T21:31:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147411+0000", 2024-09-17T21:31:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6095999999999999e-05 2024-09-17T21:31:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147438+0000", 2024-09-17T21:31:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6982999999999999e-05 2024-09-17T21:31:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147504+0000", 2024-09-17T21:31:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6156999999999997e-05 2024-09-17T21:31:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147531+0000", 2024-09-17T21:31:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6741999999999999e-05 2024-09-17T21:31:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183820+0000", 2024-09-17T21:31:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036289090000000003 2024-09-17T21:31:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183881+0000", 2024-09-17T21:31:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1328999999999995e-05 2024-09-17T21:31:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50203 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'141, mlcod=45'137)", 2024-09-17T21:31:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.286826888999997, 2024-09-17T21:31:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023138823999999999, 2024-09-17T21:31:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161486+0000", 2024-09-17T21:31:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5290000000000001e-06 2024-09-17T21:31:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161496+0000", 2024-09-17T21:31:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0063e-05 2024-09-17T21:31:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161514+0000", 2024-09-17T21:31:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7728e-05 2024-09-17T21:31:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161524+0000", 2024-09-17T21:31:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0339e-05 2024-09-17T21:31:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170872+0000", 2024-09-17T21:31:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093473640000000004 2024-09-17T21:31:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170899+0000", 2024-09-17T21:31:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7321000000000002e-05 2024-09-17T21:31:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184580+0000", 2024-09-17T21:31:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013680988999999999 2024-09-17T21:31:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184623+0000", 2024-09-17T21:31:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3491000000000001e-05 2024-09-17T21:31:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:88193 3.9 3:97498da5:::rbd_data.1120ea662e44.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-17T21:31:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.124645979, 2024-09-17T21:31:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026278261000000001, 2024-09-17T21:31:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 88193 2024-09-17T21:31:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323666+0000", 2024-09-17T21:31:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:31:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323670+0000", 2024-09-17T21:31:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.771e-06 2024-09-17T21:31:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323680+0000", 2024-09-17T21:31:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9469999999999998e-06 2024-09-17T21:31:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323685+0000", 2024-09-17T21:31:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9919999999999998e-06 2024-09-17T21:31:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323728+0000", 2024-09-17T21:31:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2811000000000001e-05 2024-09-17T21:31:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323776+0000", 2024-09-17T21:31:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8310000000000003e-05 2024-09-17T21:31:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.328583+0000", 2024-09-17T21:31:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048072899999999997 2024-09-17T21:31:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349861+0000", 2024-09-17T21:31:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021277695999999999 2024-09-17T21:31:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349877+0000", 2024-09-17T21:31:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:31:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349911+0000", 2024-09-17T21:31:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3844999999999999e-05 2024-09-17T21:31:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349944+0000", 2024-09-17T21:31:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2663000000000002e-05 2024-09-17T21:31:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:101384 3.9 3:90f2a32a:::rbd_data.11206b8c0882.0000000000000178:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T21:31:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.12177638, 2024-09-17T21:31:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025286194000000001, 2024-09-17T21:31:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 101384 2024-09-17T21:31:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326536+0000", 2024-09-17T21:31:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4799999999999997e-07 2024-09-17T21:31:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326541+0000", 2024-09-17T21:31:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8489999999999997e-06 2024-09-17T21:31:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326552+0000", 2024-09-17T21:31:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0912e-05 2024-09-17T21:31:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326557+0000", 2024-09-17T21:31:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0359999999999997e-06 2024-09-17T21:31:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326603+0000", 2024-09-17T21:31:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6428999999999999e-05 2024-09-17T21:31:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326686+0000", 2024-09-17T21:31:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2521999999999994e-05 2024-09-17T21:31:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.332698+0000", 2024-09-17T21:31:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060124599999999999 2024-09-17T21:31:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351744+0000", 2024-09-17T21:31:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019045838999999998 2024-09-17T21:31:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351754+0000", 2024-09-17T21:31:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0380999999999999e-05 2024-09-17T21:31:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351791+0000", 2024-09-17T21:31:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6678999999999999e-05 2024-09-17T21:31:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351821+0000", 2024-09-17T21:31:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0139000000000001e-05 2024-09-17T21:31:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:07.545 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:07.546 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:31:07.547 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:07.544+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:07.647 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:07.644+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:07.689 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:07.690 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:07.690 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:07.690 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:07.698 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:31:07.782 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:07.780+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:07.841 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:07.841 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:07.841 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:07.842 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:07.842 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:07.849 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:07.847+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:07.850 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:31:07.950 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:07.948+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:07.995 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:07.995 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:07.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:07.996 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:07.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.996 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10799 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:07.996 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:07.996 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.927024161000006, 2024-09-17T21:31:07.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039122745, 2024-09-17T21:31:07.996 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:07.996 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:07.996 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:07.996 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:07.996 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:07.996 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:07.996 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10799 2024-09-17T21:31:07.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.996 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:07.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:07.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:07.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:07.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:07.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:07.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:07.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065469+0000", 2024-09-17T21:31:07.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:31:07.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:07.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065477+0000", 2024-09-17T21:31:07.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1319999999999994e-06 2024-09-17T21:31:07.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:07.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065483+0000", 2024-09-17T21:31:07.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2040000000000002e-06 2024-09-17T21:31:07.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:07.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065487+0000", 2024-09-17T21:31:07.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.42e-06 2024-09-17T21:31:07.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:07.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099984+0000", 2024-09-17T21:31:07.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034496484000000001 2024-09-17T21:31:07.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:07.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100065+0000", 2024-09-17T21:31:07.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1200999999999997e-05 2024-09-17T21:31:07.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:07.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:07.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:08.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100422+0000", 2024-09-17T21:31:08.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035657700000000001 2024-09-17T21:31:08.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:08.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104214+0000", 2024-09-17T21:31:08.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003792045 2024-09-17T21:31:08.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:08.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104505+0000", 2024-09-17T21:31:08.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029141399999999999 2024-09-17T21:31:08.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104556+0000", 2024-09-17T21:31:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1388999999999999e-05 2024-09-17T21:31:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104591+0000", 2024-09-17T21:31:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4455999999999997e-05 2024-09-17T21:31:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12187 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.852931429999998, 2024-09-17T21:31:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038825750999999999, 2024-09-17T21:31:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12187 2024-09-17T21:31:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:31:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139563+0000", 2024-09-17T21:31:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.624e-06 2024-09-17T21:31:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139569+0000", 2024-09-17T21:31:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1090000000000003e-06 2024-09-17T21:31:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139573+0000", 2024-09-17T21:31:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4429999999999996e-06 2024-09-17T21:31:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178206+0000", 2024-09-17T21:31:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038632376000000003 2024-09-17T21:31:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178282+0000", 2024-09-17T21:31:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6076999999999997e-05 2024-09-17T21:31:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178387+0000", 2024-09-17T21:31:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104682 2024-09-17T21:31:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12188 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.852913248999997, 2024-09-17T21:31:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039053826999999999, 2024-09-17T21:31:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12188 2024-09-17T21:31:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139746+0000", 2024-09-17T21:31:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000167281 2024-09-17T21:31:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139749+0000", 2024-09-17T21:31:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9660000000000001e-06 2024-09-17T21:31:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139757+0000", 2024-09-17T21:31:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8029999999999997e-06 2024-09-17T21:31:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139771+0000", 2024-09-17T21:31:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3541e-05 2024-09-17T21:31:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178415+0000", 2024-09-17T21:31:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038644803999999998 2024-09-17T21:31:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178517+0000", 2024-09-17T21:31:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101298 2024-09-17T21:31:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178633+0000", 2024-09-17T21:31:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116134 2024-09-17T21:31:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12189 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.852712772000004, 2024-09-17T21:31:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039135991000000002, 2024-09-17T21:31:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12189 2024-09-17T21:31:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139780+0000", 2024-09-17T21:31:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:31:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139782+0000", 2024-09-17T21:31:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1950000000000002e-06 2024-09-17T21:31:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139788+0000", 2024-09-17T21:31:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0639999999999999e-06 2024-09-17T21:31:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139792+0000", 2024-09-17T21:31:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1960000000000001e-06 2024-09-17T21:31:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178671+0000", 2024-09-17T21:31:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038878176 2024-09-17T21:31:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178778+0000", 2024-09-17T21:31:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107278 2024-09-17T21:31:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178915+0000", 2024-09-17T21:31:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013757900000000001 2024-09-17T21:31:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12190 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.852693588999998, 2024-09-17T21:31:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039343781000000001, 2024-09-17T21:31:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12190 2024-09-17T21:31:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139801+0000", 2024-09-17T21:31:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.916e-06 2024-09-17T21:31:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139807+0000", 2024-09-17T21:31:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4540000000000004e-06 2024-09-17T21:31:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139812+0000", 2024-09-17T21:31:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9729999999999997e-06 2024-09-17T21:31:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178960+0000", 2024-09-17T21:31:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039147567000000001 2024-09-17T21:31:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179051+0000", 2024-09-17T21:31:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0754000000000006e-05 2024-09-17T21:31:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179142+0000", 2024-09-17T21:31:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1712999999999994e-05 2024-09-17T21:31:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12191 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.852673730999996, 2024-09-17T21:31:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039464336000000003, 2024-09-17T21:31:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12191 2024-09-17T21:31:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:31:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139821+0000", 2024-09-17T21:31:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1260000000000001e-06 2024-09-17T21:31:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139827+0000", 2024-09-17T21:31:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T21:31:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139840+0000", 2024-09-17T21:31:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2607e-05 2024-09-17T21:31:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179162+0000", 2024-09-17T21:31:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039322863999999999 2024-09-17T21:31:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179204+0000", 2024-09-17T21:31:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1938999999999999e-05 2024-09-17T21:31:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179283+0000", 2024-09-17T21:31:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8547000000000005e-05 2024-09-17T21:31:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12192 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.852646019000005, 2024-09-17T21:31:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039652318999999998, 2024-09-17T21:31:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12192 2024-09-17T21:31:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141458+0000", 2024-09-17T21:31:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001612098 2024-09-17T21:31:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141469+0000", 2024-09-17T21:31:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-17T21:31:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141478+0000", 2024-09-17T21:31:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T21:31:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141485+0000", 2024-09-17T21:31:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4760000000000003e-06 2024-09-17T21:31:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179317+0000", 2024-09-17T21:31:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037831832000000003 2024-09-17T21:31:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179395+0000", 2024-09-17T21:31:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8349000000000005e-05 2024-09-17T21:31:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179499+0000", 2024-09-17T21:31:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103628 2024-09-17T21:31:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12257 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.850457211000005, 2024-09-17T21:31:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038862751000000001, 2024-09-17T21:31:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12257 2024-09-17T21:31:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T21:31:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142037+0000", 2024-09-17T21:31:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.03e-06 2024-09-17T21:31:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142043+0000", 2024-09-17T21:31:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6319999999999996e-06 2024-09-17T21:31:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142052+0000", 2024-09-17T21:31:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2709999999999998e-06 2024-09-17T21:31:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180794+0000", 2024-09-17T21:31:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038741758000000001 2024-09-17T21:31:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180843+0000", 2024-09-17T21:31:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9021999999999999e-05 2024-09-17T21:31:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180898+0000", 2024-09-17T21:31:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673000000000003e-05 2024-09-17T21:31:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12258 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.850433694000003, 2024-09-17T21:31:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038950717000000003, 2024-09-17T21:31:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12258 2024-09-17T21:31:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199999999999999e-07 2024-09-17T21:31:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142061+0000", 2024-09-17T21:31:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8279999999999999e-06 2024-09-17T21:31:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142067+0000", 2024-09-17T21:31:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5600000000000001e-06 2024-09-17T21:31:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142071+0000", 2024-09-17T21:31:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6240000000000001e-06 2024-09-17T21:31:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180921+0000", 2024-09-17T21:31:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038849366000000003 2024-09-17T21:31:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180961+0000", 2024-09-17T21:31:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0219000000000002e-05 2024-09-17T21:31:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181009+0000", 2024-09-17T21:31:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8437999999999999e-05 2024-09-17T21:31:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12259 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.850406446999997, 2024-09-17T21:31:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039116832999999997, 2024-09-17T21:31:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12259 2024-09-17T21:31:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2200000000000004e-07 2024-09-17T21:31:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142088+0000", 2024-09-17T21:31:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0329999999999998e-06 2024-09-17T21:31:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142095+0000", 2024-09-17T21:31:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4320000000000004e-06 2024-09-17T21:31:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142100+0000", 2024-09-17T21:31:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:31:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181037+0000", 2024-09-17T21:31:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038937601000000002 2024-09-17T21:31:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181114+0000", 2024-09-17T21:31:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6857e-05 2024-09-17T21:31:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181203+0000", 2024-09-17T21:31:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8437999999999996e-05 2024-09-17T21:31:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12260 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.850386159999999, 2024-09-17T21:31:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039204184000000003, 2024-09-17T21:31:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12260 2024-09-17T21:31:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142107+0000", 2024-09-17T21:31:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:31:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142108+0000", 2024-09-17T21:31:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.753e-06 2024-09-17T21:31:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142114+0000", 2024-09-17T21:31:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7660000000000004e-06 2024-09-17T21:31:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142119+0000", 2024-09-17T21:31:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:31:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181216+0000", 2024-09-17T21:31:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039097191000000003 2024-09-17T21:31:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181260+0000", 2024-09-17T21:31:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4731000000000003e-05 2024-09-17T21:31:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181310+0000", 2024-09-17T21:31:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9818000000000003e-05 2024-09-17T21:31:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12261 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.850367836000004, 2024-09-17T21:31:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039292200999999999, 2024-09-17T21:31:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12261 2024-09-17T21:31:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142125+0000", 2024-09-17T21:31:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:31:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142127+0000", 2024-09-17T21:31:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.792e-06 2024-09-17T21:31:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142132+0000", 2024-09-17T21:31:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3979999999999998e-06 2024-09-17T21:31:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142136+0000", 2024-09-17T21:31:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.211e-06 2024-09-17T21:31:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181324+0000", 2024-09-17T21:31:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039187560000000003 2024-09-17T21:31:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181363+0000", 2024-09-17T21:31:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9375000000000002e-05 2024-09-17T21:31:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181417+0000", 2024-09-17T21:31:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3372000000000003e-05 2024-09-17T21:31:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12262 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.850350043000006, 2024-09-17T21:31:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039384875, 2024-09-17T21:31:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12262 2024-09-17T21:31:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142143+0000", 2024-09-17T21:31:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:31:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142148+0000", 2024-09-17T21:31:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4360000000000001e-06 2024-09-17T21:31:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142154+0000", 2024-09-17T21:31:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8719999999999999e-06 2024-09-17T21:31:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142158+0000", 2024-09-17T21:31:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.476e-06 2024-09-17T21:31:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181436+0000", 2024-09-17T21:31:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039277919000000001 2024-09-17T21:31:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181480+0000", 2024-09-17T21:31:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3649999999999997e-05 2024-09-17T21:31:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181527+0000", 2024-09-17T21:31:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7131e-05 2024-09-17T21:31:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12263 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1079296~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.850327608000001, 2024-09-17T21:31:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039473339000000003, 2024-09-17T21:31:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12263 2024-09-17T21:31:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:31:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142167+0000", 2024-09-17T21:31:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.81e-06 2024-09-17T21:31:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142172+0000", 2024-09-17T21:31:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:31:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142177+0000", 2024-09-17T21:31:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T21:31:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181553+0000", 2024-09-17T21:31:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039376130000000002 2024-09-17T21:31:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181592+0000", 2024-09-17T21:31:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9153999999999998e-05 2024-09-17T21:31:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181638+0000", 2024-09-17T21:31:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5864e-05 2024-09-17T21:31:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12278 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1095680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.850309632999995, 2024-09-17T21:31:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039454222999999997, 2024-09-17T21:31:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12278 2024-09-17T21:31:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:31:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142185+0000", 2024-09-17T21:31:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.697e-06 2024-09-17T21:31:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142191+0000", 2024-09-17T21:31:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:31:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142196+0000", 2024-09-17T21:31:08.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4829999999999999e-06 2024-09-17T21:31:08.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181548+0000", 2024-09-17T21:31:08.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039351200000000003 2024-09-17T21:31:08.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181591+0000", 2024-09-17T21:31:08.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3492000000000003e-05 2024-09-17T21:31:08.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181637+0000", 2024-09-17T21:31:08.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5847000000000003e-05 2024-09-17T21:31:08.060 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.060 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.060 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.060 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12279 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.060 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:08.060 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.850286343999997, 2024-09-17T21:31:08.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039539801999999999, 2024-09-17T21:31:08.060 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.060 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.060 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.061 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.061 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.061 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.061 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12279 2024-09-17T21:31:08.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.061 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:08.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:08.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:08.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.89e-07 2024-09-17T21:31:08.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142208+0000", 2024-09-17T21:31:08.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7579999999999999e-06 2024-09-17T21:31:08.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142214+0000", 2024-09-17T21:31:08.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:31:08.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142218+0000", 2024-09-17T21:31:08.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3429999999999998e-06 2024-09-17T21:31:08.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181653+0000", 2024-09-17T21:31:08.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039435322000000002 2024-09-17T21:31:08.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181698+0000", 2024-09-17T21:31:08.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4978999999999997e-05 2024-09-17T21:31:08.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181746+0000", 2024-09-17T21:31:08.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571999999999998e-05 2024-09-17T21:31:08.064 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.064 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.064 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.064 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12280 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1124352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.064 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:08.064 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.850268825000001, 2024-09-17T21:31:08.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039704358000000002, 2024-09-17T21:31:08.065 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.065 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.065 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.065 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.065 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.065 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.065 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12280 2024-09-17T21:31:08.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.065 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:08.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:08.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142224+0000", 2024-09-17T21:31:08.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:31:08.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142226+0000", 2024-09-17T21:31:08.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:31:08.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142231+0000", 2024-09-17T21:31:08.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:31:08.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142240+0000", 2024-09-17T21:31:08.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4590000000000004e-06 2024-09-17T21:31:08.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181774+0000", 2024-09-17T21:31:08.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039533833999999997 2024-09-17T21:31:08.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181859+0000", 2024-09-17T21:31:08.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4328999999999998e-05 2024-09-17T21:31:08.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181928+0000", 2024-09-17T21:31:08.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9139000000000003e-05 2024-09-17T21:31:08.068 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.068 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.068 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.068 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12281 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1136640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.068 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:08.068 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.850245919000002, 2024-09-17T21:31:08.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039787799999999998, 2024-09-17T21:31:08.069 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.069 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.069 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.069 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12281 2024-09-17T21:31:08.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.072 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:08.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:08.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142309+0000", 2024-09-17T21:31:08.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2323999999999995e-05 2024-09-17T21:31:08.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142317+0000", 2024-09-17T21:31:08.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1859999999999992e-06 2024-09-17T21:31:08.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142324+0000", 2024-09-17T21:31:08.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4699999999999996e-06 2024-09-17T21:31:08.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142330+0000", 2024-09-17T21:31:08.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4650000000000002e-06 2024-09-17T21:31:08.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181948+0000", 2024-09-17T21:31:08.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039618056999999998 2024-09-17T21:31:08.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181986+0000", 2024-09-17T21:31:08.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8442999999999998e-05 2024-09-17T21:31:08.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182034+0000", 2024-09-17T21:31:08.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7855000000000002e-05 2024-09-17T21:31:08.075 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.075 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.075 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.075 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12282 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1150976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.075 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:08.075 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.850153689999999, 2024-09-17T21:31:08.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039871919999999998, 2024-09-17T21:31:08.075 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.075 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.075 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.075 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.075 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.076 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.076 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12282 2024-09-17T21:31:08.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.076 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:08.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:08.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:08.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:31:08.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142341+0000", 2024-09-17T21:31:08.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.232e-06 2024-09-17T21:31:08.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142348+0000", 2024-09-17T21:31:08.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7320000000000001e-06 2024-09-17T21:31:08.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142353+0000", 2024-09-17T21:31:08.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:31:08.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182062+0000", 2024-09-17T21:31:08.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039709739000000001 2024-09-17T21:31:08.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182128+0000", 2024-09-17T21:31:08.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5739000000000002e-05 2024-09-17T21:31:08.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182210+0000", 2024-09-17T21:31:08.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:31:08.079 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.079 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.079 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.079 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12283 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:08.079 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:08.080 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.850133279000005, 2024-09-17T21:31:08.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039896434000000001, 2024-09-17T21:31:08.080 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.080 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.080 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:31:08.080 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.080 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.080 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.080 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12283 2024-09-17T21:31:08.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.080 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:08.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:08.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144852+0000", 2024-09-17T21:31:08.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024934079999999999 2024-09-17T21:31:08.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144856+0000", 2024-09-17T21:31:08.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:31:08.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144864+0000", 2024-09-17T21:31:08.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0160000000000008e-06 2024-09-17T21:31:08.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144869+0000", 2024-09-17T21:31:08.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:31:08.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182246+0000", 2024-09-17T21:31:08.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037377207000000003 2024-09-17T21:31:08.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182255+0000", 2024-09-17T21:31:08.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1840000000000005e-06 2024-09-17T21:31:08.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.083 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.083 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.083 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:08.085 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:31:08.085 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:08.083+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:08.186 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:08.184+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:08.229 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:08.229 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:08.229 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:08.229 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:08.237 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:31:08.287 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:08.285+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:08.341 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:08.341 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:08.341 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:08.341 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:08.341 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:08.350 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:31:08.391 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:08.389+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:08.489 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:08.487+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:08.508 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:08.508 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:08.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:36935 3.a 3:54c8759b:::rbd_data.1120e62eea89.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-17T21:31:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:31:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 71.546906652000004, 2024-09-17T21:31:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020540151999999999, 2024-09-17T21:31:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 36935 2024-09-17T21:31:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:31:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:31:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958910+0000", 2024-09-17T21:31:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:31:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958918+0000", 2024-09-17T21:31:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6469999999999998e-06 2024-09-17T21:31:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958933+0000", 2024-09-17T21:31:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4791e-05 2024-09-17T21:31:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958940+0000", 2024-09-17T21:31:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-17T21:31:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958990+0000", 2024-09-17T21:31:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9759000000000003e-05 2024-09-17T21:31:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.959039+0000", 2024-09-17T21:31:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9487000000000002e-05 2024-09-17T21:31:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.962098+0000", 2024-09-17T21:31:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003059258 2024-09-17T21:31:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979325+0000", 2024-09-17T21:31:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017226841 2024-09-17T21:31:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979343+0000", 2024-09-17T21:31:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7486000000000002e-05 2024-09-17T21:31:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979401+0000", 2024-09-17T21:31:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7980999999999999e-05 2024-09-17T21:31:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979449+0000", 2024-09-17T21:31:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8639e-05 2024-09-17T21:31:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39447 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 909312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537968751999998, 2024-09-17T21:31:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017419153999999999, 2024-09-17T21:31:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39447 2024-09-17T21:31:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:31:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967849+0000", 2024-09-17T21:31:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4339999999999999e-06 2024-09-17T21:31:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967853+0000", 2024-09-17T21:31:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:31:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967857+0000", 2024-09-17T21:31:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:31:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985193+0000", 2024-09-17T21:31:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017335973000000001 2024-09-17T21:31:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985228+0000", 2024-09-17T21:31:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5154999999999998e-05 2024-09-17T21:31:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985266+0000", 2024-09-17T21:31:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8448000000000002e-05 2024-09-17T21:31:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39448 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 974848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537955936000003, 2024-09-17T21:31:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017517066000000001, 2024-09-17T21:31:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39448 2024-09-17T21:31:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:08.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.72e-07 2024-09-17T21:31:08.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967861+0000", 2024-09-17T21:31:08.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.096e-06 2024-09-17T21:31:08.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967865+0000", 2024-09-17T21:31:08.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9339999999999999e-06 2024-09-17T21:31:08.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967868+0000", 2024-09-17T21:31:08.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.508e-06 2024-09-17T21:31:08.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985285+0000", 2024-09-17T21:31:08.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017417208 2024-09-17T21:31:08.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985339+0000", 2024-09-17T21:31:08.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3931000000000003e-05 2024-09-17T21:31:08.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985377+0000", 2024-09-17T21:31:08.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8016999999999999e-05 2024-09-17T21:31:08.523 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.523 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.523 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.523 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39449 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.523 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:08.523 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537943826000003, 2024-09-17T21:31:08.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017595137, 2024-09-17T21:31:08.523 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.523 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.523 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39449 2024-09-17T21:31:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2700000000000001e-07 2024-09-17T21:31:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967873+0000", 2024-09-17T21:31:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.164e-06 2024-09-17T21:31:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967877+0000", 2024-09-17T21:31:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.23e-06 2024-09-17T21:31:08.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967880+0000", 2024-09-17T21:31:08.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.286e-06 2024-09-17T21:31:08.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985397+0000", 2024-09-17T21:31:08.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017516982 2024-09-17T21:31:08.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985431+0000", 2024-09-17T21:31:08.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3899999999999997e-05 2024-09-17T21:31:08.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985467+0000", 2024-09-17T21:31:08.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6347999999999998e-05 2024-09-17T21:31:08.527 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.527 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.527 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.527 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39450 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1105920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.527 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:31:08.527 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537923569, 2024-09-17T21:31:08.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017677824000000002, 2024-09-17T21:31:08.527 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.528 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.528 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.528 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.528 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.528 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.528 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39450 2024-09-17T21:31:08.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.528 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:31:08.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:31:08.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967893+0000", 2024-09-17T21:31:08.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9100000000000004e-07 2024-09-17T21:31:08.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967895+0000", 2024-09-17T21:31:08.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:31:08.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967901+0000", 2024-09-17T21:31:08.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.083e-06 2024-09-17T21:31:08.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967906+0000", 2024-09-17T21:31:08.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:31:08.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985493+0000", 2024-09-17T21:31:08.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017587565999999999 2024-09-17T21:31:08.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985531+0000", 2024-09-17T21:31:08.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7490000000000002e-05 2024-09-17T21:31:08.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985570+0000", 2024-09-17T21:31:08.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9205000000000002e-05 2024-09-17T21:31:08.531 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.531 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.531 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.531 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39451 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1171456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.531 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:08.531 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537903189000005, 2024-09-17T21:31:08.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017752337999999999, 2024-09-17T21:31:08.532 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.532 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.532 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.532 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.532 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.532 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.532 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39451 2024-09-17T21:31:08.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.532 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:08.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:08.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:08.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:31:08.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967915+0000", 2024-09-17T21:31:08.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780000000000001e-06 2024-09-17T21:31:08.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967922+0000", 2024-09-17T21:31:08.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7020000000000004e-06 2024-09-17T21:31:08.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967926+0000", 2024-09-17T21:31:08.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.809e-06 2024-09-17T21:31:08.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985590+0000", 2024-09-17T21:31:08.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017663913999999999 2024-09-17T21:31:08.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985627+0000", 2024-09-17T21:31:08.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6896e-05 2024-09-17T21:31:08.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985665+0000", 2024-09-17T21:31:08.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7793999999999998e-05 2024-09-17T21:31:08.535 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.535 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.535 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.536 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39452 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1236992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.536 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:08.536 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537883246999996, 2024-09-17T21:31:08.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017829602999999999, 2024-09-17T21:31:08.536 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.536 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.536 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.536 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39452 2024-09-17T21:31:08.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.537 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:08.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:08.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:08.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:31:08.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967947+0000", 2024-09-17T21:31:08.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4280999999999999e-05 2024-09-17T21:31:08.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967954+0000", 2024-09-17T21:31:08.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3249999999999996e-06 2024-09-17T21:31:08.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967959+0000", 2024-09-17T21:31:08.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2299999999999999e-06 2024-09-17T21:31:08.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985687+0000", 2024-09-17T21:31:08.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017727844999999999 2024-09-17T21:31:08.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985725+0000", 2024-09-17T21:31:08.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8243e-05 2024-09-17T21:31:08.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985762+0000", 2024-09-17T21:31:08.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7023000000000001e-05 2024-09-17T21:31:08.540 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.540 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.540 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.540 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39453 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.540 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:31:08.540 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537846759000004, 2024-09-17T21:31:08.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017893793000000002, 2024-09-17T21:31:08.540 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.540 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.540 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.540 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.540 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.541 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39453 2024-09-17T21:31:08.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.541 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:31:08.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:31:08.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967970+0000", 2024-09-17T21:31:08.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:31:08.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967972+0000", 2024-09-17T21:31:08.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0229999999999999e-06 2024-09-17T21:31:08.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967978+0000", 2024-09-17T21:31:08.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0430000000000004e-06 2024-09-17T21:31:08.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967982+0000", 2024-09-17T21:31:08.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2570000000000001e-06 2024-09-17T21:31:08.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985782+0000", 2024-09-17T21:31:08.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017799683 2024-09-17T21:31:08.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985818+0000", 2024-09-17T21:31:08.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6304999999999999e-05 2024-09-17T21:31:08.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985863+0000", 2024-09-17T21:31:08.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4953999999999997e-05 2024-09-17T21:31:08.544 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.544 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.544 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.544 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39454 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1368064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.544 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:31:08.544 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537827430999997, 2024-09-17T21:31:08.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017968886, 2024-09-17T21:31:08.544 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.544 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.544 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.545 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.545 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.545 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.545 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39454 2024-09-17T21:31:08.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.545 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:31:08.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:31:08.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967989+0000", 2024-09-17T21:31:08.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.15e-07 2024-09-17T21:31:08.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967991+0000", 2024-09-17T21:31:08.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.945e-06 2024-09-17T21:31:08.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967997+0000", 2024-09-17T21:31:08.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0190000000000003e-06 2024-09-17T21:31:08.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968001+0000", 2024-09-17T21:31:08.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6650000000000002e-06 2024-09-17T21:31:08.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985883+0000", 2024-09-17T21:31:08.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017881971999999999 2024-09-17T21:31:08.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985920+0000", 2024-09-17T21:31:08.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-05 2024-09-17T21:31:08.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985957+0000", 2024-09-17T21:31:08.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7370000000000003e-05 2024-09-17T21:31:08.548 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.548 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.548 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.548 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39455 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.549 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:08.549 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537808256999995, 2024-09-17T21:31:08.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018044437999999999, 2024-09-17T21:31:08.549 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.549 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.549 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.549 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.549 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.549 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.549 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39455 2024-09-17T21:31:08.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.549 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:08.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:08.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:08.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:31:08.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968010+0000", 2024-09-17T21:31:08.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.886e-06 2024-09-17T21:31:08.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968017+0000", 2024-09-17T21:31:08.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:31:08.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968021+0000", 2024-09-17T21:31:08.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850000000000004e-06 2024-09-17T21:31:08.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985977+0000", 2024-09-17T21:31:08.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017956368 2024-09-17T21:31:08.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986015+0000", 2024-09-17T21:31:08.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7610000000000001e-05 2024-09-17T21:31:08.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986052+0000", 2024-09-17T21:31:08.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7109e-05 2024-09-17T21:31:08.552 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.553 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.553 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.553 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39456 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1499136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.553 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:08.553 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537787957000006, 2024-09-17T21:31:08.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018114514000000002, 2024-09-17T21:31:08.553 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.553 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.553 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.553 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.553 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.554 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39456 2024-09-17T21:31:08.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.554 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:08.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:08.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:08.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-17T21:31:08.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968030+0000", 2024-09-17T21:31:08.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0049999999999999e-06 2024-09-17T21:31:08.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968036+0000", 2024-09-17T21:31:08.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0129999999999999e-06 2024-09-17T21:31:08.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968041+0000", 2024-09-17T21:31:08.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8300000000000003e-06 2024-09-17T21:31:08.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986071+0000", 2024-09-17T21:31:08.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018029620999999999 2024-09-17T21:31:08.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986106+0000", 2024-09-17T21:31:08.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4867999999999999e-05 2024-09-17T21:31:08.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986142+0000", 2024-09-17T21:31:08.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6566000000000002e-05 2024-09-17T21:31:08.557 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.557 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.557 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.558 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39457 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1564672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.558 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:08.558 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537768141000001, 2024-09-17T21:31:08.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018188644, 2024-09-17T21:31:08.558 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.558 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.558 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.558 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39457 2024-09-17T21:31:08.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.558 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:08.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:08.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:08.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7199999999999999e-07 2024-09-17T21:31:08.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968050+0000", 2024-09-17T21:31:08.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1110000000000002e-06 2024-09-17T21:31:08.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968057+0000", 2024-09-17T21:31:08.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6170000000000003e-06 2024-09-17T21:31:08.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968062+0000", 2024-09-17T21:31:08.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:31:08.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986163+0000", 2024-09-17T21:31:08.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018101435999999999 2024-09-17T21:31:08.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986199+0000", 2024-09-17T21:31:08.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.591e-05 2024-09-17T21:31:08.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986236+0000", 2024-09-17T21:31:08.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7416999999999998e-05 2024-09-17T21:31:08.562 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.562 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.562 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.562 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39458 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1630208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.562 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:08.562 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537741201000003, 2024-09-17T21:31:08.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018255856000000001, 2024-09-17T21:31:08.562 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.562 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.562 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.562 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.563 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.563 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.563 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39458 2024-09-17T21:31:08.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.563 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:08.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:08.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:08.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:31:08.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968077+0000", 2024-09-17T21:31:08.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.779e-06 2024-09-17T21:31:08.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968081+0000", 2024-09-17T21:31:08.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.532e-06 2024-09-17T21:31:08.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968085+0000", 2024-09-17T21:31:08.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999999e-06 2024-09-17T21:31:08.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986256+0000", 2024-09-17T21:31:08.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018171273000000002 2024-09-17T21:31:08.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986293+0000", 2024-09-17T21:31:08.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6534e-05 2024-09-17T21:31:08.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986330+0000", 2024-09-17T21:31:08.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7536000000000001e-05 2024-09-17T21:31:08.566 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.566 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.566 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.566 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39459 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1695744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.566 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:31:08.567 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537726622999998, 2024-09-17T21:31:08.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334493, 2024-09-17T21:31:08.567 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.567 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.567 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.567 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39459 2024-09-17T21:31:08.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.567 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:31:08.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:31:08.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968090+0000", 2024-09-17T21:31:08.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:31:08.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968091+0000", 2024-09-17T21:31:08.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3319999999999999e-06 2024-09-17T21:31:08.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968094+0000", 2024-09-17T21:31:08.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2590000000000001e-06 2024-09-17T21:31:08.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968097+0000", 2024-09-17T21:31:08.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:31:08.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986350+0000", 2024-09-17T21:31:08.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018253364000000001 2024-09-17T21:31:08.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986387+0000", 2024-09-17T21:31:08.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6350999999999998e-05 2024-09-17T21:31:08.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986424+0000", 2024-09-17T21:31:08.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6897999999999998e-05 2024-09-17T21:31:08.570 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.570 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.571 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.571 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39460 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1761280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.571 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:31:08.571 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537715487, 2024-09-17T21:31:08.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018415918999999999, 2024-09-17T21:31:08.571 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.571 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.571 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.571 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.571 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.571 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.571 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39460 2024-09-17T21:31:08.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.572 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:31:08.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:31:08.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968101+0000", 2024-09-17T21:31:08.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T21:31:08.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968102+0000", 2024-09-17T21:31:08.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1689999999999999e-06 2024-09-17T21:31:08.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968105+0000", 2024-09-17T21:31:08.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:31:08.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968108+0000", 2024-09-17T21:31:08.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6630000000000002e-06 2024-09-17T21:31:08.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986442+0000", 2024-09-17T21:31:08.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334487 2024-09-17T21:31:08.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986479+0000", 2024-09-17T21:31:08.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6557000000000003e-05 2024-09-17T21:31:08.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986516+0000", 2024-09-17T21:31:08.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7648999999999998e-05 2024-09-17T21:31:08.575 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.575 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.575 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.575 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39461 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1826816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.575 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:08.575 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537704727000005, 2024-09-17T21:31:08.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018499768, 2024-09-17T21:31:08.576 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.576 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.576 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.576 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39461 2024-09-17T21:31:08.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.576 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:08.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:08.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:08.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4699999999999998e-07 2024-09-17T21:31:08.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968112+0000", 2024-09-17T21:31:08.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-06 2024-09-17T21:31:08.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968116+0000", 2024-09-17T21:31:08.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1080000000000001e-06 2024-09-17T21:31:08.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968122+0000", 2024-09-17T21:31:08.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1070000000000001e-06 2024-09-17T21:31:08.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986537+0000", 2024-09-17T21:31:08.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018414834000000001 2024-09-17T21:31:08.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986572+0000", 2024-09-17T21:31:08.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5726000000000003e-05 2024-09-17T21:31:08.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986611+0000", 2024-09-17T21:31:08.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8584000000000002e-05 2024-09-17T21:31:08.579 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.579 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.579 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.579 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39462 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1892352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.579 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:31:08.580 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537690362999996, 2024-09-17T21:31:08.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018583252000000001, 2024-09-17T21:31:08.580 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.580 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.580 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.580 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.580 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.580 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.580 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39462 2024-09-17T21:31:08.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.580 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:31:08.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:31:08.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968126+0000", 2024-09-17T21:31:08.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:08.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968127+0000", 2024-09-17T21:31:08.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.229e-06 2024-09-17T21:31:08.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968131+0000", 2024-09-17T21:31:08.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8010000000000001e-06 2024-09-17T21:31:08.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968134+0000", 2024-09-17T21:31:08.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8880000000000001e-06 2024-09-17T21:31:08.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986634+0000", 2024-09-17T21:31:08.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018500501999999999 2024-09-17T21:31:08.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986671+0000", 2024-09-17T21:31:08.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6881000000000001e-05 2024-09-17T21:31:08.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986709+0000", 2024-09-17T21:31:08.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7546999999999998e-05 2024-09-17T21:31:08.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.583 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.584 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39463 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1957888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.584 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:08.584 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537678748000005, 2024-09-17T21:31:08.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018667186999999998, 2024-09-17T21:31:08.584 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.584 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.584 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.584 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.584 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.584 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.584 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39463 2024-09-17T21:31:08.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.584 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:08.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:08.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:08.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:31:08.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968138+0000", 2024-09-17T21:31:08.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:31:08.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968142+0000", 2024-09-17T21:31:08.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.145e-06 2024-09-17T21:31:08.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968144+0000", 2024-09-17T21:31:08.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6869999999999999e-06 2024-09-17T21:31:08.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986728+0000", 2024-09-17T21:31:08.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018584163000000001 2024-09-17T21:31:08.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986766+0000", 2024-09-17T21:31:08.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7987000000000001e-05 2024-09-17T21:31:08.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986804+0000", 2024-09-17T21:31:08.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7892000000000002e-05 2024-09-17T21:31:08.588 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.588 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.588 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.588 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39464 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.588 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:31:08.588 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537668439000001, 2024-09-17T21:31:08.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018753723999999999, 2024-09-17T21:31:08.588 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.588 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.588 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.588 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.589 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.589 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.589 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39464 2024-09-17T21:31:08.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.589 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:31:08.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:31:08.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968148+0000", 2024-09-17T21:31:08.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:31:08.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968152+0000", 2024-09-17T21:31:08.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3749999999999996e-06 2024-09-17T21:31:08.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968155+0000", 2024-09-17T21:31:08.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.371e-06 2024-09-17T21:31:08.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968158+0000", 2024-09-17T21:31:08.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T21:31:08.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986824+0000", 2024-09-17T21:31:08.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018665779 2024-09-17T21:31:08.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986863+0000", 2024-09-17T21:31:08.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9273000000000002e-05 2024-09-17T21:31:08.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986901+0000", 2024-09-17T21:31:08.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7978000000000001e-05 2024-09-17T21:31:08.592 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.592 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.592 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.592 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39465 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:08.592 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:08.593 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.537653039999995, 2024-09-17T21:31:08.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018825341999999998, 2024-09-17T21:31:08.593 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:08.593 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:08.593 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:08.593 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:08.593 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:08.593 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:08.593 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39465 2024-09-17T21:31:08.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.593 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:08.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:08.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:08.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:08.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:08.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:08.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:08.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:08.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-17T21:31:08.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:08.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968164+0000", 2024-09-17T21:31:08.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1289999999999999e-06 2024-09-17T21:31:08.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:08.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968167+0000", 2024-09-17T21:31:08.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1379999999999999e-06 2024-09-17T21:31:08.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:08.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968170+0000", 2024-09-17T21:31:08.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:31:08.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:08.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986914+0000", 2024-09-17T21:31:08.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018743829 2024-09-17T21:31:08.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:08.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986950+0000", 2024-09-17T21:31:08.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5819999999999999e-05 2024-09-17T21:31:08.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:08.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:08.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:08.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986988+0000", 2024-09-17T21:31:08.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8362000000000003e-05 2024-09-17T21:31:08.596 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.596 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.596 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.597 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:08.597 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:08.597 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:08.598 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:31:08.599 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:08.596+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:08.699 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:08.697+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:08.742 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:08.742 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:08.742 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:08.742 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:08.751 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:31:08.836 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:08.834+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:08.895 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:08.896 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:08.901 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:08.899+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:08.904 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:31:09.002 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:08.999+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:09.013 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:31:09.049 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:31:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:31:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.10727268300001, 2024-09-17T21:31:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61572000699999996, 2024-09-17T21:31:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:31:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:31:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T21:31:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:31:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:31:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939745+0000", 2024-09-17T21:31:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6899999999999999e-06 2024-09-17T21:31:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939753+0000", 2024-09-17T21:31:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1950000000000003e-06 2024-09-17T21:31:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939767+0000", 2024-09-17T21:31:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3429e-05 2024-09-17T21:31:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939776+0000", 2024-09-17T21:31:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:31:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939858+0000", 2024-09-17T21:31:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2496000000000004e-05 2024-09-17T21:31:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T21:31:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939868+0000", 2024-09-17T21:31:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1910000000000006e-06 2024-09-17T21:31:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940612+0000", 2024-09-17T21:31:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074429599999999998 2024-09-17T21:31:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:31:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940621+0000", 2024-09-17T21:31:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9139999999999997e-06 2024-09-17T21:31:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554006+0000", 2024-09-17T21:31:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61338505099999996 2024-09-17T21:31:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554166+0000", 2024-09-17T21:31:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015982600000000001 2024-09-17T21:31:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:31:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554296+0000", 2024-09-17T21:31:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013002399999999999 2024-09-17T21:31:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554936+0000", 2024-09-17T21:31:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064009400000000001 2024-09-17T21:31:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555389+0000", 2024-09-17T21:31:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045355200000000002 2024-09-17T21:31:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555432+0000", 2024-09-17T21:31:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2477e-05 2024-09-17T21:31:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555463+0000", 2024-09-17T21:31:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1612000000000002e-05 2024-09-17T21:31:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17527 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1005568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:31:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.642044851999998, 2024-09-17T21:31:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046452090000000001, 2024-09-17T21:31:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17527 2024-09-17T21:31:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:31:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:31:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405496+0000", 2024-09-17T21:31:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052498400000000004 2024-09-17T21:31:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405500+0000", 2024-09-17T21:31:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8410000000000002e-06 2024-09-17T21:31:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405511+0000", 2024-09-17T21:31:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0749e-05 2024-09-17T21:31:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405517+0000", 2024-09-17T21:31:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:31:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451275+0000", 2024-09-17T21:31:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045758134999999998 2024-09-17T21:31:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451355+0000", 2024-09-17T21:31:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9453e-05 2024-09-17T21:31:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451423+0000", 2024-09-17T21:31:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8782999999999995e-05 2024-09-17T21:31:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17615 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.636804364, 2024-09-17T21:31:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046741309000000002, 2024-09-17T21:31:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17615 2024-09-17T21:31:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:31:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410213+0000", 2024-09-17T21:31:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:31:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410217+0000", 2024-09-17T21:31:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8199999999999998e-06 2024-09-17T21:31:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410220+0000", 2024-09-17T21:31:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6900000000000001e-06 2024-09-17T21:31:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455264+0000", 2024-09-17T21:31:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045044002999999999 2024-09-17T21:31:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455338+0000", 2024-09-17T21:31:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4182000000000002e-05 2024-09-17T21:31:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.456953+0000", 2024-09-17T21:31:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016150089999999999 2024-09-17T21:31:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17616 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 872448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.636792311999997, 2024-09-17T21:31:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946039000000002, 2024-09-17T21:31:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17616 2024-09-17T21:31:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T21:31:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410225+0000", 2024-09-17T21:31:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3459999999999999e-06 2024-09-17T21:31:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410228+0000", 2024-09-17T21:31:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.083e-06 2024-09-17T21:31:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410232+0000", 2024-09-17T21:31:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4419999999999998e-06 2024-09-17T21:31:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457004+0000", 2024-09-17T21:31:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046771873999999998 2024-09-17T21:31:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457084+0000", 2024-09-17T21:31:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0506000000000005e-05 2024-09-17T21:31:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457170+0000", 2024-09-17T21:31:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5494000000000005e-05 2024-09-17T21:31:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17617 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.636780909999999, 2024-09-17T21:31:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047084168000000003, 2024-09-17T21:31:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17617 2024-09-17T21:31:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-17T21:31:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410237+0000", 2024-09-17T21:31:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1960000000000001e-06 2024-09-17T21:31:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410246+0000", 2024-09-17T21:31:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7880000000000001e-06 2024-09-17T21:31:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410250+0000", 2024-09-17T21:31:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.518e-06 2024-09-17T21:31:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457209+0000", 2024-09-17T21:31:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046959283999999997 2024-09-17T21:31:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457264+0000", 2024-09-17T21:31:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673999999999999e-05 2024-09-17T21:31:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457319+0000", 2024-09-17T21:31:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5497000000000001e-05 2024-09-17T21:31:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17618 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 899072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.636761465999996, 2024-09-17T21:31:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047204208999999997, 2024-09-17T21:31:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17618 2024-09-17T21:31:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8799999999999998e-07 2024-09-17T21:31:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410256+0000", 2024-09-17T21:31:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T21:31:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410260+0000", 2024-09-17T21:31:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6500000000000002e-06 2024-09-17T21:31:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410263+0000", 2024-09-17T21:31:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7089999999999998e-06 2024-09-17T21:31:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457355+0000", 2024-09-17T21:31:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047092756999999999 2024-09-17T21:31:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457407+0000", 2024-09-17T21:31:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1808999999999998e-05 2024-09-17T21:31:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457459+0000", 2024-09-17T21:31:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1684000000000002e-05 2024-09-17T21:31:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17619 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 913408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.636750227999997, 2024-09-17T21:31:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047293443999999997, 2024-09-17T21:31:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17619 2024-09-17T21:31:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:31:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410267+0000", 2024-09-17T21:31:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:31:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410270+0000", 2024-09-17T21:31:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.01e-06 2024-09-17T21:31:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410273+0000", 2024-09-17T21:31:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6410000000000002e-06 2024-09-17T21:31:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457473+0000", 2024-09-17T21:31:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047199829999999998 2024-09-17T21:31:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457517+0000", 2024-09-17T21:31:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4058999999999999e-05 2024-09-17T21:31:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457559+0000", 2024-09-17T21:31:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2404999999999998e-05 2024-09-17T21:31:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17620 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 927744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.636739055999996, 2024-09-17T21:31:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047402409, 2024-09-17T21:31:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17620 2024-09-17T21:31:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:09.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:31:09.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410278+0000", 2024-09-17T21:31:09.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1790000000000001e-06 2024-09-17T21:31:09.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410281+0000", 2024-09-17T21:31:09.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.061e-06 2024-09-17T21:31:09.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410284+0000", 2024-09-17T21:31:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.65e-06 2024-09-17T21:31:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457587+0000", 2024-09-17T21:31:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047303206 2024-09-17T21:31:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457633+0000", 2024-09-17T21:31:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6063999999999998e-05 2024-09-17T21:31:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457679+0000", 2024-09-17T21:31:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6007000000000002e-05 2024-09-17T21:31:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17621 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 940032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:31:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.636728640000001, 2024-09-17T21:31:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047486550000000002, 2024-09-17T21:31:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17621 2024-09-17T21:31:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:31:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:31:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410288+0000", 2024-09-17T21:31:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:31:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410289+0000", 2024-09-17T21:31:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1179999999999999e-06 2024-09-17T21:31:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410292+0000", 2024-09-17T21:31:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:31:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410294+0000", 2024-09-17T21:31:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T21:31:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457694+0000", 2024-09-17T21:31:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047399416999999999 2024-09-17T21:31:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457734+0000", 2024-09-17T21:31:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0528999999999998e-05 2024-09-17T21:31:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457774+0000", 2024-09-17T21:31:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9529e-05 2024-09-17T21:31:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17622 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.636718458000004, 2024-09-17T21:31:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047596619, 2024-09-17T21:31:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17622 2024-09-17T21:31:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:31:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410303+0000", 2024-09-17T21:31:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2979999999999999e-06 2024-09-17T21:31:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410307+0000", 2024-09-17T21:31:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:31:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410310+0000", 2024-09-17T21:31:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.146e-06 2024-09-17T21:31:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457801+0000", 2024-09-17T21:31:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047491322000000002 2024-09-17T21:31:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457850+0000", 2024-09-17T21:31:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8585000000000003e-05 2024-09-17T21:31:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457894+0000", 2024-09-17T21:31:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4455e-05 2024-09-17T21:31:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17623 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 968704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.636702073999999, 2024-09-17T21:31:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047689377999999998, 2024-09-17T21:31:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17623 2024-09-17T21:31:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:31:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410315+0000", 2024-09-17T21:31:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0499999999999999e-06 2024-09-17T21:31:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410319+0000", 2024-09-17T21:31:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.517e-06 2024-09-17T21:31:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410321+0000", 2024-09-17T21:31:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:31:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457922+0000", 2024-09-17T21:31:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047600638000000001 2024-09-17T21:31:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457964+0000", 2024-09-17T21:31:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2126000000000003e-05 2024-09-17T21:31:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458003+0000", 2024-09-17T21:31:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9138999999999999e-05 2024-09-17T21:31:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17624 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.636691490000004, 2024-09-17T21:31:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047798292999999999, 2024-09-17T21:31:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17624 2024-09-17T21:31:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:31:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410326+0000", 2024-09-17T21:31:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0610000000000001e-06 2024-09-17T21:31:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410329+0000", 2024-09-17T21:31:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0319999999999999e-06 2024-09-17T21:31:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410331+0000", 2024-09-17T21:31:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5749999999999999e-06 2024-09-17T21:31:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458033+0000", 2024-09-17T21:31:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047701371999999999 2024-09-17T21:31:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458079+0000", 2024-09-17T21:31:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:31:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458123+0000", 2024-09-17T21:31:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850999999999997e-05 2024-09-17T21:31:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17625 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.636681330000002, 2024-09-17T21:31:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047883051000000003, 2024-09-17T21:31:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17625 2024-09-17T21:31:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:31:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410336+0000", 2024-09-17T21:31:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1310000000000001e-06 2024-09-17T21:31:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410339+0000", 2024-09-17T21:31:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000001e-06 2024-09-17T21:31:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410342+0000", 2024-09-17T21:31:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7460000000000001e-06 2024-09-17T21:31:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458137+0000", 2024-09-17T21:31:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047794931999999998 2024-09-17T21:31:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458178+0000", 2024-09-17T21:31:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1013999999999997e-05 2024-09-17T21:31:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458218+0000", 2024-09-17T21:31:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9675999999999998e-05 2024-09-17T21:31:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17626 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1009664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:31:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.636670709000001, 2024-09-17T21:31:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047991601000000002, 2024-09-17T21:31:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17626 2024-09-17T21:31:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:31:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:31:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410346+0000", 2024-09-17T21:31:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:31:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410347+0000", 2024-09-17T21:31:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.114e-06 2024-09-17T21:31:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410350+0000", 2024-09-17T21:31:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.101e-06 2024-09-17T21:31:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410353+0000", 2024-09-17T21:31:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T21:31:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458245+0000", 2024-09-17T21:31:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0478921 2024-09-17T21:31:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458292+0000", 2024-09-17T21:31:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397999999999999e-05 2024-09-17T21:31:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458337+0000", 2024-09-17T21:31:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5221e-05 2024-09-17T21:31:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17627 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:31:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.636633607999997, 2024-09-17T21:31:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048047383999999999, 2024-09-17T21:31:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17627 2024-09-17T21:31:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:31:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:31:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410383+0000", 2024-09-17T21:31:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:31:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410385+0000", 2024-09-17T21:31:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:31:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410391+0000", 2024-09-17T21:31:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:31:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410395+0000", 2024-09-17T21:31:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6539999999999998e-06 2024-09-17T21:31:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458350+0000", 2024-09-17T21:31:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047954855999999997 2024-09-17T21:31:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458392+0000", 2024-09-17T21:31:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2094000000000001e-05 2024-09-17T21:31:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458430+0000", 2024-09-17T21:31:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7474000000000001e-05 2024-09-17T21:31:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17628 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.636614046999995, 2024-09-17T21:31:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048142676000000002, 2024-09-17T21:31:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17628 2024-09-17T21:31:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:31:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410404+0000", 2024-09-17T21:31:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-06 2024-09-17T21:31:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410411+0000", 2024-09-17T21:31:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6850000000000004e-06 2024-09-17T21:31:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410415+0000", 2024-09-17T21:31:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6750000000000001e-06 2024-09-17T21:31:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458457+0000", 2024-09-17T21:31:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048042045999999998 2024-09-17T21:31:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458501+0000", 2024-09-17T21:31:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3661000000000001e-05 2024-09-17T21:31:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458545+0000", 2024-09-17T21:31:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3529000000000003e-05 2024-09-17T21:31:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17687 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:31:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.634169094000001, 2024-09-17T21:31:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050670448, 2024-09-17T21:31:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17687 2024-09-17T21:31:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:31:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:31:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412848+0000", 2024-09-17T21:31:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9100000000000003e-07 2024-09-17T21:31:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413102+0000", 2024-09-17T21:31:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025415600000000001 2024-09-17T21:31:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413111+0000", 2024-09-17T21:31:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2010000000000005e-06 2024-09-17T21:31:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413116+0000", 2024-09-17T21:31:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:31:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458559+0000", 2024-09-17T21:31:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045442914000000001 2024-09-17T21:31:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458632+0000", 2024-09-17T21:31:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3207000000000006e-05 2024-09-17T21:31:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:31:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458785+0000", 2024-09-17T21:31:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015226700000000001 2024-09-17T21:31:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.461648+0000", 2024-09-17T21:31:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028633069999999998 2024-09-17T21:31:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463416+0000", 2024-09-17T21:31:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017680230000000001 2024-09-17T21:31:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463475+0000", 2024-09-17T21:31:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9333000000000003e-05 2024-09-17T21:31:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463517+0000", 2024-09-17T21:31:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2162000000000001e-05 2024-09-17T21:31:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17688 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [write 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.633890226000005, 2024-09-17T21:31:09.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050625098, 2024-09-17T21:31:09.132 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.132 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.132 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:09.132 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.132 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.132 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.132 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17688 2024-09-17T21:31:09.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.132 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:09.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:09.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:09.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:31:09.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413128+0000", 2024-09-17T21:31:09.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.813e-06 2024-09-17T21:31:09.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413132+0000", 2024-09-17T21:31:09.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:31:09.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413135+0000", 2024-09-17T21:31:09.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:31:09.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459756+0000", 2024-09-17T21:31:09.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046621293000000001 2024-09-17T21:31:09.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459816+0000", 2024-09-17T21:31:09.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9156000000000001e-05 2024-09-17T21:31:09.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:31:09.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459912+0000", 2024-09-17T21:31:09.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6302000000000001e-05 2024-09-17T21:31:09.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:09.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463231+0000", 2024-09-17T21:31:09.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033188290000000001 2024-09-17T21:31:09.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:09.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463701+0000", 2024-09-17T21:31:09.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047068199999999999 2024-09-17T21:31:09.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:09.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463728+0000", 2024-09-17T21:31:09.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6726000000000001e-05 2024-09-17T21:31:09.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463751+0000", 2024-09-17T21:31:09.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2824e-05 2024-09-17T21:31:09.137 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.137 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.137 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.137 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17691 3.d 3:b3a055f3:::rbd_data.11207744cc2c.0000000000000083:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:09.137 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:31:09.137 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.630345323, 2024-09-17T21:31:09.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049309370999999998, 2024-09-17T21:31:09.137 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.138 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.138 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:09.138 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.138 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.138 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.138 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17691 2024-09-17T21:31:09.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.138 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:31:09.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:31:09.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416672+0000", 2024-09-17T21:31:09.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.47e-07 2024-09-17T21:31:09.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416915+0000", 2024-09-17T21:31:09.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024344500000000001 2024-09-17T21:31:09.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416924+0000", 2024-09-17T21:31:09.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7739999999999993e-06 2024-09-17T21:31:09.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416929+0000", 2024-09-17T21:31:09.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6990000000000002e-06 2024-09-17T21:31:09.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460414+0000", 2024-09-17T21:31:09.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043485466 2024-09-17T21:31:09.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460540+0000", 2024-09-17T21:31:09.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012608899999999999 2024-09-17T21:31:09.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:31:09.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460746+0000", 2024-09-17T21:31:09.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000205577 2024-09-17T21:31:09.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:09.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463800+0000", 2024-09-17T21:31:09.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030542030000000001 2024-09-17T21:31:09.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:09.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465867+0000", 2024-09-17T21:31:09.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020665929999999998 2024-09-17T21:31:09.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:09.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465914+0000", 2024-09-17T21:31:09.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497000000000003e-05 2024-09-17T21:31:09.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465980+0000", 2024-09-17T21:31:09.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6081000000000006e-05 2024-09-17T21:31:09.142 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.143 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.143 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.143 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:70218 3.16 3:68692d8f:::rbd_data.1120b9193b9f.000000000000001b:head [stat,delete] snapc 1d=[1d] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-17T21:31:09.143 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:09.143 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.241378234000003, 2024-09-17T21:31:09.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.951400786, 2024-09-17T21:31:09.143 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:09.143 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:09.143 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:09.143 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:09.143 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:09.143 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:09.143 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 70218 2024-09-17T21:31:09.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.144 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:09.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:09.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:09.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:09.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:09.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:09.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:09.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:09.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4799999999999998e-07 2024-09-17T21:31:09.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:09.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805642+0000", 2024-09-17T21:31:09.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2009999999999999e-06 2024-09-17T21:31:09.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:09.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805650+0000", 2024-09-17T21:31:09.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1000000000000004e-06 2024-09-17T21:31:09.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:09.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805655+0000", 2024-09-17T21:31:09.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:31:09.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805722+0000", 2024-09-17T21:31:09.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6878000000000006e-05 2024-09-17T21:31:09.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:31:09.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805733+0000", 2024-09-17T21:31:09.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0895e-05 2024-09-17T21:31:09.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:09.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755252+0000", 2024-09-17T21:31:09.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.94951843800000002 2024-09-17T21:31:09.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:09.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755398+0000", 2024-09-17T21:31:09.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014622600000000001 2024-09-17T21:31:09.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:09.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:09.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:09.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.757039+0000", 2024-09-17T21:31:09.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001640819 2024-09-17T21:31:09.147 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.147 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.147 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.147 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:09.148 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:09.148 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:09.149 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:09.149 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:09.150 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:31:09.150 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:09.147+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:09.156 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:09.156 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:09.173 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:09.173 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:09.189 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:09.189 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:09.206 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:09.206 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:09.222 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:09.223 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:09.239 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:09.239 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:09.250 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:09.247+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:09.255 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:09.255 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:09.255 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:09.255 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:09.264 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:31:09.350 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:09.349+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:09.409 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:09.409 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:09.421 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:31:09.421 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:31:09.463 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:09.461+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:09.553 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:09.551+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:09.577 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:09.577 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:09.587 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:31:09.671 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:09.670+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:09.733 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:09.733 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:09.742 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:31:09.754 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:09.753+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:09.855 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:09.853+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:09.884 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:09.884 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:09.893 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:31:09.956 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:09.955+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:09.985 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-17T21:31:09.985 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:31:09.985 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-09-17T21:31:09.985 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2024-09-17T21:31:09.996 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:09.996 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:10.005 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:31:10.057 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:10.055+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:10.156 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:10.156 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:10.158 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:10.156+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:10.165 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:31:10.259 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:10.257+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:10.293 INFO:tasks.ceph.mon.a.smithi129.stderr:2024-09-17T21:31:10.292+0000 7f70111af640 -1 mon.a@0(leader).osd e100 definitely_dead 0 2024-09-17T21:31:10.308 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:10.308 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:10.317 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:31:10.360 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:10.357+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:10.413 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is unset 2024-09-17T21:31:10.423 INFO:teuthology.orchestra.run.smithi129.stderr:marked in osd.5. 2024-09-17T21:31:10.441 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-09-17T21:31:10.470 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:10.470 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:10.471 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:10.469+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:10.479 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:31:10.562 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:10.560+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:10.623 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:10.623 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:10.632 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:31:10.663 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:10.661+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:10.735 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:10.735 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:10.743 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:31:10.787 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:10.786+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:10.846 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:10.847 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:10.855 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:31:10.896 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:10.895+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:10.966 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:10.964+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:10.971 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:10.971 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:10.980 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:31:11.067 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:11.065+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:11.125 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:11.126 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:11.135 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:31:11.168 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:11.166+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:11.267 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:31:11.268 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:11.267+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:11.278 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:11.278 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:11.287 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:31:11.369 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:11.367+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:11.388 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:11.388 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:11.392 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:11.393 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:11.402 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:31:11.405 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:11.405 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:11.425 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:11.425 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:11.443 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:11.444 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:11.459 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:11.459 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:11.470 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:11.468+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:11.476 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:11.476 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:11.492 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:11.492 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:11.508 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:11.508 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:11.510 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:11.510 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:11.511 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:11.511 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:11.511 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:11.519 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:31:11.570 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:11.568+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:11.625 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:11.625 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:11.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:11.626 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:11.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.626 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10799 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.626 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:11.626 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.557032456000002, 2024-09-17T21:31:11.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039122745, 2024-09-17T21:31:11.626 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.626 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.626 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:11.626 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.626 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.626 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.626 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10799 2024-09-17T21:31:11.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.627 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:11.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:11.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065469+0000", 2024-09-17T21:31:11.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:31:11.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065477+0000", 2024-09-17T21:31:11.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1319999999999994e-06 2024-09-17T21:31:11.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065483+0000", 2024-09-17T21:31:11.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2040000000000002e-06 2024-09-17T21:31:11.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065487+0000", 2024-09-17T21:31:11.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.42e-06 2024-09-17T21:31:11.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099984+0000", 2024-09-17T21:31:11.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034496484000000001 2024-09-17T21:31:11.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100065+0000", 2024-09-17T21:31:11.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1200999999999997e-05 2024-09-17T21:31:11.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:11.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100422+0000", 2024-09-17T21:31:11.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035657700000000001 2024-09-17T21:31:11.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:11.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104214+0000", 2024-09-17T21:31:11.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003792045 2024-09-17T21:31:11.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:11.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104505+0000", 2024-09-17T21:31:11.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029141399999999999 2024-09-17T21:31:11.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:11.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104556+0000", 2024-09-17T21:31:11.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1388999999999999e-05 2024-09-17T21:31:11.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104591+0000", 2024-09-17T21:31:11.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4455999999999997e-05 2024-09-17T21:31:11.631 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.632 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.632 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.632 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12187 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.632 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:11.632 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.482939724999994, 2024-09-17T21:31:11.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038825750999999999, 2024-09-17T21:31:11.632 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.632 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.632 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:11.632 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.632 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.633 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.633 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12187 2024-09-17T21:31:11.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.633 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:11.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:11.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:11.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:31:11.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139563+0000", 2024-09-17T21:31:11.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.624e-06 2024-09-17T21:31:11.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139569+0000", 2024-09-17T21:31:11.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1090000000000003e-06 2024-09-17T21:31:11.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139573+0000", 2024-09-17T21:31:11.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4429999999999996e-06 2024-09-17T21:31:11.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178206+0000", 2024-09-17T21:31:11.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038632376000000003 2024-09-17T21:31:11.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178282+0000", 2024-09-17T21:31:11.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6076999999999997e-05 2024-09-17T21:31:11.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178387+0000", 2024-09-17T21:31:11.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104682 2024-09-17T21:31:11.636 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.636 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.636 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.636 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12188 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.636 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:11.637 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.482921544000007, 2024-09-17T21:31:11.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039053826999999999, 2024-09-17T21:31:11.637 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.637 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.637 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:11.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.637 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12188 2024-09-17T21:31:11.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.637 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:11.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:11.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139746+0000", 2024-09-17T21:31:11.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000167281 2024-09-17T21:31:11.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139749+0000", 2024-09-17T21:31:11.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9660000000000001e-06 2024-09-17T21:31:11.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139757+0000", 2024-09-17T21:31:11.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8029999999999997e-06 2024-09-17T21:31:11.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139771+0000", 2024-09-17T21:31:11.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3541e-05 2024-09-17T21:31:11.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178415+0000", 2024-09-17T21:31:11.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038644803999999998 2024-09-17T21:31:11.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178517+0000", 2024-09-17T21:31:11.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101298 2024-09-17T21:31:11.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178633+0000", 2024-09-17T21:31:11.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116134 2024-09-17T21:31:11.641 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.641 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.641 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.641 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12189 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.641 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:11.641 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.482721067, 2024-09-17T21:31:11.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039135991000000002, 2024-09-17T21:31:11.641 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.641 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.641 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:11.642 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.642 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.642 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.642 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12189 2024-09-17T21:31:11.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.642 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:11.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:11.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139780+0000", 2024-09-17T21:31:11.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:31:11.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139782+0000", 2024-09-17T21:31:11.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1950000000000002e-06 2024-09-17T21:31:11.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139788+0000", 2024-09-17T21:31:11.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0639999999999999e-06 2024-09-17T21:31:11.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139792+0000", 2024-09-17T21:31:11.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1960000000000001e-06 2024-09-17T21:31:11.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178671+0000", 2024-09-17T21:31:11.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038878176 2024-09-17T21:31:11.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178778+0000", 2024-09-17T21:31:11.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107278 2024-09-17T21:31:11.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178915+0000", 2024-09-17T21:31:11.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013757900000000001 2024-09-17T21:31:11.645 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.645 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.645 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.645 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12190 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.646 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:11.646 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.482701883999994, 2024-09-17T21:31:11.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039343781000000001, 2024-09-17T21:31:11.646 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.646 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.646 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:11.646 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.646 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.646 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.646 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12190 2024-09-17T21:31:11.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.646 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:11.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:11.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:11.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:11.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139801+0000", 2024-09-17T21:31:11.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.916e-06 2024-09-17T21:31:11.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139807+0000", 2024-09-17T21:31:11.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4540000000000004e-06 2024-09-17T21:31:11.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139812+0000", 2024-09-17T21:31:11.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9729999999999997e-06 2024-09-17T21:31:11.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178960+0000", 2024-09-17T21:31:11.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039147567000000001 2024-09-17T21:31:11.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179051+0000", 2024-09-17T21:31:11.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0754000000000006e-05 2024-09-17T21:31:11.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179142+0000", 2024-09-17T21:31:11.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1712999999999994e-05 2024-09-17T21:31:11.650 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.650 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.650 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.650 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12191 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.650 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:11.650 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.482682026000006, 2024-09-17T21:31:11.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039464336000000003, 2024-09-17T21:31:11.650 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.650 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.650 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:11.651 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.651 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.651 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.651 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12191 2024-09-17T21:31:11.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.651 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:11.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:11.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:11.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:31:11.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139821+0000", 2024-09-17T21:31:11.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1260000000000001e-06 2024-09-17T21:31:11.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139827+0000", 2024-09-17T21:31:11.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T21:31:11.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139840+0000", 2024-09-17T21:31:11.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2607e-05 2024-09-17T21:31:11.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179162+0000", 2024-09-17T21:31:11.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039322863999999999 2024-09-17T21:31:11.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179204+0000", 2024-09-17T21:31:11.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1938999999999999e-05 2024-09-17T21:31:11.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179283+0000", 2024-09-17T21:31:11.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8547000000000005e-05 2024-09-17T21:31:11.654 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.654 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.654 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.655 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12192 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.655 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:11.655 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.482654314000001, 2024-09-17T21:31:11.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039652318999999998, 2024-09-17T21:31:11.655 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.655 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.655 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:11.655 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.655 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.655 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.655 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12192 2024-09-17T21:31:11.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.655 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:11.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:11.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141458+0000", 2024-09-17T21:31:11.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001612098 2024-09-17T21:31:11.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141469+0000", 2024-09-17T21:31:11.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-17T21:31:11.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141478+0000", 2024-09-17T21:31:11.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T21:31:11.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141485+0000", 2024-09-17T21:31:11.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4760000000000003e-06 2024-09-17T21:31:11.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179317+0000", 2024-09-17T21:31:11.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037831832000000003 2024-09-17T21:31:11.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179395+0000", 2024-09-17T21:31:11.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8349000000000005e-05 2024-09-17T21:31:11.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179499+0000", 2024-09-17T21:31:11.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103628 2024-09-17T21:31:11.659 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.659 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.659 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.659 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12257 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.659 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:11.659 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.480465506000002, 2024-09-17T21:31:11.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038862751000000001, 2024-09-17T21:31:11.659 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.660 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.660 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:11.660 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.660 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.660 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.660 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12257 2024-09-17T21:31:11.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.660 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:11.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:11.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:11.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T21:31:11.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142037+0000", 2024-09-17T21:31:11.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.03e-06 2024-09-17T21:31:11.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142043+0000", 2024-09-17T21:31:11.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6319999999999996e-06 2024-09-17T21:31:11.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142052+0000", 2024-09-17T21:31:11.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2709999999999998e-06 2024-09-17T21:31:11.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180794+0000", 2024-09-17T21:31:11.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038741758000000001 2024-09-17T21:31:11.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180843+0000", 2024-09-17T21:31:11.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9021999999999999e-05 2024-09-17T21:31:11.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180898+0000", 2024-09-17T21:31:11.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673000000000003e-05 2024-09-17T21:31:11.663 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.663 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.663 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.664 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12258 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.664 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:11.664 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.480441988999999, 2024-09-17T21:31:11.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038950717000000003, 2024-09-17T21:31:11.664 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.664 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.664 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:11.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.664 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12258 2024-09-17T21:31:11.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.665 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:11.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:11.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:11.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199999999999999e-07 2024-09-17T21:31:11.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142061+0000", 2024-09-17T21:31:11.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8279999999999999e-06 2024-09-17T21:31:11.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142067+0000", 2024-09-17T21:31:11.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5600000000000001e-06 2024-09-17T21:31:11.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142071+0000", 2024-09-17T21:31:11.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6240000000000001e-06 2024-09-17T21:31:11.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180921+0000", 2024-09-17T21:31:11.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038849366000000003 2024-09-17T21:31:11.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180961+0000", 2024-09-17T21:31:11.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0219000000000002e-05 2024-09-17T21:31:11.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181009+0000", 2024-09-17T21:31:11.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8437999999999999e-05 2024-09-17T21:31:11.668 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.668 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.668 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.668 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12259 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.668 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:11.668 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.480414741999994, 2024-09-17T21:31:11.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039116832999999997, 2024-09-17T21:31:11.669 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.669 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.669 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:11.669 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.669 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.669 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.669 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12259 2024-09-17T21:31:11.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.669 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:11.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:11.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:11.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2200000000000004e-07 2024-09-17T21:31:11.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142088+0000", 2024-09-17T21:31:11.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0329999999999998e-06 2024-09-17T21:31:11.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142095+0000", 2024-09-17T21:31:11.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4320000000000004e-06 2024-09-17T21:31:11.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142100+0000", 2024-09-17T21:31:11.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:31:11.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181037+0000", 2024-09-17T21:31:11.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038937601000000002 2024-09-17T21:31:11.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181114+0000", 2024-09-17T21:31:11.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6857e-05 2024-09-17T21:31:11.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181203+0000", 2024-09-17T21:31:11.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8437999999999996e-05 2024-09-17T21:31:11.673 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.673 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.673 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.673 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12260 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.673 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:11.673 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.480394454999995, 2024-09-17T21:31:11.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039204184000000003, 2024-09-17T21:31:11.673 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.673 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.673 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:11.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.674 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.674 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12260 2024-09-17T21:31:11.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.674 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:11.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:11.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142107+0000", 2024-09-17T21:31:11.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:31:11.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142108+0000", 2024-09-17T21:31:11.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.753e-06 2024-09-17T21:31:11.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142114+0000", 2024-09-17T21:31:11.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7660000000000004e-06 2024-09-17T21:31:11.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142119+0000", 2024-09-17T21:31:11.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:31:11.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181216+0000", 2024-09-17T21:31:11.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039097191000000003 2024-09-17T21:31:11.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181260+0000", 2024-09-17T21:31:11.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4731000000000003e-05 2024-09-17T21:31:11.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181310+0000", 2024-09-17T21:31:11.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9818000000000003e-05 2024-09-17T21:31:11.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.677 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.677 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12261 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.678 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:11.678 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.480376131, 2024-09-17T21:31:11.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039292200999999999, 2024-09-17T21:31:11.678 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.678 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.678 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:11.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.678 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12261 2024-09-17T21:31:11.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.678 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:11.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:11.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142125+0000", 2024-09-17T21:31:11.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:31:11.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142127+0000", 2024-09-17T21:31:11.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.792e-06 2024-09-17T21:31:11.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142132+0000", 2024-09-17T21:31:11.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3979999999999998e-06 2024-09-17T21:31:11.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142136+0000", 2024-09-17T21:31:11.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.211e-06 2024-09-17T21:31:11.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181324+0000", 2024-09-17T21:31:11.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039187560000000003 2024-09-17T21:31:11.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181363+0000", 2024-09-17T21:31:11.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9375000000000002e-05 2024-09-17T21:31:11.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181417+0000", 2024-09-17T21:31:11.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3372000000000003e-05 2024-09-17T21:31:11.682 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.682 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.682 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.682 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12262 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.682 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:11.682 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.480358338000002, 2024-09-17T21:31:11.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039384875, 2024-09-17T21:31:11.682 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.682 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.682 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:11.682 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.683 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.683 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.683 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12262 2024-09-17T21:31:11.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.683 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:11.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:11.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142143+0000", 2024-09-17T21:31:11.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:31:11.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142148+0000", 2024-09-17T21:31:11.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4360000000000001e-06 2024-09-17T21:31:11.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142154+0000", 2024-09-17T21:31:11.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8719999999999999e-06 2024-09-17T21:31:11.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142158+0000", 2024-09-17T21:31:11.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.476e-06 2024-09-17T21:31:11.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181436+0000", 2024-09-17T21:31:11.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039277919000000001 2024-09-17T21:31:11.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181480+0000", 2024-09-17T21:31:11.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3649999999999997e-05 2024-09-17T21:31:11.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181527+0000", 2024-09-17T21:31:11.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7131e-05 2024-09-17T21:31:11.686 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.686 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.686 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.687 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12263 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1079296~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.687 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:11.687 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.480335902999997, 2024-09-17T21:31:11.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039473339000000003, 2024-09-17T21:31:11.687 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.687 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.687 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:11.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.687 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12263 2024-09-17T21:31:11.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.687 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:11.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:11.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:11.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:31:11.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142167+0000", 2024-09-17T21:31:11.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.81e-06 2024-09-17T21:31:11.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142172+0000", 2024-09-17T21:31:11.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:31:11.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142177+0000", 2024-09-17T21:31:11.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T21:31:11.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181553+0000", 2024-09-17T21:31:11.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039376130000000002 2024-09-17T21:31:11.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181592+0000", 2024-09-17T21:31:11.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9153999999999998e-05 2024-09-17T21:31:11.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181638+0000", 2024-09-17T21:31:11.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5864e-05 2024-09-17T21:31:11.691 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.691 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.691 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.691 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12278 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1095680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.691 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:11.691 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.480317928000005, 2024-09-17T21:31:11.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039454222999999997, 2024-09-17T21:31:11.691 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.692 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.692 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:11.692 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.692 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.692 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.692 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12278 2024-09-17T21:31:11.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.692 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:11.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:11.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:11.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:31:11.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142185+0000", 2024-09-17T21:31:11.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.697e-06 2024-09-17T21:31:11.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142191+0000", 2024-09-17T21:31:11.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:31:11.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142196+0000", 2024-09-17T21:31:11.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4829999999999999e-06 2024-09-17T21:31:11.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181548+0000", 2024-09-17T21:31:11.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039351200000000003 2024-09-17T21:31:11.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181591+0000", 2024-09-17T21:31:11.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3492000000000003e-05 2024-09-17T21:31:11.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181637+0000", 2024-09-17T21:31:11.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5847000000000003e-05 2024-09-17T21:31:11.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.695 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.696 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12279 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.696 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:11.696 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.480294638999993, 2024-09-17T21:31:11.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039539801999999999, 2024-09-17T21:31:11.696 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.696 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.696 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:11.696 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.696 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.696 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.696 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12279 2024-09-17T21:31:11.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.697 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:11.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:11.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:11.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.89e-07 2024-09-17T21:31:11.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142208+0000", 2024-09-17T21:31:11.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7579999999999999e-06 2024-09-17T21:31:11.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142214+0000", 2024-09-17T21:31:11.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:31:11.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142218+0000", 2024-09-17T21:31:11.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3429999999999998e-06 2024-09-17T21:31:11.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181653+0000", 2024-09-17T21:31:11.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039435322000000002 2024-09-17T21:31:11.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181698+0000", 2024-09-17T21:31:11.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4978999999999997e-05 2024-09-17T21:31:11.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181746+0000", 2024-09-17T21:31:11.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571999999999998e-05 2024-09-17T21:31:11.700 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.700 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.700 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.700 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12280 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1124352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.700 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:11.700 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.480277119999997, 2024-09-17T21:31:11.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039704358000000002, 2024-09-17T21:31:11.700 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.701 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.701 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:11.701 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.701 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.701 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.701 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12280 2024-09-17T21:31:11.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.701 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:11.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:11.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142224+0000", 2024-09-17T21:31:11.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:31:11.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142226+0000", 2024-09-17T21:31:11.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:31:11.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142231+0000", 2024-09-17T21:31:11.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:31:11.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142240+0000", 2024-09-17T21:31:11.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4590000000000004e-06 2024-09-17T21:31:11.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181774+0000", 2024-09-17T21:31:11.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039533833999999997 2024-09-17T21:31:11.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181859+0000", 2024-09-17T21:31:11.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4328999999999998e-05 2024-09-17T21:31:11.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181928+0000", 2024-09-17T21:31:11.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9139000000000003e-05 2024-09-17T21:31:11.704 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.704 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.704 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.705 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12281 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1136640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.705 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:11.705 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.480254213999999, 2024-09-17T21:31:11.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039787799999999998, 2024-09-17T21:31:11.705 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.705 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.705 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:11.705 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.705 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.705 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.705 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12281 2024-09-17T21:31:11.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.706 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:11.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:11.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142309+0000", 2024-09-17T21:31:11.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2323999999999995e-05 2024-09-17T21:31:11.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142317+0000", 2024-09-17T21:31:11.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1859999999999992e-06 2024-09-17T21:31:11.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142324+0000", 2024-09-17T21:31:11.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4699999999999996e-06 2024-09-17T21:31:11.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142330+0000", 2024-09-17T21:31:11.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4650000000000002e-06 2024-09-17T21:31:11.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181948+0000", 2024-09-17T21:31:11.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039618056999999998 2024-09-17T21:31:11.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181986+0000", 2024-09-17T21:31:11.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8442999999999998e-05 2024-09-17T21:31:11.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182034+0000", 2024-09-17T21:31:11.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7855000000000002e-05 2024-09-17T21:31:11.709 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.709 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.709 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.709 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12282 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1150976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.709 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:11.709 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.480161984999995, 2024-09-17T21:31:11.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039871919999999998, 2024-09-17T21:31:11.709 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.710 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.710 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:11.710 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.710 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.710 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.710 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12282 2024-09-17T21:31:11.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.710 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:11.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:11.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:11.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:31:11.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142341+0000", 2024-09-17T21:31:11.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.232e-06 2024-09-17T21:31:11.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142348+0000", 2024-09-17T21:31:11.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7320000000000001e-06 2024-09-17T21:31:11.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142353+0000", 2024-09-17T21:31:11.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:31:11.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182062+0000", 2024-09-17T21:31:11.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039709739000000001 2024-09-17T21:31:11.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:11.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182128+0000", 2024-09-17T21:31:11.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5739000000000002e-05 2024-09-17T21:31:11.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182210+0000", 2024-09-17T21:31:11.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:31:11.713 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.713 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.714 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12283 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:11.714 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:11.714 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.480141574000001, 2024-09-17T21:31:11.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039896434000000001, 2024-09-17T21:31:11.714 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:11.714 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:11.714 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:31:11.714 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:11.714 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:11.714 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:11.714 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12283 2024-09-17T21:31:11.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.715 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:11.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:11.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:11.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:11.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:11.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:11.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:11.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144852+0000", 2024-09-17T21:31:11.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024934079999999999 2024-09-17T21:31:11.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:11.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144856+0000", 2024-09-17T21:31:11.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:31:11.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:11.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144864+0000", 2024-09-17T21:31:11.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0160000000000008e-06 2024-09-17T21:31:11.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:11.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144869+0000", 2024-09-17T21:31:11.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:31:11.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:11.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182246+0000", 2024-09-17T21:31:11.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037377207000000003 2024-09-17T21:31:11.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:11.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:11.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:11.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182255+0000", 2024-09-17T21:31:11.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1840000000000005e-06 2024-09-17T21:31:11.717 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.718 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.718 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.718 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:11.718 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:11.718 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:11.719 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:31:11.720 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:11.718+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:11.820 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:11.819+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:11.865 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:11.866 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:11.866 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:11.866 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:11.874 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:31:11.921 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:11.919+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:11.977 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:11.978 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:11.987 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:31:12.032 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:12.030+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:12.092 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:12.092 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:12.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:12.092 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:12.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.093 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:36935 3.a 3:54c8759b:::rbd_data.1120e62eea89.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-17T21:31:12.093 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:31:12.093 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.130767136000003, 2024-09-17T21:31:12.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020540151999999999, 2024-09-17T21:31:12.093 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.093 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.093 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:12.093 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.093 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.093 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.093 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 36935 2024-09-17T21:31:12.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.093 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:31:12.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:31:12.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958910+0000", 2024-09-17T21:31:12.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:31:12.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958918+0000", 2024-09-17T21:31:12.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6469999999999998e-06 2024-09-17T21:31:12.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958933+0000", 2024-09-17T21:31:12.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4791e-05 2024-09-17T21:31:12.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958940+0000", 2024-09-17T21:31:12.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-17T21:31:12.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958990+0000", 2024-09-17T21:31:12.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9759000000000003e-05 2024-09-17T21:31:12.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.959039+0000", 2024-09-17T21:31:12.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9487000000000002e-05 2024-09-17T21:31:12.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:12.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.962098+0000", 2024-09-17T21:31:12.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003059258 2024-09-17T21:31:12.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:12.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979325+0000", 2024-09-17T21:31:12.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017226841 2024-09-17T21:31:12.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:12.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979343+0000", 2024-09-17T21:31:12.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7486000000000002e-05 2024-09-17T21:31:12.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:12.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979401+0000", 2024-09-17T21:31:12.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7980999999999999e-05 2024-09-17T21:31:12.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979449+0000", 2024-09-17T21:31:12.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8639e-05 2024-09-17T21:31:12.098 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.098 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.098 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.098 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39447 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 909312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.098 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:12.098 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121829235999996, 2024-09-17T21:31:12.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017419153999999999, 2024-09-17T21:31:12.098 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.098 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.098 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.099 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.099 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.099 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.099 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39447 2024-09-17T21:31:12.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.099 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:12.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:12.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:12.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:31:12.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967849+0000", 2024-09-17T21:31:12.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4339999999999999e-06 2024-09-17T21:31:12.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967853+0000", 2024-09-17T21:31:12.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:31:12.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967857+0000", 2024-09-17T21:31:12.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:31:12.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985193+0000", 2024-09-17T21:31:12.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017335973000000001 2024-09-17T21:31:12.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985228+0000", 2024-09-17T21:31:12.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5154999999999998e-05 2024-09-17T21:31:12.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985266+0000", 2024-09-17T21:31:12.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8448000000000002e-05 2024-09-17T21:31:12.102 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.102 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.102 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.102 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39448 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 974848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.102 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:12.102 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121816420000002, 2024-09-17T21:31:12.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017517066000000001, 2024-09-17T21:31:12.103 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.103 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.103 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.103 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.103 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.103 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.103 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39448 2024-09-17T21:31:12.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.103 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:12.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:12.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:12.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.72e-07 2024-09-17T21:31:12.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967861+0000", 2024-09-17T21:31:12.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.096e-06 2024-09-17T21:31:12.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967865+0000", 2024-09-17T21:31:12.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9339999999999999e-06 2024-09-17T21:31:12.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967868+0000", 2024-09-17T21:31:12.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.508e-06 2024-09-17T21:31:12.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985285+0000", 2024-09-17T21:31:12.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017417208 2024-09-17T21:31:12.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985339+0000", 2024-09-17T21:31:12.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3931000000000003e-05 2024-09-17T21:31:12.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985377+0000", 2024-09-17T21:31:12.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8016999999999999e-05 2024-09-17T21:31:12.106 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.106 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.106 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.106 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39449 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.106 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:12.106 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121804310000002, 2024-09-17T21:31:12.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017595137, 2024-09-17T21:31:12.106 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.107 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.107 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.107 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.107 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.107 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.107 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39449 2024-09-17T21:31:12.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.107 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:12.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:12.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:12.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2700000000000001e-07 2024-09-17T21:31:12.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967873+0000", 2024-09-17T21:31:12.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.164e-06 2024-09-17T21:31:12.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967877+0000", 2024-09-17T21:31:12.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.23e-06 2024-09-17T21:31:12.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967880+0000", 2024-09-17T21:31:12.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.286e-06 2024-09-17T21:31:12.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985397+0000", 2024-09-17T21:31:12.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017516982 2024-09-17T21:31:12.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985431+0000", 2024-09-17T21:31:12.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3899999999999997e-05 2024-09-17T21:31:12.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985467+0000", 2024-09-17T21:31:12.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6347999999999998e-05 2024-09-17T21:31:12.110 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.110 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.110 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.110 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39450 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1105920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.110 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:31:12.110 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121784052999999, 2024-09-17T21:31:12.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017677824000000002, 2024-09-17T21:31:12.111 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.111 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.111 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.111 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.111 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.111 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.111 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39450 2024-09-17T21:31:12.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.111 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:31:12.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:31:12.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967893+0000", 2024-09-17T21:31:12.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9100000000000004e-07 2024-09-17T21:31:12.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967895+0000", 2024-09-17T21:31:12.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:31:12.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967901+0000", 2024-09-17T21:31:12.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.083e-06 2024-09-17T21:31:12.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967906+0000", 2024-09-17T21:31:12.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:31:12.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985493+0000", 2024-09-17T21:31:12.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017587565999999999 2024-09-17T21:31:12.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985531+0000", 2024-09-17T21:31:12.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7490000000000002e-05 2024-09-17T21:31:12.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985570+0000", 2024-09-17T21:31:12.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9205000000000002e-05 2024-09-17T21:31:12.114 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.114 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.114 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.114 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39451 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1171456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.115 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:12.115 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121763673000004, 2024-09-17T21:31:12.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017752337999999999, 2024-09-17T21:31:12.115 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.115 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.115 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.115 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.115 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.115 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.115 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39451 2024-09-17T21:31:12.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.115 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:12.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:12.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:12.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:31:12.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967915+0000", 2024-09-17T21:31:12.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780000000000001e-06 2024-09-17T21:31:12.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967922+0000", 2024-09-17T21:31:12.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7020000000000004e-06 2024-09-17T21:31:12.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967926+0000", 2024-09-17T21:31:12.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.809e-06 2024-09-17T21:31:12.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985590+0000", 2024-09-17T21:31:12.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017663913999999999 2024-09-17T21:31:12.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985627+0000", 2024-09-17T21:31:12.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6896e-05 2024-09-17T21:31:12.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985665+0000", 2024-09-17T21:31:12.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7793999999999998e-05 2024-09-17T21:31:12.118 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.118 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.118 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.118 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39452 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1236992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.119 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:12.119 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121743730999995, 2024-09-17T21:31:12.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017829602999999999, 2024-09-17T21:31:12.119 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.119 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.119 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.119 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.119 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.119 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.119 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39452 2024-09-17T21:31:12.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.119 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:12.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:12.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:12.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:31:12.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967947+0000", 2024-09-17T21:31:12.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4280999999999999e-05 2024-09-17T21:31:12.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967954+0000", 2024-09-17T21:31:12.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3249999999999996e-06 2024-09-17T21:31:12.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967959+0000", 2024-09-17T21:31:12.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2299999999999999e-06 2024-09-17T21:31:12.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985687+0000", 2024-09-17T21:31:12.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017727844999999999 2024-09-17T21:31:12.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985725+0000", 2024-09-17T21:31:12.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8243e-05 2024-09-17T21:31:12.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985762+0000", 2024-09-17T21:31:12.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7023000000000001e-05 2024-09-17T21:31:12.122 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.122 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.122 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.123 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39453 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.123 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:31:12.123 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121707243000003, 2024-09-17T21:31:12.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017893793000000002, 2024-09-17T21:31:12.123 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.123 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.123 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.123 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.123 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.123 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.123 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39453 2024-09-17T21:31:12.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.123 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:31:12.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:31:12.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967970+0000", 2024-09-17T21:31:12.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:31:12.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967972+0000", 2024-09-17T21:31:12.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0229999999999999e-06 2024-09-17T21:31:12.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967978+0000", 2024-09-17T21:31:12.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0430000000000004e-06 2024-09-17T21:31:12.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967982+0000", 2024-09-17T21:31:12.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2570000000000001e-06 2024-09-17T21:31:12.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985782+0000", 2024-09-17T21:31:12.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017799683 2024-09-17T21:31:12.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985818+0000", 2024-09-17T21:31:12.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6304999999999999e-05 2024-09-17T21:31:12.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985863+0000", 2024-09-17T21:31:12.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4953999999999997e-05 2024-09-17T21:31:12.127 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.127 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.127 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.127 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39454 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1368064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.127 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:31:12.127 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121687914999995, 2024-09-17T21:31:12.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017968886, 2024-09-17T21:31:12.127 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.127 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.127 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.128 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.128 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.128 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39454 2024-09-17T21:31:12.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.128 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:31:12.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:31:12.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967989+0000", 2024-09-17T21:31:12.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.15e-07 2024-09-17T21:31:12.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967991+0000", 2024-09-17T21:31:12.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.945e-06 2024-09-17T21:31:12.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967997+0000", 2024-09-17T21:31:12.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0190000000000003e-06 2024-09-17T21:31:12.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968001+0000", 2024-09-17T21:31:12.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6650000000000002e-06 2024-09-17T21:31:12.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985883+0000", 2024-09-17T21:31:12.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017881971999999999 2024-09-17T21:31:12.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985920+0000", 2024-09-17T21:31:12.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-05 2024-09-17T21:31:12.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985957+0000", 2024-09-17T21:31:12.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7370000000000003e-05 2024-09-17T21:31:12.131 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.131 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.131 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.131 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39455 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.131 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:12.131 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121668740999993, 2024-09-17T21:31:12.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018044437999999999, 2024-09-17T21:31:12.131 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.131 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.131 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.132 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.132 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.132 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.132 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39455 2024-09-17T21:31:12.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.132 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:12.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:12.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:12.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:31:12.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968010+0000", 2024-09-17T21:31:12.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.886e-06 2024-09-17T21:31:12.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968017+0000", 2024-09-17T21:31:12.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:31:12.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968021+0000", 2024-09-17T21:31:12.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850000000000004e-06 2024-09-17T21:31:12.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985977+0000", 2024-09-17T21:31:12.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017956368 2024-09-17T21:31:12.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986015+0000", 2024-09-17T21:31:12.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7610000000000001e-05 2024-09-17T21:31:12.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986052+0000", 2024-09-17T21:31:12.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7109e-05 2024-09-17T21:31:12.135 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.135 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.135 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.135 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39456 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1499136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.136 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:12.136 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121648441000005, 2024-09-17T21:31:12.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018114514000000002, 2024-09-17T21:31:12.136 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.136 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.136 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.136 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39456 2024-09-17T21:31:12.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.136 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:12.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:12.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:12.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-17T21:31:12.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968030+0000", 2024-09-17T21:31:12.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0049999999999999e-06 2024-09-17T21:31:12.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968036+0000", 2024-09-17T21:31:12.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0129999999999999e-06 2024-09-17T21:31:12.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968041+0000", 2024-09-17T21:31:12.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8300000000000003e-06 2024-09-17T21:31:12.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986071+0000", 2024-09-17T21:31:12.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018029620999999999 2024-09-17T21:31:12.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986106+0000", 2024-09-17T21:31:12.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4867999999999999e-05 2024-09-17T21:31:12.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986142+0000", 2024-09-17T21:31:12.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6566000000000002e-05 2024-09-17T21:31:12.139 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.139 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.139 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.139 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39457 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1564672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.140 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:12.140 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121628625, 2024-09-17T21:31:12.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018188644, 2024-09-17T21:31:12.140 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.140 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.140 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.140 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39457 2024-09-17T21:31:12.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.140 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:12.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:12.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:12.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7199999999999999e-07 2024-09-17T21:31:12.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968050+0000", 2024-09-17T21:31:12.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1110000000000002e-06 2024-09-17T21:31:12.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968057+0000", 2024-09-17T21:31:12.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6170000000000003e-06 2024-09-17T21:31:12.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968062+0000", 2024-09-17T21:31:12.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:31:12.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986163+0000", 2024-09-17T21:31:12.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018101435999999999 2024-09-17T21:31:12.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986199+0000", 2024-09-17T21:31:12.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.591e-05 2024-09-17T21:31:12.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986236+0000", 2024-09-17T21:31:12.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7416999999999998e-05 2024-09-17T21:31:12.143 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.143 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.143 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.144 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39458 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1630208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.144 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:12.144 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121601685000002, 2024-09-17T21:31:12.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018255856000000001, 2024-09-17T21:31:12.144 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.144 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.144 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.144 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.144 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.144 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.144 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39458 2024-09-17T21:31:12.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.144 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:12.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:12.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:12.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:31:12.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968077+0000", 2024-09-17T21:31:12.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.779e-06 2024-09-17T21:31:12.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968081+0000", 2024-09-17T21:31:12.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.532e-06 2024-09-17T21:31:12.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968085+0000", 2024-09-17T21:31:12.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999999e-06 2024-09-17T21:31:12.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986256+0000", 2024-09-17T21:31:12.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018171273000000002 2024-09-17T21:31:12.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986293+0000", 2024-09-17T21:31:12.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6534e-05 2024-09-17T21:31:12.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986330+0000", 2024-09-17T21:31:12.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7536000000000001e-05 2024-09-17T21:31:12.147 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.147 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.147 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.148 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39459 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1695744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.148 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:31:12.148 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121587106999996, 2024-09-17T21:31:12.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334493, 2024-09-17T21:31:12.148 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.148 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.148 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.148 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.148 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.148 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.148 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39459 2024-09-17T21:31:12.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.148 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:31:12.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:31:12.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968090+0000", 2024-09-17T21:31:12.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:31:12.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968091+0000", 2024-09-17T21:31:12.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3319999999999999e-06 2024-09-17T21:31:12.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968094+0000", 2024-09-17T21:31:12.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2590000000000001e-06 2024-09-17T21:31:12.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968097+0000", 2024-09-17T21:31:12.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:31:12.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986350+0000", 2024-09-17T21:31:12.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018253364000000001 2024-09-17T21:31:12.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986387+0000", 2024-09-17T21:31:12.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6350999999999998e-05 2024-09-17T21:31:12.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986424+0000", 2024-09-17T21:31:12.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6897999999999998e-05 2024-09-17T21:31:12.154 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.154 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.154 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.154 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39460 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1761280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.154 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:31:12.155 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121575970999999, 2024-09-17T21:31:12.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018415918999999999, 2024-09-17T21:31:12.155 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.155 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.155 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.155 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.155 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.155 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.155 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39460 2024-09-17T21:31:12.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.155 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:31:12.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:31:12.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968101+0000", 2024-09-17T21:31:12.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T21:31:12.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968102+0000", 2024-09-17T21:31:12.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1689999999999999e-06 2024-09-17T21:31:12.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968105+0000", 2024-09-17T21:31:12.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:31:12.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968108+0000", 2024-09-17T21:31:12.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6630000000000002e-06 2024-09-17T21:31:12.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986442+0000", 2024-09-17T21:31:12.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334487 2024-09-17T21:31:12.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986479+0000", 2024-09-17T21:31:12.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6557000000000003e-05 2024-09-17T21:31:12.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986516+0000", 2024-09-17T21:31:12.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7648999999999998e-05 2024-09-17T21:31:12.159 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.159 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.159 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.159 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39461 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1826816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.159 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:12.159 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121565211000004, 2024-09-17T21:31:12.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018499768, 2024-09-17T21:31:12.159 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.159 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.159 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.160 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.160 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.160 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.160 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39461 2024-09-17T21:31:12.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.160 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:12.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:12.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:12.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4699999999999998e-07 2024-09-17T21:31:12.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968112+0000", 2024-09-17T21:31:12.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-06 2024-09-17T21:31:12.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968116+0000", 2024-09-17T21:31:12.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1080000000000001e-06 2024-09-17T21:31:12.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968122+0000", 2024-09-17T21:31:12.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1070000000000001e-06 2024-09-17T21:31:12.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986537+0000", 2024-09-17T21:31:12.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018414834000000001 2024-09-17T21:31:12.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986572+0000", 2024-09-17T21:31:12.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5726000000000003e-05 2024-09-17T21:31:12.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986611+0000", 2024-09-17T21:31:12.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8584000000000002e-05 2024-09-17T21:31:12.163 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.163 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.163 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.164 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39462 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1892352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.164 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:31:12.164 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121550846999995, 2024-09-17T21:31:12.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018583252000000001, 2024-09-17T21:31:12.164 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.164 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.164 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.164 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.164 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.164 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.164 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39462 2024-09-17T21:31:12.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.165 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:31:12.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:31:12.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968126+0000", 2024-09-17T21:31:12.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:12.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968127+0000", 2024-09-17T21:31:12.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.229e-06 2024-09-17T21:31:12.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968131+0000", 2024-09-17T21:31:12.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8010000000000001e-06 2024-09-17T21:31:12.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968134+0000", 2024-09-17T21:31:12.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8880000000000001e-06 2024-09-17T21:31:12.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986634+0000", 2024-09-17T21:31:12.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018500501999999999 2024-09-17T21:31:12.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986671+0000", 2024-09-17T21:31:12.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6881000000000001e-05 2024-09-17T21:31:12.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986709+0000", 2024-09-17T21:31:12.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7546999999999998e-05 2024-09-17T21:31:12.168 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.168 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.168 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.168 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39463 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1957888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.168 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:12.168 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121539232000003, 2024-09-17T21:31:12.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018667186999999998, 2024-09-17T21:31:12.168 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.168 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.168 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.169 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.169 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.169 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.169 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39463 2024-09-17T21:31:12.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.169 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:12.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:12.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:12.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:31:12.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968138+0000", 2024-09-17T21:31:12.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:31:12.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968142+0000", 2024-09-17T21:31:12.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.145e-06 2024-09-17T21:31:12.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968144+0000", 2024-09-17T21:31:12.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6869999999999999e-06 2024-09-17T21:31:12.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986728+0000", 2024-09-17T21:31:12.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018584163000000001 2024-09-17T21:31:12.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986766+0000", 2024-09-17T21:31:12.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7987000000000001e-05 2024-09-17T21:31:12.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986804+0000", 2024-09-17T21:31:12.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7892000000000002e-05 2024-09-17T21:31:12.172 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.172 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.172 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.172 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39464 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.172 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:31:12.172 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121528923, 2024-09-17T21:31:12.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018753723999999999, 2024-09-17T21:31:12.173 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.173 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.173 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.173 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.173 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.173 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.173 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39464 2024-09-17T21:31:12.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.173 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:31:12.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:31:12.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968148+0000", 2024-09-17T21:31:12.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:31:12.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968152+0000", 2024-09-17T21:31:12.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3749999999999996e-06 2024-09-17T21:31:12.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968155+0000", 2024-09-17T21:31:12.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.371e-06 2024-09-17T21:31:12.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968158+0000", 2024-09-17T21:31:12.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T21:31:12.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986824+0000", 2024-09-17T21:31:12.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018665779 2024-09-17T21:31:12.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986863+0000", 2024-09-17T21:31:12.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9273000000000002e-05 2024-09-17T21:31:12.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986901+0000", 2024-09-17T21:31:12.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7978000000000001e-05 2024-09-17T21:31:12.176 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.176 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.176 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.177 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39465 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:12.177 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:12.177 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.121513523999994, 2024-09-17T21:31:12.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018825341999999998, 2024-09-17T21:31:12.177 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.177 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.177 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.177 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.177 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.177 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.177 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39465 2024-09-17T21:31:12.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.178 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:12.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:12.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:12.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-17T21:31:12.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968164+0000", 2024-09-17T21:31:12.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1289999999999999e-06 2024-09-17T21:31:12.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968167+0000", 2024-09-17T21:31:12.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1379999999999999e-06 2024-09-17T21:31:12.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968170+0000", 2024-09-17T21:31:12.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:31:12.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986914+0000", 2024-09-17T21:31:12.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018743829 2024-09-17T21:31:12.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986950+0000", 2024-09-17T21:31:12.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5819999999999999e-05 2024-09-17T21:31:12.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986988+0000", 2024-09-17T21:31:12.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8362000000000003e-05 2024-09-17T21:31:12.181 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.181 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.181 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.181 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.181 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.181 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:12.183 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:31:12.183 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:12.181+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:12.283 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:12.282+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:12.327 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:12.327 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:12.327 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:12.328 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:12.336 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:31:12.384 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:12.383+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:12.434 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:31:12.458 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:12.458 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:12.467 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:31:12.511 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:12.509+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:12.586 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:31:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:31:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 104.643678849, 2024-09-17T21:31:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61572000699999996, 2024-09-17T21:31:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:31:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:31:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T21:31:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:31:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:31:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939745+0000", 2024-09-17T21:31:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6899999999999999e-06 2024-09-17T21:31:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939753+0000", 2024-09-17T21:31:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1950000000000003e-06 2024-09-17T21:31:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939767+0000", 2024-09-17T21:31:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3429e-05 2024-09-17T21:31:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939776+0000", 2024-09-17T21:31:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:31:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939858+0000", 2024-09-17T21:31:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2496000000000004e-05 2024-09-17T21:31:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T21:31:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939868+0000", 2024-09-17T21:31:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1910000000000006e-06 2024-09-17T21:31:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940612+0000", 2024-09-17T21:31:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074429599999999998 2024-09-17T21:31:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:31:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940621+0000", 2024-09-17T21:31:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9139999999999997e-06 2024-09-17T21:31:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554006+0000", 2024-09-17T21:31:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61338505099999996 2024-09-17T21:31:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554166+0000", 2024-09-17T21:31:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015982600000000001 2024-09-17T21:31:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:31:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554296+0000", 2024-09-17T21:31:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013002399999999999 2024-09-17T21:31:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554936+0000", 2024-09-17T21:31:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064009400000000001 2024-09-17T21:31:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555389+0000", 2024-09-17T21:31:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045355200000000002 2024-09-17T21:31:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555432+0000", 2024-09-17T21:31:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2477e-05 2024-09-17T21:31:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555463+0000", 2024-09-17T21:31:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1612000000000002e-05 2024-09-17T21:31:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17527 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1005568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:31:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.178451018000004, 2024-09-17T21:31:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046452090000000001, 2024-09-17T21:31:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17527 2024-09-17T21:31:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:31:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:31:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405496+0000", 2024-09-17T21:31:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052498400000000004 2024-09-17T21:31:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405500+0000", 2024-09-17T21:31:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8410000000000002e-06 2024-09-17T21:31:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405511+0000", 2024-09-17T21:31:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0749e-05 2024-09-17T21:31:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405517+0000", 2024-09-17T21:31:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:31:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451275+0000", 2024-09-17T21:31:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045758134999999998 2024-09-17T21:31:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451355+0000", 2024-09-17T21:31:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9453e-05 2024-09-17T21:31:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451423+0000", 2024-09-17T21:31:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8782999999999995e-05 2024-09-17T21:31:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17615 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.173210530000006, 2024-09-17T21:31:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046741309000000002, 2024-09-17T21:31:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17615 2024-09-17T21:31:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:31:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410213+0000", 2024-09-17T21:31:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:31:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410217+0000", 2024-09-17T21:31:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8199999999999998e-06 2024-09-17T21:31:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410220+0000", 2024-09-17T21:31:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6900000000000001e-06 2024-09-17T21:31:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455264+0000", 2024-09-17T21:31:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045044002999999999 2024-09-17T21:31:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455338+0000", 2024-09-17T21:31:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4182000000000002e-05 2024-09-17T21:31:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.456953+0000", 2024-09-17T21:31:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016150089999999999 2024-09-17T21:31:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17616 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 872448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.173198478000003, 2024-09-17T21:31:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946039000000002, 2024-09-17T21:31:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17616 2024-09-17T21:31:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T21:31:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410225+0000", 2024-09-17T21:31:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3459999999999999e-06 2024-09-17T21:31:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410228+0000", 2024-09-17T21:31:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.083e-06 2024-09-17T21:31:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410232+0000", 2024-09-17T21:31:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4419999999999998e-06 2024-09-17T21:31:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457004+0000", 2024-09-17T21:31:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046771873999999998 2024-09-17T21:31:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457084+0000", 2024-09-17T21:31:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0506000000000005e-05 2024-09-17T21:31:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457170+0000", 2024-09-17T21:31:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5494000000000005e-05 2024-09-17T21:31:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17617 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.173187076000005, 2024-09-17T21:31:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047084168000000003, 2024-09-17T21:31:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17617 2024-09-17T21:31:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-17T21:31:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410237+0000", 2024-09-17T21:31:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1960000000000001e-06 2024-09-17T21:31:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410246+0000", 2024-09-17T21:31:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7880000000000001e-06 2024-09-17T21:31:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410250+0000", 2024-09-17T21:31:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.518e-06 2024-09-17T21:31:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457209+0000", 2024-09-17T21:31:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046959283999999997 2024-09-17T21:31:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457264+0000", 2024-09-17T21:31:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673999999999999e-05 2024-09-17T21:31:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457319+0000", 2024-09-17T21:31:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5497000000000001e-05 2024-09-17T21:31:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17618 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 899072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.173167632000002, 2024-09-17T21:31:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047204208999999997, 2024-09-17T21:31:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17618 2024-09-17T21:31:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8799999999999998e-07 2024-09-17T21:31:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410256+0000", 2024-09-17T21:31:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T21:31:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410260+0000", 2024-09-17T21:31:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6500000000000002e-06 2024-09-17T21:31:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410263+0000", 2024-09-17T21:31:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7089999999999998e-06 2024-09-17T21:31:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457355+0000", 2024-09-17T21:31:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047092756999999999 2024-09-17T21:31:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457407+0000", 2024-09-17T21:31:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1808999999999998e-05 2024-09-17T21:31:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457459+0000", 2024-09-17T21:31:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1684000000000002e-05 2024-09-17T21:31:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17619 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 913408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.173156394000003, 2024-09-17T21:31:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047293443999999997, 2024-09-17T21:31:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17619 2024-09-17T21:31:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:31:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410267+0000", 2024-09-17T21:31:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:31:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410270+0000", 2024-09-17T21:31:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.01e-06 2024-09-17T21:31:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410273+0000", 2024-09-17T21:31:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6410000000000002e-06 2024-09-17T21:31:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457473+0000", 2024-09-17T21:31:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047199829999999998 2024-09-17T21:31:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457517+0000", 2024-09-17T21:31:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4058999999999999e-05 2024-09-17T21:31:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457559+0000", 2024-09-17T21:31:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2404999999999998e-05 2024-09-17T21:31:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17620 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 927744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.173145222000002, 2024-09-17T21:31:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047402409, 2024-09-17T21:31:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17620 2024-09-17T21:31:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:31:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410278+0000", 2024-09-17T21:31:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1790000000000001e-06 2024-09-17T21:31:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410281+0000", 2024-09-17T21:31:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.061e-06 2024-09-17T21:31:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410284+0000", 2024-09-17T21:31:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.65e-06 2024-09-17T21:31:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457587+0000", 2024-09-17T21:31:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047303206 2024-09-17T21:31:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457633+0000", 2024-09-17T21:31:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6063999999999998e-05 2024-09-17T21:31:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457679+0000", 2024-09-17T21:31:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6007000000000002e-05 2024-09-17T21:31:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17621 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 940032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:31:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.173134805999993, 2024-09-17T21:31:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047486550000000002, 2024-09-17T21:31:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17621 2024-09-17T21:31:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:31:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:31:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410288+0000", 2024-09-17T21:31:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:31:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410289+0000", 2024-09-17T21:31:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1179999999999999e-06 2024-09-17T21:31:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410292+0000", 2024-09-17T21:31:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:31:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410294+0000", 2024-09-17T21:31:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T21:31:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457694+0000", 2024-09-17T21:31:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047399416999999999 2024-09-17T21:31:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457734+0000", 2024-09-17T21:31:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0528999999999998e-05 2024-09-17T21:31:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457774+0000", 2024-09-17T21:31:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9529e-05 2024-09-17T21:31:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17622 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.173124623999996, 2024-09-17T21:31:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047596619, 2024-09-17T21:31:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17622 2024-09-17T21:31:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:31:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410303+0000", 2024-09-17T21:31:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2979999999999999e-06 2024-09-17T21:31:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410307+0000", 2024-09-17T21:31:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:31:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410310+0000", 2024-09-17T21:31:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.146e-06 2024-09-17T21:31:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457801+0000", 2024-09-17T21:31:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047491322000000002 2024-09-17T21:31:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457850+0000", 2024-09-17T21:31:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8585000000000003e-05 2024-09-17T21:31:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457894+0000", 2024-09-17T21:31:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4455e-05 2024-09-17T21:31:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17623 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 968704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.173108240000005, 2024-09-17T21:31:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047689377999999998, 2024-09-17T21:31:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17623 2024-09-17T21:31:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:31:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410315+0000", 2024-09-17T21:31:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0499999999999999e-06 2024-09-17T21:31:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410319+0000", 2024-09-17T21:31:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.517e-06 2024-09-17T21:31:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410321+0000", 2024-09-17T21:31:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:31:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457922+0000", 2024-09-17T21:31:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047600638000000001 2024-09-17T21:31:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457964+0000", 2024-09-17T21:31:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2126000000000003e-05 2024-09-17T21:31:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458003+0000", 2024-09-17T21:31:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9138999999999999e-05 2024-09-17T21:31:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17624 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.173097655999996, 2024-09-17T21:31:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047798292999999999, 2024-09-17T21:31:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17624 2024-09-17T21:31:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:31:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410326+0000", 2024-09-17T21:31:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0610000000000001e-06 2024-09-17T21:31:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410329+0000", 2024-09-17T21:31:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0319999999999999e-06 2024-09-17T21:31:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410331+0000", 2024-09-17T21:31:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5749999999999999e-06 2024-09-17T21:31:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458033+0000", 2024-09-17T21:31:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047701371999999999 2024-09-17T21:31:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458079+0000", 2024-09-17T21:31:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:31:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458123+0000", 2024-09-17T21:31:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850999999999997e-05 2024-09-17T21:31:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17625 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.173087495999994, 2024-09-17T21:31:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047883051000000003, 2024-09-17T21:31:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17625 2024-09-17T21:31:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:31:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410336+0000", 2024-09-17T21:31:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1310000000000001e-06 2024-09-17T21:31:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410339+0000", 2024-09-17T21:31:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000001e-06 2024-09-17T21:31:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410342+0000", 2024-09-17T21:31:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7460000000000001e-06 2024-09-17T21:31:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458137+0000", 2024-09-17T21:31:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047794931999999998 2024-09-17T21:31:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458178+0000", 2024-09-17T21:31:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1013999999999997e-05 2024-09-17T21:31:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458218+0000", 2024-09-17T21:31:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9675999999999998e-05 2024-09-17T21:31:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17626 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1009664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:31:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.173076875000007, 2024-09-17T21:31:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047991601000000002, 2024-09-17T21:31:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17626 2024-09-17T21:31:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:31:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:31:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410346+0000", 2024-09-17T21:31:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:31:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410347+0000", 2024-09-17T21:31:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.114e-06 2024-09-17T21:31:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410350+0000", 2024-09-17T21:31:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.101e-06 2024-09-17T21:31:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410353+0000", 2024-09-17T21:31:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T21:31:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458245+0000", 2024-09-17T21:31:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0478921 2024-09-17T21:31:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458292+0000", 2024-09-17T21:31:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397999999999999e-05 2024-09-17T21:31:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458337+0000", 2024-09-17T21:31:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5221e-05 2024-09-17T21:31:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17627 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:31:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.173039774000003, 2024-09-17T21:31:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048047383999999999, 2024-09-17T21:31:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17627 2024-09-17T21:31:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:31:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:31:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410383+0000", 2024-09-17T21:31:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:31:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410385+0000", 2024-09-17T21:31:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:31:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410391+0000", 2024-09-17T21:31:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:31:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410395+0000", 2024-09-17T21:31:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6539999999999998e-06 2024-09-17T21:31:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458350+0000", 2024-09-17T21:31:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047954855999999997 2024-09-17T21:31:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458392+0000", 2024-09-17T21:31:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2094000000000001e-05 2024-09-17T21:31:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458430+0000", 2024-09-17T21:31:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7474000000000001e-05 2024-09-17T21:31:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17628 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.173020213000001, 2024-09-17T21:31:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048142676000000002, 2024-09-17T21:31:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17628 2024-09-17T21:31:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:31:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410404+0000", 2024-09-17T21:31:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-06 2024-09-17T21:31:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410411+0000", 2024-09-17T21:31:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6850000000000004e-06 2024-09-17T21:31:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410415+0000", 2024-09-17T21:31:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6750000000000001e-06 2024-09-17T21:31:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458457+0000", 2024-09-17T21:31:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048042045999999998 2024-09-17T21:31:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458501+0000", 2024-09-17T21:31:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3661000000000001e-05 2024-09-17T21:31:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458545+0000", 2024-09-17T21:31:12.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3529000000000003e-05 2024-09-17T21:31:12.658 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.658 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.658 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.658 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17687 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:12.658 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:31:12.658 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.170575260000007, 2024-09-17T21:31:12.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050670448, 2024-09-17T21:31:12.659 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.659 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.659 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:12.659 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.659 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.659 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.659 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17687 2024-09-17T21:31:12.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.659 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:31:12.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:31:12.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412848+0000", 2024-09-17T21:31:12.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9100000000000003e-07 2024-09-17T21:31:12.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413102+0000", 2024-09-17T21:31:12.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025415600000000001 2024-09-17T21:31:12.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413111+0000", 2024-09-17T21:31:12.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2010000000000005e-06 2024-09-17T21:31:12.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413116+0000", 2024-09-17T21:31:12.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:31:12.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458559+0000", 2024-09-17T21:31:12.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045442914000000001 2024-09-17T21:31:12.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458632+0000", 2024-09-17T21:31:12.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3207000000000006e-05 2024-09-17T21:31:12.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:31:12.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458785+0000", 2024-09-17T21:31:12.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015226700000000001 2024-09-17T21:31:12.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:12.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.461648+0000", 2024-09-17T21:31:12.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028633069999999998 2024-09-17T21:31:12.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:12.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463416+0000", 2024-09-17T21:31:12.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017680230000000001 2024-09-17T21:31:12.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:12.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463475+0000", 2024-09-17T21:31:12.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9333000000000003e-05 2024-09-17T21:31:12.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463517+0000", 2024-09-17T21:31:12.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2162000000000001e-05 2024-09-17T21:31:12.664 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.664 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.664 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.664 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17688 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [write 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:12.664 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:12.665 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.170296391999997, 2024-09-17T21:31:12.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050625098, 2024-09-17T21:31:12.665 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.665 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.665 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:12.665 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.665 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.665 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.665 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17688 2024-09-17T21:31:12.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.665 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:12.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:12.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:12.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:31:12.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413128+0000", 2024-09-17T21:31:12.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.813e-06 2024-09-17T21:31:12.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413132+0000", 2024-09-17T21:31:12.667 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:12.583+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:12.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:31:12.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413135+0000", 2024-09-17T21:31:12.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:31:12.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459756+0000", 2024-09-17T21:31:12.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046621293000000001 2024-09-17T21:31:12.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459816+0000", 2024-09-17T21:31:12.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9156000000000001e-05 2024-09-17T21:31:12.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:31:12.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459912+0000", 2024-09-17T21:31:12.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6302000000000001e-05 2024-09-17T21:31:12.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:12.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463231+0000", 2024-09-17T21:31:12.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033188290000000001 2024-09-17T21:31:12.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:12.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463701+0000", 2024-09-17T21:31:12.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047068199999999999 2024-09-17T21:31:12.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:12.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463728+0000", 2024-09-17T21:31:12.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6726000000000001e-05 2024-09-17T21:31:12.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463751+0000", 2024-09-17T21:31:12.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2824e-05 2024-09-17T21:31:12.671 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.671 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.671 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.671 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17691 3.d 3:b3a055f3:::rbd_data.11207744cc2c.0000000000000083:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:12.671 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:31:12.672 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.166751489000006, 2024-09-17T21:31:12.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049309370999999998, 2024-09-17T21:31:12.672 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.672 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.672 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:12.672 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.672 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.672 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.672 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17691 2024-09-17T21:31:12.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.672 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:31:12.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:31:12.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416672+0000", 2024-09-17T21:31:12.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.47e-07 2024-09-17T21:31:12.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416915+0000", 2024-09-17T21:31:12.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024344500000000001 2024-09-17T21:31:12.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416924+0000", 2024-09-17T21:31:12.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7739999999999993e-06 2024-09-17T21:31:12.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416929+0000", 2024-09-17T21:31:12.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6990000000000002e-06 2024-09-17T21:31:12.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460414+0000", 2024-09-17T21:31:12.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043485466 2024-09-17T21:31:12.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460540+0000", 2024-09-17T21:31:12.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012608899999999999 2024-09-17T21:31:12.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:31:12.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460746+0000", 2024-09-17T21:31:12.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000205577 2024-09-17T21:31:12.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:12.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463800+0000", 2024-09-17T21:31:12.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030542030000000001 2024-09-17T21:31:12.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:12.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465867+0000", 2024-09-17T21:31:12.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020665929999999998 2024-09-17T21:31:12.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:12.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465914+0000", 2024-09-17T21:31:12.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497000000000003e-05 2024-09-17T21:31:12.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465980+0000", 2024-09-17T21:31:12.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6081000000000006e-05 2024-09-17T21:31:12.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.677 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.677 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:70218 3.16 3:68692d8f:::rbd_data.1120b9193b9f.000000000000001b:head [stat,delete] snapc 1d=[1d] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-17T21:31:12.677 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:12.677 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.777784400000002, 2024-09-17T21:31:12.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.951400786, 2024-09-17T21:31:12.678 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:12.678 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:12.678 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:12.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:12.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:12.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:12.678 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 70218 2024-09-17T21:31:12.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.678 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:12.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:12.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:12.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:12.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:12.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:12.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:12.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:12.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4799999999999998e-07 2024-09-17T21:31:12.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:12.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805642+0000", 2024-09-17T21:31:12.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2009999999999999e-06 2024-09-17T21:31:12.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:12.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805650+0000", 2024-09-17T21:31:12.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1000000000000004e-06 2024-09-17T21:31:12.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:12.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805655+0000", 2024-09-17T21:31:12.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:31:12.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805722+0000", 2024-09-17T21:31:12.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6878000000000006e-05 2024-09-17T21:31:12.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:31:12.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805733+0000", 2024-09-17T21:31:12.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0895e-05 2024-09-17T21:31:12.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:12.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755252+0000", 2024-09-17T21:31:12.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.94951843800000002 2024-09-17T21:31:12.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:12.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755398+0000", 2024-09-17T21:31:12.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014622600000000001 2024-09-17T21:31:12.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:12.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:12.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:12.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.757039+0000", 2024-09-17T21:31:12.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001640819 2024-09-17T21:31:12.682 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.682 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.682 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.683 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:12.683 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:12.683 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:12.683 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:31:12.726 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:12.724+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:12.787 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:12.785+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:12.789 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:12.789 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:12.789 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:12.789 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:12.798 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:31:12.888 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:12.886+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:12.943 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:12.943 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:12.943 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:12.943 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:12.943 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:12.952 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:31:12.996 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:12.993+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:13.062 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97778 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 245760~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:31:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092538474000001, 2024-09-17T21:31:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050933728999999997, 2024-09-17T21:31:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97778 2024-09-17T21:31:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:31:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:31:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966899+0000", 2024-09-17T21:31:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:31:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966900+0000", 2024-09-17T21:31:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.384e-06 2024-09-17T21:31:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966905+0000", 2024-09-17T21:31:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6530000000000002e-06 2024-09-17T21:31:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966909+0000", 2024-09-17T21:31:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2889999999999999e-06 2024-09-17T21:31:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017585+0000", 2024-09-17T21:31:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050675691000000002 2024-09-17T21:31:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017642+0000", 2024-09-17T21:31:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7167999999999999e-05 2024-09-17T21:31:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017832+0000", 2024-09-17T21:31:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019022 2024-09-17T21:31:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97779 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:31:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092521405999999, 2024-09-17T21:31:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051245027999999998, 2024-09-17T21:31:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97779 2024-09-17T21:31:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:31:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:31:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966916+0000", 2024-09-17T21:31:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:31:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966918+0000", 2024-09-17T21:31:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8369999999999999e-06 2024-09-17T21:31:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966924+0000", 2024-09-17T21:31:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2260000000000002e-06 2024-09-17T21:31:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966928+0000", 2024-09-17T21:31:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3819999999999997e-06 2024-09-17T21:31:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017866+0000", 2024-09-17T21:31:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050938079999999997 2024-09-17T21:31:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017945+0000", 2024-09-17T21:31:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8275000000000005e-05 2024-09-17T21:31:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018161+0000", 2024-09-17T21:31:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021579499999999999 2024-09-17T21:31:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97780 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 344064~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092497128000002, 2024-09-17T21:31:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051535301999999998, 2024-09-17T21:31:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97780 2024-09-17T21:31:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:31:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966942+0000", 2024-09-17T21:31:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7439999999999999e-06 2024-09-17T21:31:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966946+0000", 2024-09-17T21:31:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5510000000000001e-06 2024-09-17T21:31:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966951+0000", 2024-09-17T21:31:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:31:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018191+0000", 2024-09-17T21:31:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051240290000000001 2024-09-17T21:31:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018269+0000", 2024-09-17T21:31:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7416999999999994e-05 2024-09-17T21:31:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018475+0000", 2024-09-17T21:31:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206486 2024-09-17T21:31:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97789 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 40960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092476958999999, 2024-09-17T21:31:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051862143999999999, 2024-09-17T21:31:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97789 2024-09-17T21:31:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5899999999999996e-07 2024-09-17T21:31:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966962+0000", 2024-09-17T21:31:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9700000000000002e-06 2024-09-17T21:31:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966969+0000", 2024-09-17T21:31:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6939999999999998e-06 2024-09-17T21:31:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966980+0000", 2024-09-17T21:31:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0679999999999999e-05 2024-09-17T21:31:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018513+0000", 2024-09-17T21:31:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051533420000000003 2024-09-17T21:31:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018595+0000", 2024-09-17T21:31:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1260999999999993e-05 2024-09-17T21:31:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018822+0000", 2024-09-17T21:31:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022756 2024-09-17T21:31:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97790 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 90112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:31:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092446461000002, 2024-09-17T21:31:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052173479000000002, 2024-09-17T21:31:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97790 2024-09-17T21:31:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:31:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:31:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966991+0000", 2024-09-17T21:31:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9500000000000003e-07 2024-09-17T21:31:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966993+0000", 2024-09-17T21:31:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.238e-06 2024-09-17T21:31:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967000+0000", 2024-09-17T21:31:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9500000000000004e-06 2024-09-17T21:31:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967005+0000", 2024-09-17T21:31:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:31:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018852+0000", 2024-09-17T21:31:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051847250999999997 2024-09-17T21:31:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018927+0000", 2024-09-17T21:31:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4944000000000006e-05 2024-09-17T21:31:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019164+0000", 2024-09-17T21:31:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000236803 2024-09-17T21:31:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97791 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 139264~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:31:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092424717, 2024-09-17T21:31:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052464594000000003, 2024-09-17T21:31:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97791 2024-09-17T21:31:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:31:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:31:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967013+0000", 2024-09-17T21:31:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:31:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967014+0000", 2024-09-17T21:31:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8679999999999999e-06 2024-09-17T21:31:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967020+0000", 2024-09-17T21:31:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.322e-06 2024-09-17T21:31:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967024+0000", 2024-09-17T21:31:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2660000000000003e-06 2024-09-17T21:31:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019193+0000", 2024-09-17T21:31:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052169010000000002 2024-09-17T21:31:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019269+0000", 2024-09-17T21:31:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6149999999999994e-05 2024-09-17T21:31:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019477+0000", 2024-09-17T21:31:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020753699999999999 2024-09-17T21:31:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97792 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 188416~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092406978, 2024-09-17T21:31:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052764958000000001, 2024-09-17T21:31:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97792 2024-09-17T21:31:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0900000000000002e-07 2024-09-17T21:31:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967032+0000", 2024-09-17T21:31:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9740000000000001e-06 2024-09-17T21:31:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967039+0000", 2024-09-17T21:31:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3679999999999998e-06 2024-09-17T21:31:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967043+0000", 2024-09-17T21:31:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3379999999999998e-06 2024-09-17T21:31:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019505+0000", 2024-09-17T21:31:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052462264000000002 2024-09-17T21:31:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019582+0000", 2024-09-17T21:31:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7146999999999998e-05 2024-09-17T21:31:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019795+0000", 2024-09-17T21:31:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000212458 2024-09-17T21:31:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97793 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 237568~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:31:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092385478000001, 2024-09-17T21:31:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053070436999999998, 2024-09-17T21:31:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97793 2024-09-17T21:31:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:31:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:31:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967052+0000", 2024-09-17T21:31:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7500000000000001e-07 2024-09-17T21:31:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967054+0000", 2024-09-17T21:31:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:31:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967059+0000", 2024-09-17T21:31:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5840000000000002e-06 2024-09-17T21:31:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967064+0000", 2024-09-17T21:31:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:31:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019826+0000", 2024-09-17T21:31:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052762396000000003 2024-09-17T21:31:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019900+0000", 2024-09-17T21:31:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3594e-05 2024-09-17T21:31:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020122+0000", 2024-09-17T21:31:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022185999999999999 2024-09-17T21:31:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97794 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 286720~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092365342000001, 2024-09-17T21:31:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053453734000000003, 2024-09-17T21:31:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97794 2024-09-17T21:31:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2199999999999999e-07 2024-09-17T21:31:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967074+0000", 2024-09-17T21:31:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8419999999999999e-06 2024-09-17T21:31:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967080+0000", 2024-09-17T21:31:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5660000000000003e-06 2024-09-17T21:31:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967120+0000", 2024-09-17T21:31:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9504e-05 2024-09-17T21:31:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020189+0000", 2024-09-17T21:31:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053069144999999998 2024-09-17T21:31:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020287+0000", 2024-09-17T21:31:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8259999999999998e-05 2024-09-17T21:31:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020525+0000", 2024-09-17T21:31:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023799499999999999 2024-09-17T21:31:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97795 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 335872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092307208000001, 2024-09-17T21:31:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053707207999999999, 2024-09-17T21:31:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97795 2024-09-17T21:31:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:31:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967139+0000", 2024-09-17T21:31:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5790000000000004e-06 2024-09-17T21:31:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967145+0000", 2024-09-17T21:31:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3550000000000001e-06 2024-09-17T21:31:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967155+0000", 2024-09-17T21:31:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8859999999999999e-06 2024-09-17T21:31:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020555+0000", 2024-09-17T21:31:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053400359000000001 2024-09-17T21:31:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020630+0000", 2024-09-17T21:31:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4894999999999994e-05 2024-09-17T21:31:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020837+0000", 2024-09-17T21:31:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020670500000000001 2024-09-17T21:31:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97796 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 385024~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092274948, 2024-09-17T21:31:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053990227000000002, 2024-09-17T21:31:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97796 2024-09-17T21:31:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:31:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967164+0000", 2024-09-17T21:31:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0370000000000001e-06 2024-09-17T21:31:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967171+0000", 2024-09-17T21:31:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2190000000000001e-06 2024-09-17T21:31:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967175+0000", 2024-09-17T21:31:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2599999999999999e-06 2024-09-17T21:31:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020866+0000", 2024-09-17T21:31:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053691539000000003 2024-09-17T21:31:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020942+0000", 2024-09-17T21:31:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5487000000000004e-05 2024-09-17T21:31:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021152+0000", 2024-09-17T21:31:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021021599999999999 2024-09-17T21:31:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97805 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 8192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092256268, 2024-09-17T21:31:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054331771000000001, 2024-09-17T21:31:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97805 2024-09-17T21:31:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:31:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967183+0000", 2024-09-17T21:31:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6840000000000001e-06 2024-09-17T21:31:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967189+0000", 2024-09-17T21:31:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0299999999999999e-06 2024-09-17T21:31:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967193+0000", 2024-09-17T21:31:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-17T21:31:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021182+0000", 2024-09-17T21:31:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053989319000000001 2024-09-17T21:31:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021261+0000", 2024-09-17T21:31:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8460000000000004e-05 2024-09-17T21:31:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021512+0000", 2024-09-17T21:31:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025158500000000002 2024-09-17T21:31:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97806 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092238264999999, 2024-09-17T21:31:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054654343000000001, 2024-09-17T21:31:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97806 2024-09-17T21:31:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:31:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967201+0000", 2024-09-17T21:31:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.691e-06 2024-09-17T21:31:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967206+0000", 2024-09-17T21:31:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7780000000000004e-06 2024-09-17T21:31:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967224+0000", 2024-09-17T21:31:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7357999999999999e-05 2024-09-17T21:31:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021546+0000", 2024-09-17T21:31:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054322237000000002 2024-09-17T21:31:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021618+0000", 2024-09-17T21:31:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2007000000000004e-05 2024-09-17T21:31:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021853+0000", 2024-09-17T21:31:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023495800000000001 2024-09-17T21:31:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97807 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 106496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:31:13.119 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092205865, 2024-09-17T21:31:13.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055014183000000001, 2024-09-17T21:31:13.120 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.120 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.120 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.120 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.121 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97807 2024-09-17T21:31:13.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.121 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:31:13.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:31:13.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967232+0000", 2024-09-17T21:31:13.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:31:13.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967234+0000", 2024-09-17T21:31:13.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.283e-06 2024-09-17T21:31:13.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967240+0000", 2024-09-17T21:31:13.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.144e-06 2024-09-17T21:31:13.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967252+0000", 2024-09-17T21:31:13.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2322e-05 2024-09-17T21:31:13.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021882+0000", 2024-09-17T21:31:13.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054629952000000002 2024-09-17T21:31:13.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021952+0000", 2024-09-17T21:31:13.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9919999999999995e-05 2024-09-17T21:31:13.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022245+0000", 2024-09-17T21:31:13.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00029302100000000003 2024-09-17T21:31:13.124 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.124 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.124 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.124 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97808 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 155648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.124 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:13.124 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092177266, 2024-09-17T21:31:13.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055447485999999997, 2024-09-17T21:31:13.125 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.125 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.125 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.125 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.125 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.125 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.125 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97808 2024-09-17T21:31:13.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.125 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:13.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:13.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:13.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4299999999999998e-07 2024-09-17T21:31:13.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967267+0000", 2024-09-17T21:31:13.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9820000000000002e-06 2024-09-17T21:31:13.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967273+0000", 2024-09-17T21:31:13.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:31:13.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967277+0000", 2024-09-17T21:31:13.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5050000000000001e-06 2024-09-17T21:31:13.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022297+0000", 2024-09-17T21:31:13.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055020131999999999 2024-09-17T21:31:13.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022379+0000", 2024-09-17T21:31:13.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1810000000000004e-05 2024-09-17T21:31:13.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022707+0000", 2024-09-17T21:31:13.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00032784300000000003 2024-09-17T21:31:13.128 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.128 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.128 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.128 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97809 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.129 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:31:13.129 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092153751000001, 2024-09-17T21:31:13.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056012109999999997, 2024-09-17T21:31:13.129 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.129 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.129 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.129 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.129 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.129 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.129 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97809 2024-09-17T21:31:13.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.129 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:31:13.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:31:13.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967284+0000", 2024-09-17T21:31:13.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:31:13.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967285+0000", 2024-09-17T21:31:13.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.826e-06 2024-09-17T21:31:13.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967291+0000", 2024-09-17T21:31:13.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:31:13.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967302+0000", 2024-09-17T21:31:13.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1114999999999999e-05 2024-09-17T21:31:13.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022901+0000", 2024-09-17T21:31:13.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055599335 2024-09-17T21:31:13.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022990+0000", 2024-09-17T21:31:13.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8671999999999994e-05 2024-09-17T21:31:13.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023295+0000", 2024-09-17T21:31:13.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030554900000000001 2024-09-17T21:31:13.132 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.132 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.133 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.133 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97810 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 253952~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.133 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:13.133 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092127323, 2024-09-17T21:31:13.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056355668999999997, 2024-09-17T21:31:13.133 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.133 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.133 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.133 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.133 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.133 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.133 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97810 2024-09-17T21:31:13.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.134 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:13.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:13.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:13.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0200000000000002e-07 2024-09-17T21:31:13.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967313+0000", 2024-09-17T21:31:13.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4549999999999998e-06 2024-09-17T21:31:13.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967319+0000", 2024-09-17T21:31:13.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3080000000000004e-06 2024-09-17T21:31:13.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967324+0000", 2024-09-17T21:31:13.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0159999999999999e-06 2024-09-17T21:31:13.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023342+0000", 2024-09-17T21:31:13.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056018635999999997 2024-09-17T21:31:13.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023423+0000", 2024-09-17T21:31:13.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0473e-05 2024-09-17T21:31:13.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023665+0000", 2024-09-17T21:31:13.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242279 2024-09-17T21:31:13.137 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.137 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.137 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.137 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97811 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 303104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.137 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:13.137 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092101065000001, 2024-09-17T21:31:13.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056743978, 2024-09-17T21:31:13.137 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.137 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.137 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.138 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.138 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.138 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.138 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97811 2024-09-17T21:31:13.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.138 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:13.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:13.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:13.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1300000000000001e-07 2024-09-17T21:31:13.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967338+0000", 2024-09-17T21:31:13.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9740000000000001e-06 2024-09-17T21:31:13.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967346+0000", 2024-09-17T21:31:13.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8129999999999996e-06 2024-09-17T21:31:13.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967351+0000", 2024-09-17T21:31:13.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6129999999999997e-06 2024-09-17T21:31:13.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023750+0000", 2024-09-17T21:31:13.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056399796000000002 2024-09-17T21:31:13.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023839+0000", 2024-09-17T21:31:13.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.852e-05 2024-09-17T21:31:13.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024080+0000", 2024-09-17T21:31:13.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024084900000000001 2024-09-17T21:31:13.141 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.141 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.141 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.142 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97812 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 352256~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:13.142 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:31:13.142 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.092080449000001, 2024-09-17T21:31:13.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.057024455000000002, 2024-09-17T21:31:13.142 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.142 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.142 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.142 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.142 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.142 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.142 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97812 2024-09-17T21:31:13.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.143 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:31:13.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:31:13.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967357+0000", 2024-09-17T21:31:13.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:31:13.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967359+0000", 2024-09-17T21:31:13.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:31:13.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967365+0000", 2024-09-17T21:31:13.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:31:13.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967369+0000", 2024-09-17T21:31:13.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5029999999999999e-06 2024-09-17T21:31:13.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024109+0000", 2024-09-17T21:31:13.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056740038 2024-09-17T21:31:13.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024175+0000", 2024-09-17T21:31:13.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5106999999999999e-05 2024-09-17T21:31:13.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024381+0000", 2024-09-17T21:31:13.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020629700000000001 2024-09-17T21:31:13.146 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.146 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.146 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.146 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:112033 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-09-17T21:31:13.146 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:31:11.421210+0000", 2024-09-17T21:31:13.146 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.6382266830000001, 2024-09-17T21:31:13.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0104142869999999, 2024-09-17T21:31:13.146 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.147 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.147 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.147 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 112033 2024-09-17T21:31:13.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.147 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421210+0000", 2024-09-17T21:31:13.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421210+0000", 2024-09-17T21:31:13.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421212+0000", 2024-09-17T21:31:13.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.376e-06 2024-09-17T21:31:13.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421217+0000", 2024-09-17T21:31:13.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.259e-06 2024-09-17T21:31:13.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421228+0000", 2024-09-17T21:31:13.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0944000000000001e-05 2024-09-17T21:31:13.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421238+0000", 2024-09-17T21:31:13.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:31:13.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421626+0000", 2024-09-17T21:31:13.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00038804099999999999 2024-09-17T21:31:13.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:31:13.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421637+0000", 2024-09-17T21:31:13.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1783000000000001e-05 2024-09-17T21:31:13.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:12.431245+0000", 2024-09-17T21:31:13.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.00960765 2024-09-17T21:31:13.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:12.431413+0000", 2024-09-17T21:31:13.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000167784 2024-09-17T21:31:13.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:12.431624+0000", 2024-09-17T21:31:13.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000211704 2024-09-17T21:31:13.151 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.151 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.151 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.151 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.151 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.152 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:13.153 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:31:13.154 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:13.151+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:13.254 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:13.252+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:13.303 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:13.303 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:13.303 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:13.304 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:13.313 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:31:13.357 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:13.355+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:13.422 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:13.422 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:13.422 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:13.423 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:13.423 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:13.432 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:31:13.437 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is unset 2024-09-17T21:31:13.457 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:13.454+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:13.521 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:31:13.556 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:13.554+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:13.557 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:13.557 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:13.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:13.558 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:13.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.558 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:37817 3.17 3:eab85a8d:::rbd_header.1120e62eea89:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:31:13.558 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:31:13.558 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.85930544, 2024-09-17T21:31:13.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86635970399999995, 2024-09-17T21:31:13.558 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.558 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.558 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.558 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.558 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.558 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.558 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37817 2024-09-17T21:31:13.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.559 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:31:13.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:31:13.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695457+0000", 2024-09-17T21:31:13.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5799999999999998e-07 2024-09-17T21:31:13.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695461+0000", 2024-09-17T21:31:13.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1110000000000001e-06 2024-09-17T21:31:13.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695471+0000", 2024-09-17T21:31:13.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0441e-05 2024-09-17T21:31:13.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695477+0000", 2024-09-17T21:31:13.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7599999999999999e-06 2024-09-17T21:31:13.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695533+0000", 2024-09-17T21:31:13.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6570999999999998e-05 2024-09-17T21:31:13.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:31:13.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695543+0000", 2024-09-17T21:31:13.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9979999999999998e-06 2024-09-17T21:31:13.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561472+0000", 2024-09-17T21:31:13.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86592841899999995 2024-09-17T21:31:13.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561610+0000", 2024-09-17T21:31:13.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013854699999999999 2024-09-17T21:31:13.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561816+0000", 2024-09-17T21:31:13.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020509900000000001 2024-09-17T21:31:13.563 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.563 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.563 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.563 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50202 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.563 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:31:13.563 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.447134650000002, 2024-09-17T21:31:13.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.076620261999999995, 2024-09-17T21:31:13.563 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.563 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.563 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:13.563 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.564 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.564 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.564 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50202 2024-09-17T21:31:13.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.564 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:31:13.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:31:13.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107629+0000", 2024-09-17T21:31:13.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3709999999999998e-06 2024-09-17T21:31:13.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119765+0000", 2024-09-17T21:31:13.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012135683 2024-09-17T21:31:13.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119787+0000", 2024-09-17T21:31:13.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2748999999999999e-05 2024-09-17T21:31:13.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119797+0000", 2024-09-17T21:31:13.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8169999999999994e-06 2024-09-17T21:31:13.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119872+0000", 2024-09-17T21:31:13.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270999999999994e-05 2024-09-17T21:31:13.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119987+0000", 2024-09-17T21:31:13.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011557699999999999 2024-09-17T21:31:13.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:13.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.121766+0000", 2024-09-17T21:31:13.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00177853 2024-09-17T21:31:13.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:13.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178307+0000", 2024-09-17T21:31:13.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056541559999999998 2024-09-17T21:31:13.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:13.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184097+0000", 2024-09-17T21:31:13.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057899029999999999 2024-09-17T21:31:13.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:13.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184172+0000", 2024-09-17T21:31:13.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5226000000000001e-05 2024-09-17T21:31:13.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184247+0000", 2024-09-17T21:31:13.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4574999999999997e-05 2024-09-17T21:31:13.569 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.569 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.569 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.569 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50203 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [truncate 20971520] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.569 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:31:13.569 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.434934745, 2024-09-17T21:31:13.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065009779000000004, 2024-09-17T21:31:13.569 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.569 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.570 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:13.570 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.570 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.570 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.570 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50203 2024-09-17T21:31:13.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.570 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:31:13.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:31:13.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119828+0000", 2024-09-17T21:31:13.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6390000000000001e-06 2024-09-17T21:31:13.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119832+0000", 2024-09-17T21:31:13.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7699999999999999e-06 2024-09-17T21:31:13.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119845+0000", 2024-09-17T21:31:13.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3093e-05 2024-09-17T21:31:13.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119852+0000", 2024-09-17T21:31:13.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6950000000000003e-06 2024-09-17T21:31:13.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.160945+0000", 2024-09-17T21:31:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041093076999999999 2024-09-17T21:31:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161071+0000", 2024-09-17T21:31:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012569799999999999 2024-09-17T21:31:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161283+0000", 2024-09-17T21:31:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021201999999999999 2024-09-17T21:31:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.179360+0000", 2024-09-17T21:31:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018077784999999999 2024-09-17T21:31:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184726+0000", 2024-09-17T21:31:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053652960000000003 2024-09-17T21:31:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184787+0000", 2024-09-17T21:31:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1402000000000006e-05 2024-09-17T21:31:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184836+0000", 2024-09-17T21:31:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9304000000000001e-05 2024-09-17T21:31:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50206 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:31:13.576 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.397853204999997, 2024-09-17T21:31:13.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.090361968000000001, 2024-09-17T21:31:13.576 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.576 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.576 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:13.576 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.576 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.576 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.576 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50206 2024-09-17T21:31:13.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.576 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:31:13.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:31:13.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156911+0000", 2024-09-17T21:31:13.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-17T21:31:13.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173670+0000", 2024-09-17T21:31:13.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016758907 2024-09-17T21:31:13.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173696+0000", 2024-09-17T21:31:13.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6329000000000002e-05 2024-09-17T21:31:13.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173708+0000", 2024-09-17T21:31:13.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2512e-05 2024-09-17T21:31:13.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173788+0000", 2024-09-17T21:31:13.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9332999999999994e-05 2024-09-17T21:31:13.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173918+0000", 2024-09-17T21:31:13.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013063300000000001 2024-09-17T21:31:13.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:13.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.176228+0000", 2024-09-17T21:31:13.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002309757 2024-09-17T21:31:13.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:13.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.245164+0000", 2024-09-17T21:31:13.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068936301000000005 2024-09-17T21:31:13.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:13.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247176+0000", 2024-09-17T21:31:13.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020117590000000001 2024-09-17T21:31:13.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:13.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247215+0000", 2024-09-17T21:31:13.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9094000000000002e-05 2024-09-17T21:31:13.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247270+0000", 2024-09-17T21:31:13.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4793999999999997e-05 2024-09-17T21:31:13.582 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.582 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.582 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.582 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50207 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [truncate 21556327] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.582 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:31:13.582 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.381004224999998, 2024-09-17T21:31:13.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.074055363999999999, 2024-09-17T21:31:13.583 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.583 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.583 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:13.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.583 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50207 2024-09-17T21:31:13.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.583 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:31:13.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:31:13.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173759+0000", 2024-09-17T21:31:13.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8160000000000001e-06 2024-09-17T21:31:13.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173763+0000", 2024-09-17T21:31:13.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5360000000000003e-06 2024-09-17T21:31:13.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173771+0000", 2024-09-17T21:31:13.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8849999999999999e-06 2024-09-17T21:31:13.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173777+0000", 2024-09-17T21:31:13.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7799999999999997e-06 2024-09-17T21:31:13.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221363+0000", 2024-09-17T21:31:13.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047585493 2024-09-17T21:31:13.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221430+0000", 2024-09-17T21:31:13.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7129000000000001e-05 2024-09-17T21:31:13.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:13.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221550+0000", 2024-09-17T21:31:13.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120658 2024-09-17T21:31:13.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:13.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246113+0000", 2024-09-17T21:31:13.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02456262 2024-09-17T21:31:13.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:13.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247744+0000", 2024-09-17T21:31:13.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001630692 2024-09-17T21:31:13.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:13.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247777+0000", 2024-09-17T21:31:13.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3015000000000002e-05 2024-09-17T21:31:13.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247812+0000", 2024-09-17T21:31:13.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.574e-05 2024-09-17T21:31:13.588 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.588 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.588 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.588 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50828 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.588 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:13.588 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.072892293000002, 2024-09-17T21:31:13.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047324185999999997, 2024-09-17T21:31:13.588 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.588 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.588 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.588 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.589 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.589 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.589 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50828 2024-09-17T21:31:13.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.589 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:13.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:13.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:13.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.46e-07 2024-09-17T21:31:13.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481871+0000", 2024-09-17T21:31:13.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T21:31:13.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481878+0000", 2024-09-17T21:31:13.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:31:13.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481882+0000", 2024-09-17T21:31:13.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:31:13.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529038+0000", 2024-09-17T21:31:13.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047156034999999999 2024-09-17T21:31:13.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529099+0000", 2024-09-17T21:31:13.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0316000000000003e-05 2024-09-17T21:31:13.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529193+0000", 2024-09-17T21:31:13.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4369000000000004e-05 2024-09-17T21:31:13.592 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.592 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.592 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.592 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50829 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 12845056~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.592 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:13.593 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.072873499000004, 2024-09-17T21:31:13.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047887596999999997, 2024-09-17T21:31:13.593 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.593 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.593 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.593 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.593 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.593 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.593 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50829 2024-09-17T21:31:13.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.593 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:13.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:13.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:13.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:31:13.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481890+0000", 2024-09-17T21:31:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.877e-06 2024-09-17T21:31:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481895+0000", 2024-09-17T21:31:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:31:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481900+0000", 2024-09-17T21:31:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4440000000000001e-06 2024-09-17T21:31:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529225+0000", 2024-09-17T21:31:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047325117999999999 2024-09-17T21:31:13.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529292+0000", 2024-09-17T21:31:13.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7100000000000005e-05 2024-09-17T21:31:13.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529775+0000", 2024-09-17T21:31:13.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048322799999999999 2024-09-17T21:31:13.596 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.596 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.596 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.597 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50830 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.597 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:31:13.597 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.072855097000001, 2024-09-17T21:31:13.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048062123999999998, 2024-09-17T21:31:13.597 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.597 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.597 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.597 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.597 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.597 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.597 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50830 2024-09-17T21:31:13.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.597 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:31:13.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:31:13.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481907+0000", 2024-09-17T21:31:13.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:31:13.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481909+0000", 2024-09-17T21:31:13.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T21:31:13.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481915+0000", 2024-09-17T21:31:13.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.02e-06 2024-09-17T21:31:13.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481920+0000", 2024-09-17T21:31:13.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6799999999999998e-06 2024-09-17T21:31:13.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529793+0000", 2024-09-17T21:31:13.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047873042999999997 2024-09-17T21:31:13.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529870+0000", 2024-09-17T21:31:13.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6967999999999999e-05 2024-09-17T21:31:13.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529968+0000", 2024-09-17T21:31:13.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8085999999999996e-05 2024-09-17T21:31:13.601 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.601 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.601 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.601 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50831 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 18087936~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.601 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:13.601 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.072834751000002, 2024-09-17T21:31:13.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048663304999999997, 2024-09-17T21:31:13.601 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.601 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.601 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.602 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.602 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.602 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.602 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50831 2024-09-17T21:31:13.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.602 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:13.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:13.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:13.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:31:13.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481929+0000", 2024-09-17T21:31:13.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2189999999999998e-06 2024-09-17T21:31:13.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481935+0000", 2024-09-17T21:31:13.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.383e-06 2024-09-17T21:31:13.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481939+0000", 2024-09-17T21:31:13.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2479999999999998e-06 2024-09-17T21:31:13.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530095+0000", 2024-09-17T21:31:13.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048156617999999998 2024-09-17T21:31:13.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530159+0000", 2024-09-17T21:31:13.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3394999999999999e-05 2024-09-17T21:31:13.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530590+0000", 2024-09-17T21:31:13.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043101199999999999 2024-09-17T21:31:13.605 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.605 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.605 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.605 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50832 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 19660800~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.605 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:31:13.606 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.072816103999997, 2024-09-17T21:31:13.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049339466999999998, 2024-09-17T21:31:13.606 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.606 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.606 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.606 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.606 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.606 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.606 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50832 2024-09-17T21:31:13.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.606 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:31:13.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:31:13.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481946+0000", 2024-09-17T21:31:13.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:13.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481951+0000", 2024-09-17T21:31:13.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1479999999999997e-06 2024-09-17T21:31:13.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481956+0000", 2024-09-17T21:31:13.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7339999999999996e-06 2024-09-17T21:31:13.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481961+0000", 2024-09-17T21:31:13.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4889999999999997e-06 2024-09-17T21:31:13.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530630+0000", 2024-09-17T21:31:13.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048668596000000001 2024-09-17T21:31:13.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530720+0000", 2024-09-17T21:31:13.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0163999999999999e-05 2024-09-17T21:31:13.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531285+0000", 2024-09-17T21:31:13.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056493199999999998 2024-09-17T21:31:13.609 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.609 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.609 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.609 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50833 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.610 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:31:13.610 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.072793959999998, 2024-09-17T21:31:13.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049908069999999999, 2024-09-17T21:31:13.610 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.610 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.610 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.610 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.610 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.610 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.610 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50833 2024-09-17T21:31:13.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.610 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:31:13.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:31:13.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481968+0000", 2024-09-17T21:31:13.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5600000000000001e-07 2024-09-17T21:31:13.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481970+0000", 2024-09-17T21:31:13.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:31:13.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481975+0000", 2024-09-17T21:31:13.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8499999999999999e-06 2024-09-17T21:31:13.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481980+0000", 2024-09-17T21:31:13.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-17T21:31:13.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531337+0000", 2024-09-17T21:31:13.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049356874000000002 2024-09-17T21:31:13.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531392+0000", 2024-09-17T21:31:13.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4826e-05 2024-09-17T21:31:13.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531875+0000", 2024-09-17T21:31:13.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048345099999999998 2024-09-17T21:31:13.613 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.613 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.614 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.614 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50834 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 24903680~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.614 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:31:13.614 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.072769192000003, 2024-09-17T21:31:13.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050534015000000002, 2024-09-17T21:31:13.614 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.614 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.614 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.614 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.614 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.614 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.614 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50834 2024-09-17T21:31:13.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.614 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:31:13.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:31:13.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481993+0000", 2024-09-17T21:31:13.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:31:13.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481995+0000", 2024-09-17T21:31:13.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.199e-06 2024-09-17T21:31:13.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482001+0000", 2024-09-17T21:31:13.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.772e-06 2024-09-17T21:31:13.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482006+0000", 2024-09-17T21:31:13.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0660000000000003e-06 2024-09-17T21:31:13.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531923+0000", 2024-09-17T21:31:13.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049917200000000002 2024-09-17T21:31:13.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532004+0000", 2024-09-17T21:31:13.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0758000000000002e-05 2024-09-17T21:31:13.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532526+0000", 2024-09-17T21:31:13.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00052258899999999995 2024-09-17T21:31:13.617 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.618 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.618 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.618 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50835 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 28573696~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.618 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:31:13.618 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.072749209000001, 2024-09-17T21:31:13.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051204701999999998, 2024-09-17T21:31:13.618 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.618 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.618 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.618 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.618 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.618 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.618 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50835 2024-09-17T21:31:13.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.619 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:31:13.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:31:13.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482013+0000", 2024-09-17T21:31:13.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:31:13.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482015+0000", 2024-09-17T21:31:13.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.272e-06 2024-09-17T21:31:13.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482020+0000", 2024-09-17T21:31:13.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3040000000000004e-06 2024-09-17T21:31:13.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482025+0000", 2024-09-17T21:31:13.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5650000000000003e-06 2024-09-17T21:31:13.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532569+0000", 2024-09-17T21:31:13.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050544661999999997 2024-09-17T21:31:13.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532649+0000", 2024-09-17T21:31:13.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9640000000000003e-05 2024-09-17T21:31:13.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533217+0000", 2024-09-17T21:31:13.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056784100000000003 2024-09-17T21:31:13.622 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.622 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.622 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.622 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50836 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 30146560~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.622 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:13.622 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.072729592000002, 2024-09-17T21:31:13.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051875589, 2024-09-17T21:31:13.622 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.622 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.622 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.622 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.622 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.622 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.623 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50836 2024-09-17T21:31:13.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.623 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:13.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:13.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:13.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3700000000000001e-07 2024-09-17T21:31:13.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482034+0000", 2024-09-17T21:31:13.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0059999999999999e-06 2024-09-17T21:31:13.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482040+0000", 2024-09-17T21:31:13.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:31:13.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482044+0000", 2024-09-17T21:31:13.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:31:13.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533260+0000", 2024-09-17T21:31:13.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051215416 2024-09-17T21:31:13.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533348+0000", 2024-09-17T21:31:13.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8695000000000004e-05 2024-09-17T21:31:13.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533907+0000", 2024-09-17T21:31:13.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055902199999999995 2024-09-17T21:31:13.626 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.626 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.626 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.626 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50844 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.626 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:31:13.626 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.072566166000001, 2024-09-17T21:31:13.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0475644, 2024-09-17T21:31:13.626 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.626 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.627 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.627 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50844 2024-09-17T21:31:13.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.627 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:31:13.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:31:13.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482196+0000", 2024-09-17T21:31:13.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.39e-07 2024-09-17T21:31:13.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482198+0000", 2024-09-17T21:31:13.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0099999999999998e-06 2024-09-17T21:31:13.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482204+0000", 2024-09-17T21:31:13.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.93e-06 2024-09-17T21:31:13.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482208+0000", 2024-09-17T21:31:13.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7439999999999998e-06 2024-09-17T21:31:13.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529318+0000", 2024-09-17T21:31:13.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047110051999999999 2024-09-17T21:31:13.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529396+0000", 2024-09-17T21:31:13.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7712000000000004e-05 2024-09-17T21:31:13.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529760+0000", 2024-09-17T21:31:13.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00036351299999999999 2024-09-17T21:31:13.630 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.630 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.630 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.630 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50845 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 21757952~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.630 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:31:13.630 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.072547063000002, 2024-09-17T21:31:13.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048084625999999998, 2024-09-17T21:31:13.631 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.631 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.631 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.631 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.631 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.631 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.631 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50845 2024-09-17T21:31:13.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.631 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:31:13.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:31:13.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482215+0000", 2024-09-17T21:31:13.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:31:13.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482216+0000", 2024-09-17T21:31:13.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.823e-06 2024-09-17T21:31:13.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482222+0000", 2024-09-17T21:31:13.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.507e-06 2024-09-17T21:31:13.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482227+0000", 2024-09-17T21:31:13.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6609999999999999e-06 2024-09-17T21:31:13.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529792+0000", 2024-09-17T21:31:13.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047565075999999998 2024-09-17T21:31:13.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529871+0000", 2024-09-17T21:31:13.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8855000000000003e-05 2024-09-17T21:31:13.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530299+0000", 2024-09-17T21:31:13.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000428307 2024-09-17T21:31:13.634 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.634 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.634 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.635 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50846 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 25427968~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.635 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:13.635 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.072524676, 2024-09-17T21:31:13.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048735142000000002, 2024-09-17T21:31:13.635 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.635 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.635 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.635 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.635 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.635 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.635 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50846 2024-09-17T21:31:13.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.636 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:13.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:13.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:13.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:31:13.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482239+0000", 2024-09-17T21:31:13.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.871e-06 2024-09-17T21:31:13.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482244+0000", 2024-09-17T21:31:13.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.49e-06 2024-09-17T21:31:13.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482249+0000", 2024-09-17T21:31:13.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6269999999999999e-06 2024-09-17T21:31:13.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530347+0000", 2024-09-17T21:31:13.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048097721000000003 2024-09-17T21:31:13.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530437+0000", 2024-09-17T21:31:13.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0604000000000002e-05 2024-09-17T21:31:13.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530972+0000", 2024-09-17T21:31:13.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053445000000000001 2024-09-17T21:31:13.639 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.639 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.639 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.639 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50847 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 27000832~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.639 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:13.639 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.072506779999998, 2024-09-17T21:31:13.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049308668, 2024-09-17T21:31:13.639 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.639 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.639 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.639 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.640 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.640 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.640 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50847 2024-09-17T21:31:13.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.640 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:13.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:13.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:13.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:31:13.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482257+0000", 2024-09-17T21:31:13.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.846e-06 2024-09-17T21:31:13.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482264+0000", 2024-09-17T21:31:13.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8170000000000001e-06 2024-09-17T21:31:13.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482283+0000", 2024-09-17T21:31:13.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9718e-05 2024-09-17T21:31:13.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531018+0000", 2024-09-17T21:31:13.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048734497000000002 2024-09-17T21:31:13.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531080+0000", 2024-09-17T21:31:13.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1795e-05 2024-09-17T21:31:13.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531563+0000", 2024-09-17T21:31:13.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048366499999999998 2024-09-17T21:31:13.643 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.643 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.643 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.643 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50848 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 30670848~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.643 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:31:13.643 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.072470238000001, 2024-09-17T21:31:13.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049893337000000003, 2024-09-17T21:31:13.644 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.644 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.644 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.644 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.644 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.644 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.644 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50848 2024-09-17T21:31:13.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.644 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:31:13.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:31:13.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482292+0000", 2024-09-17T21:31:13.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T21:31:13.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482294+0000", 2024-09-17T21:31:13.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3209999999999999e-06 2024-09-17T21:31:13.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482300+0000", 2024-09-17T21:31:13.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7189999999999998e-06 2024-09-17T21:31:13.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482305+0000", 2024-09-17T21:31:13.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T21:31:13.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531606+0000", 2024-09-17T21:31:13.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049301273999999999 2024-09-17T21:31:13.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531696+0000", 2024-09-17T21:31:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0161999999999995e-05 2024-09-17T21:31:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532184+0000", 2024-09-17T21:31:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048806400000000002 2024-09-17T21:31:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50849 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 32243712~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:31:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.072450797000002, 2024-09-17T21:31:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050444607000000002, 2024-09-17T21:31:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50849 2024-09-17T21:31:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:31:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:31:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482311+0000", 2024-09-17T21:31:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482316+0000", 2024-09-17T21:31:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1209999999999998e-06 2024-09-17T21:31:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482321+0000", 2024-09-17T21:31:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4779999999999999e-06 2024-09-17T21:31:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482326+0000", 2024-09-17T21:31:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7099999999999998e-06 2024-09-17T21:31:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532221+0000", 2024-09-17T21:31:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049894936000000001 2024-09-17T21:31:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532287+0000", 2024-09-17T21:31:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5914000000000006e-05 2024-09-17T21:31:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532755+0000", 2024-09-17T21:31:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000468107 2024-09-17T21:31:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:13.651 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:13.652 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:31:13.657 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:13.655+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:13.658 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:13.659 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:13.676 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:13.676 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:13.693 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:13.694 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:13.710 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:13.710 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:13.727 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:13.727 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:13.744 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:13.745 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:13.758 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:13.755+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:13.764 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:13.765 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:13.775 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:13.775 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:13.784 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:13.784 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:13.785 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:31:13.878 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:13.876+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:13.941 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:13.942 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:13.950 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:31:13.991 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:13.989+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:14.060 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:14.058+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:14.093 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:14.093 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:14.102 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:31:14.160 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:14.158+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:14.205 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:14.205 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:14.214 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:31:14.261 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:14.259+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:14.317 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:14.317 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:14.325 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:31:14.362 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:14.360+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:14.463 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:14.461+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:14.477 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:14.477 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:14.486 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:31:14.564 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:14.562+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:14.590 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:14.590 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:14.601 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:31:14.685 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:14.683+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:14.744 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:14.744 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:14.753 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:31:14.766 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:14.764+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:14.867 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:14.864+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:14.897 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:14.897 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:14.905 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:31:14.989 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:14.987+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:15.049 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:15.049 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:15.058 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:31:15.099 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:15.097+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:15.169 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:15.167+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:15.201 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:15.201 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:15.211 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:31:15.271 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:15.268+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:15.314 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:15.314 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:15.323 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:31:15.371 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:15.369+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:15.427 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:15.427 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:15.436 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:31:15.443 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-17T21:31:15.443 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:31:15.443 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T21:31:15.443 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T21:31:15.457 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:31:15.488 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:15.486+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:15.572 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:15.570+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:15.587 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:15.587 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:15.597 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:31:15.673 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:15.670+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:15.741 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:15.742 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:15.751 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:31:15.794 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:15.792+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:15.796 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:31:15.800 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:31:15.800 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:15.800 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 106, 2024-09-17T21:31:15.800 INFO:teuthology.orchestra.run.smithi129.stdout: "fsid": "f6d3a80c-8a1f-4716-b25d-e97b28890c0d", 2024-09-17T21:31:15.800 INFO:teuthology.orchestra.run.smithi129.stdout: "created": "2024-09-17T21:29:12.606637+0000", 2024-09-17T21:31:15.800 INFO:teuthology.orchestra.run.smithi129.stdout: "modified": "2024-09-17T21:31:15.444452+0000", 2024-09-17T21:31:15.800 INFO:teuthology.orchestra.run.smithi129.stdout: "last_up_change": "2024-09-17T21:30:27.721456+0000", 2024-09-17T21:31:15.800 INFO:teuthology.orchestra.run.smithi129.stdout: "last_in_change": "2024-09-17T21:31:10.410190+0000", 2024-09-17T21:31:15.800 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T21:31:15.800 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_num": 5799936, 2024-09-17T21:31:15.800 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_set": [ 2024-09-17T21:31:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: "pglog_hardlimit", 2024-09-17T21:31:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: "purged_snapdirs", 2024-09-17T21:31:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_deletes", 2024-09-17T21:31:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: "sortbitwise" 2024-09-17T21:31:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:31:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_version": 6, 2024-09-17T21:31:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T21:31:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T21:31:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T21:31:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_snapshot": "", 2024-09-17T21:31:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_max": 3, 2024-09-17T21:31:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: "max_osd": 8, 2024-09-17T21:31:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: "require_min_compat_client": "luminous", 2024-09-17T21:31:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: "min_compat_client": "jewel", 2024-09-17T21:31:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: "require_osd_release": "squid", 2024-09-17T21:31:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_crimson": false, 2024-09-17T21:31:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: "pools": [ 2024-09-17T21:31:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 1, 2024-09-17T21:31:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": ".mgr", 2024-09-17T21:31:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: "create_time": "2024-09-17T21:29:26.876763+0000", 2024-09-17T21:31:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-17T21:31:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_names": "hashpspool", 2024-09-17T21:31:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: "type": 1, 2024-09-17T21:31:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 2, 2024-09-17T21:31:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": 1, 2024-09-17T21:31:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_rule": 0, 2024-09-17T21:31:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:31:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:31:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:31:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:31:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: "is_stretch_pool": false, 2024-09-17T21:31:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: "object_hash": 2, 2024-09-17T21:31:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:31:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": 1, 2024-09-17T21:31:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num": 1, 2024-09-17T21:31:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num_target": 1, 2024-09-17T21:31:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_target": 1, 2024-09-17T21:31:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_pending": 1, 2024-09-17T21:31:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: "last_pg_merge_meta": { 2024-09-17T21:31:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: "source_pgid": "0.0", 2024-09-17T21:31:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: "ready_epoch": 0, 2024-09-17T21:31:15.804 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_started": 0, 2024-09-17T21:31:15.804 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_clean": 0, 2024-09-17T21:31:15.804 INFO:teuthology.orchestra.run.smithi129.stdout: "source_version": "0'0", 2024-09-17T21:31:15.804 INFO:teuthology.orchestra.run.smithi129.stdout: "target_version": "0'0" 2024-09-17T21:31:15.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.804 INFO:teuthology.orchestra.run.smithi129.stdout: "last_change": "19", 2024-09-17T21:31:15.804 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend": "0", 2024-09-17T21:31:15.804 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:31:15.804 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:31:15.804 INFO:teuthology.orchestra.run.smithi129.stdout: "auid": 0, 2024-09-17T21:31:15.804 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:31:15.804 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_seq": 0, 2024-09-17T21:31:15.804 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_epoch": 0, 2024-09-17T21:31:15.804 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_snaps": [], 2024-09-17T21:31:15.805 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps": "[]", 2024-09-17T21:31:15.805 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_bytes": 0, 2024-09-17T21:31:15.805 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_objects": 0, 2024-09-17T21:31:15.805 INFO:teuthology.orchestra.run.smithi129.stdout: "tiers": [], 2024-09-17T21:31:15.805 INFO:teuthology.orchestra.run.smithi129.stdout: "tier_of": -1, 2024-09-17T21:31:15.805 INFO:teuthology.orchestra.run.smithi129.stdout: "read_tier": -1, 2024-09-17T21:31:15.805 INFO:teuthology.orchestra.run.smithi129.stdout: "write_tier": -1, 2024-09-17T21:31:15.805 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_mode": "none", 2024-09-17T21:31:15.805 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_bytes": 0, 2024-09-17T21:31:15.805 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_objects": 0, 2024-09-17T21:31:15.805 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:31:15.805 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:31:15.805 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:31:15.806 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_flush_age": 0, 2024-09-17T21:31:15.806 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_evict_age": 0, 2024-09-17T21:31:15.806 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profile": "", 2024-09-17T21:31:15.806 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_params": { 2024-09-17T21:31:15.806 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "none" 2024-09-17T21:31:15.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.806 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_period": 0, 2024-09-17T21:31:15.806 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_count": 0, 2024-09-17T21:31:15.806 INFO:teuthology.orchestra.run.smithi129.stdout: "use_gmt_hitset": true, 2024-09-17T21:31:15.806 INFO:teuthology.orchestra.run.smithi129.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:31:15.807 INFO:teuthology.orchestra.run.smithi129.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:31:15.807 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:31:15.807 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:31:15.807 INFO:teuthology.orchestra.run.smithi129.stdout: "grade_table": [], 2024-09-17T21:31:15.807 INFO:teuthology.orchestra.run.smithi129.stdout: "stripe_width": 0, 2024-09-17T21:31:15.807 INFO:teuthology.orchestra.run.smithi129.stdout: "expected_num_objects": 0, 2024-09-17T21:31:15.807 INFO:teuthology.orchestra.run.smithi129.stdout: "fast_read": false, 2024-09-17T21:31:15.807 INFO:teuthology.orchestra.run.smithi129.stdout: "options": { 2024-09-17T21:31:15.807 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_max": 32, 2024-09-17T21:31:15.807 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_min": 1 2024-09-17T21:31:15.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.807 INFO:teuthology.orchestra.run.smithi129.stdout: "application_metadata": { 2024-09-17T21:31:15.807 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr": {} 2024-09-17T21:31:15.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.808 INFO:teuthology.orchestra.run.smithi129.stdout: "read_balance": { 2024-09-17T21:31:15.808 INFO:teuthology.orchestra.run.smithi129.stdout: "score_type": "Fair distribution", 2024-09-17T21:31:15.808 INFO:teuthology.orchestra.run.smithi129.stdout: "score_acting": 8, 2024-09-17T21:31:15.808 INFO:teuthology.orchestra.run.smithi129.stdout: "score_stable": 8, 2024-09-17T21:31:15.808 INFO:teuthology.orchestra.run.smithi129.stdout: "optimal_score": 0.25, 2024-09-17T21:31:15.808 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_acting": 2, 2024-09-17T21:31:15.808 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_stable": 2, 2024-09-17T21:31:15.808 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity_weighted": 0.99871444702148438, 2024-09-17T21:31:15.808 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity": 1, 2024-09-17T21:31:15.808 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:31:15.808 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.809 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 2, 2024-09-17T21:31:15.809 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": "rbd", 2024-09-17T21:31:15.809 INFO:teuthology.orchestra.run.smithi129.stdout: "create_time": "2024-09-17T21:29:31.078466+0000", 2024-09-17T21:31:15.809 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 8193, 2024-09-17T21:31:15.809 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:31:15.809 INFO:teuthology.orchestra.run.smithi129.stdout: "type": 1, 2024-09-17T21:31:15.809 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 2, 2024-09-17T21:31:15.809 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": 1, 2024-09-17T21:31:15.809 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_rule": 0, 2024-09-17T21:31:15.809 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:31:15.809 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:31:15.809 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:31:15.809 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:31:15.809 INFO:teuthology.orchestra.run.smithi129.stdout: "is_stretch_pool": false, 2024-09-17T21:31:15.810 INFO:teuthology.orchestra.run.smithi129.stdout: "object_hash": 2, 2024-09-17T21:31:15.810 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:31:15.810 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": 8, 2024-09-17T21:31:15.810 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num": 8, 2024-09-17T21:31:15.810 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num_target": 8, 2024-09-17T21:31:15.810 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_target": 8, 2024-09-17T21:31:15.810 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_pending": 8, 2024-09-17T21:31:15.810 INFO:teuthology.orchestra.run.smithi129.stdout: "last_pg_merge_meta": { 2024-09-17T21:31:15.810 INFO:teuthology.orchestra.run.smithi129.stdout: "source_pgid": "0.0", 2024-09-17T21:31:15.810 INFO:teuthology.orchestra.run.smithi129.stdout: "ready_epoch": 0, 2024-09-17T21:31:15.810 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_started": 0, 2024-09-17T21:31:15.810 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_clean": 0, 2024-09-17T21:31:15.810 INFO:teuthology.orchestra.run.smithi129.stdout: "source_version": "0'0", 2024-09-17T21:31:15.811 INFO:teuthology.orchestra.run.smithi129.stdout: "target_version": "0'0" 2024-09-17T21:31:15.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.811 INFO:teuthology.orchestra.run.smithi129.stdout: "last_change": "23", 2024-09-17T21:31:15.811 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend": "0", 2024-09-17T21:31:15.811 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:31:15.811 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:31:15.811 INFO:teuthology.orchestra.run.smithi129.stdout: "auid": 0, 2024-09-17T21:31:15.811 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:31:15.811 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_seq": 2, 2024-09-17T21:31:15.811 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_epoch": 23, 2024-09-17T21:31:15.811 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_snaps": [], 2024-09-17T21:31:15.811 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps": "[]", 2024-09-17T21:31:15.811 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_bytes": 0, 2024-09-17T21:31:15.811 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_objects": 0, 2024-09-17T21:31:15.812 INFO:teuthology.orchestra.run.smithi129.stdout: "tiers": [], 2024-09-17T21:31:15.812 INFO:teuthology.orchestra.run.smithi129.stdout: "tier_of": -1, 2024-09-17T21:31:15.812 INFO:teuthology.orchestra.run.smithi129.stdout: "read_tier": -1, 2024-09-17T21:31:15.812 INFO:teuthology.orchestra.run.smithi129.stdout: "write_tier": -1, 2024-09-17T21:31:15.812 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_mode": "none", 2024-09-17T21:31:15.812 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_bytes": 0, 2024-09-17T21:31:15.812 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_objects": 0, 2024-09-17T21:31:15.812 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:31:15.812 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:31:15.812 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:31:15.812 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_flush_age": 0, 2024-09-17T21:31:15.812 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_evict_age": 0, 2024-09-17T21:31:15.813 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profile": "", 2024-09-17T21:31:15.813 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_params": { 2024-09-17T21:31:15.813 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "none" 2024-09-17T21:31:15.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.813 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_period": 0, 2024-09-17T21:31:15.813 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_count": 0, 2024-09-17T21:31:15.813 INFO:teuthology.orchestra.run.smithi129.stdout: "use_gmt_hitset": true, 2024-09-17T21:31:15.813 INFO:teuthology.orchestra.run.smithi129.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:31:15.813 INFO:teuthology.orchestra.run.smithi129.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:31:15.813 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:31:15.813 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:31:15.813 INFO:teuthology.orchestra.run.smithi129.stdout: "grade_table": [], 2024-09-17T21:31:15.813 INFO:teuthology.orchestra.run.smithi129.stdout: "stripe_width": 0, 2024-09-17T21:31:15.814 INFO:teuthology.orchestra.run.smithi129.stdout: "expected_num_objects": 0, 2024-09-17T21:31:15.814 INFO:teuthology.orchestra.run.smithi129.stdout: "fast_read": false, 2024-09-17T21:31:15.814 INFO:teuthology.orchestra.run.smithi129.stdout: "options": {}, 2024-09-17T21:31:15.814 INFO:teuthology.orchestra.run.smithi129.stdout: "application_metadata": { 2024-09-17T21:31:15.814 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd": {} 2024-09-17T21:31:15.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.814 INFO:teuthology.orchestra.run.smithi129.stdout: "read_balance": { 2024-09-17T21:31:15.814 INFO:teuthology.orchestra.run.smithi129.stdout: "score_type": "Fair distribution", 2024-09-17T21:31:15.814 INFO:teuthology.orchestra.run.smithi129.stdout: "score_acting": 1.9900000095367432, 2024-09-17T21:31:15.814 INFO:teuthology.orchestra.run.smithi129.stdout: "score_stable": 1.9900000095367432, 2024-09-17T21:31:15.814 INFO:teuthology.orchestra.run.smithi129.stdout: "optimal_score": 0.87999999523162842, 2024-09-17T21:31:15.814 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_acting": 1.75, 2024-09-17T21:31:15.814 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_stable": 1.75, 2024-09-17T21:31:15.814 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity_weighted": 0.99871444702148438, 2024-09-17T21:31:15.815 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity": 1, 2024-09-17T21:31:15.815 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:31:15.815 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.815 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 3, 2024-09-17T21:31:15.815 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": "pool_client.0", 2024-09-17T21:31:15.815 INFO:teuthology.orchestra.run.smithi129.stdout: "create_time": "2024-09-17T21:29:39.892418+0000", 2024-09-17T21:31:15.815 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 8193, 2024-09-17T21:31:15.815 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:31:15.815 INFO:teuthology.orchestra.run.smithi129.stdout: "type": 1, 2024-09-17T21:31:15.815 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 2, 2024-09-17T21:31:15.815 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": 1, 2024-09-17T21:31:15.815 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_rule": 0, 2024-09-17T21:31:15.815 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:31:15.816 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:31:15.816 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:31:15.816 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:31:15.816 INFO:teuthology.orchestra.run.smithi129.stdout: "is_stretch_pool": false, 2024-09-17T21:31:15.816 INFO:teuthology.orchestra.run.smithi129.stdout: "object_hash": 2, 2024-09-17T21:31:15.816 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:31:15.816 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": 32, 2024-09-17T21:31:15.816 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num": 32, 2024-09-17T21:31:15.816 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num_target": 32, 2024-09-17T21:31:15.816 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_target": 32, 2024-09-17T21:31:15.816 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_pending": 32, 2024-09-17T21:31:15.816 INFO:teuthology.orchestra.run.smithi129.stdout: "last_pg_merge_meta": { 2024-09-17T21:31:15.816 INFO:teuthology.orchestra.run.smithi129.stdout: "source_pgid": "0.0", 2024-09-17T21:31:15.816 INFO:teuthology.orchestra.run.smithi129.stdout: "ready_epoch": 0, 2024-09-17T21:31:15.817 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_started": 0, 2024-09-17T21:31:15.817 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_clean": 0, 2024-09-17T21:31:15.817 INFO:teuthology.orchestra.run.smithi129.stdout: "source_version": "0'0", 2024-09-17T21:31:15.817 INFO:teuthology.orchestra.run.smithi129.stdout: "target_version": "0'0" 2024-09-17T21:31:15.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.817 INFO:teuthology.orchestra.run.smithi129.stdout: "last_change": "104", 2024-09-17T21:31:15.817 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend": "0", 2024-09-17T21:31:15.817 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:31:15.817 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:31:15.817 INFO:teuthology.orchestra.run.smithi129.stdout: "auid": 0, 2024-09-17T21:31:15.817 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:31:15.817 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_seq": 47, 2024-09-17T21:31:15.817 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_epoch": 104, 2024-09-17T21:31:15.818 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_snaps": [], 2024-09-17T21:31:15.818 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps": "[]", 2024-09-17T21:31:15.818 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_bytes": 0, 2024-09-17T21:31:15.818 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_objects": 0, 2024-09-17T21:31:15.818 INFO:teuthology.orchestra.run.smithi129.stdout: "tiers": [], 2024-09-17T21:31:15.818 INFO:teuthology.orchestra.run.smithi129.stdout: "tier_of": -1, 2024-09-17T21:31:15.818 INFO:teuthology.orchestra.run.smithi129.stdout: "read_tier": -1, 2024-09-17T21:31:15.818 INFO:teuthology.orchestra.run.smithi129.stdout: "write_tier": -1, 2024-09-17T21:31:15.818 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_mode": "none", 2024-09-17T21:31:15.818 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_bytes": 0, 2024-09-17T21:31:15.818 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_objects": 0, 2024-09-17T21:31:15.818 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:31:15.818 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:31:15.818 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:31:15.818 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_flush_age": 0, 2024-09-17T21:31:15.819 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_evict_age": 0, 2024-09-17T21:31:15.819 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profile": "", 2024-09-17T21:31:15.819 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_params": { 2024-09-17T21:31:15.819 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "none" 2024-09-17T21:31:15.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.819 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_period": 0, 2024-09-17T21:31:15.819 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_count": 0, 2024-09-17T21:31:15.819 INFO:teuthology.orchestra.run.smithi129.stdout: "use_gmt_hitset": true, 2024-09-17T21:31:15.819 INFO:teuthology.orchestra.run.smithi129.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:31:15.819 INFO:teuthology.orchestra.run.smithi129.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:31:15.819 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:31:15.819 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:31:15.819 INFO:teuthology.orchestra.run.smithi129.stdout: "grade_table": [], 2024-09-17T21:31:15.819 INFO:teuthology.orchestra.run.smithi129.stdout: "stripe_width": 0, 2024-09-17T21:31:15.820 INFO:teuthology.orchestra.run.smithi129.stdout: "expected_num_objects": 0, 2024-09-17T21:31:15.820 INFO:teuthology.orchestra.run.smithi129.stdout: "fast_read": false, 2024-09-17T21:31:15.820 INFO:teuthology.orchestra.run.smithi129.stdout: "options": {}, 2024-09-17T21:31:15.820 INFO:teuthology.orchestra.run.smithi129.stdout: "application_metadata": { 2024-09-17T21:31:15.820 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd": {} 2024-09-17T21:31:15.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.820 INFO:teuthology.orchestra.run.smithi129.stdout: "read_balance": { 2024-09-17T21:31:15.820 INFO:teuthology.orchestra.run.smithi129.stdout: "score_type": "Fair distribution", 2024-09-17T21:31:15.820 INFO:teuthology.orchestra.run.smithi129.stdout: "score_acting": 1.5, 2024-09-17T21:31:15.820 INFO:teuthology.orchestra.run.smithi129.stdout: "score_stable": 1.5, 2024-09-17T21:31:15.820 INFO:teuthology.orchestra.run.smithi129.stdout: "optimal_score": 1, 2024-09-17T21:31:15.820 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_acting": 1.5, 2024-09-17T21:31:15.820 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_stable": 1.5, 2024-09-17T21:31:15.820 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity_weighted": 0.99871444702148438, 2024-09-17T21:31:15.821 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity": 1, 2024-09-17T21:31:15.821 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:31:15.821 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.821 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.821 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:31:15.821 INFO:teuthology.orchestra.run.smithi129.stdout: "osds": [ 2024-09-17T21:31:15.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.821 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 0, 2024-09-17T21:31:15.821 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "ad067b51-9402-4f69-a0f0-08fbf068dcfc", 2024-09-17T21:31:15.821 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:31:15.821 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:31:15.821 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:31:15.821 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:31:15.821 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:31:15.821 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:31:15.824 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 13, 2024-09-17T21:31:15.824 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 105, 2024-09-17T21:31:15.824 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:31:15.824 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:31:15.824 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:31:15.824 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.824 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.824 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6800", 2024-09-17T21:31:15.824 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:31:15.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.824 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.825 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6801", 2024-09-17T21:31:15.825 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:31:15.825 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.825 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.825 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:31:15.825 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.825 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.825 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6802", 2024-09-17T21:31:15.825 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:31:15.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.825 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.826 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6803", 2024-09-17T21:31:15.826 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:31:15.826 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.826 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.826 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:31:15.826 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.826 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.826 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6806", 2024-09-17T21:31:15.826 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:31:15.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.826 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.826 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6807", 2024-09-17T21:31:15.827 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:31:15.827 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.827 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.827 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:31:15.827 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.827 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.827 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6804", 2024-09-17T21:31:15.827 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:31:15.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.827 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.828 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6805", 2024-09-17T21:31:15.828 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:31:15.828 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.828 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.828 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6801/3635367421", 2024-09-17T21:31:15.828 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6803/3635367421", 2024-09-17T21:31:15.828 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6807/3635367421", 2024-09-17T21:31:15.828 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6805/3635367421", 2024-09-17T21:31:15.828 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:31:15.828 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:31:15.828 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:31:15.828 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.829 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 1, 2024-09-17T21:31:15.829 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "30385fc0-ef5f-45f5-a3ec-37e890d4826e", 2024-09-17T21:31:15.829 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:31:15.829 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:31:15.829 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:31:15.829 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:31:15.829 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 15, 2024-09-17T21:31:15.829 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 47, 2024-09-17T21:31:15.829 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 63, 2024-09-17T21:31:15.829 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 104, 2024-09-17T21:31:15.829 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 48, 2024-09-17T21:31:15.829 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:31:15.829 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:31:15.830 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.830 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.830 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6818", 2024-09-17T21:31:15.830 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:31:15.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.830 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.830 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6819", 2024-09-17T21:31:15.830 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:31:15.830 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.830 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.830 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:31:15.831 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.831 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.831 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6820", 2024-09-17T21:31:15.831 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:31:15.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.831 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.831 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6821", 2024-09-17T21:31:15.831 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:31:15.831 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.831 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.831 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:31:15.831 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.832 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.832 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6824", 2024-09-17T21:31:15.832 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:31:15.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.832 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.832 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6825", 2024-09-17T21:31:15.832 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:31:15.832 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.832 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.832 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:31:15.832 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.833 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.833 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6822", 2024-09-17T21:31:15.833 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:31:15.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.833 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.833 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6823", 2024-09-17T21:31:15.833 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:31:15.833 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.833 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.833 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6819/842542464", 2024-09-17T21:31:15.833 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6821/842542464", 2024-09-17T21:31:15.834 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6825/842542464", 2024-09-17T21:31:15.834 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6823/842542464", 2024-09-17T21:31:15.834 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:31:15.834 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:31:15.834 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:31:15.834 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.834 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 2, 2024-09-17T21:31:15.834 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "76be9b2b-b63f-402a-9ab5-2564bd671ac3", 2024-09-17T21:31:15.834 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:31:15.834 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:31:15.834 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:31:15.834 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:31:15.835 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:31:15.835 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:31:15.835 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:31:15.835 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 103, 2024-09-17T21:31:15.835 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:31:15.835 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:31:15.835 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:31:15.835 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.835 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.835 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6808", 2024-09-17T21:31:15.835 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:31:15.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.836 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.836 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6809", 2024-09-17T21:31:15.836 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:31:15.836 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.836 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.836 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:31:15.836 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.836 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.836 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6810", 2024-09-17T21:31:15.837 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:31:15.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.837 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.837 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6811", 2024-09-17T21:31:15.837 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:31:15.837 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.837 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.837 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:31:15.837 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.837 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.837 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6814", 2024-09-17T21:31:15.838 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:31:15.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.838 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.838 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6815", 2024-09-17T21:31:15.838 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:31:15.838 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.838 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.838 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:31:15.838 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.838 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.838 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6812", 2024-09-17T21:31:15.839 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:31:15.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.839 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.839 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6813", 2024-09-17T21:31:15.839 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:31:15.839 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.839 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.839 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6809/1790706405", 2024-09-17T21:31:15.839 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6811/1790706405", 2024-09-17T21:31:15.839 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6815/1790706405", 2024-09-17T21:31:15.839 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6813/1790706405", 2024-09-17T21:31:15.839 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:31:15.840 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:31:15.840 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:31:15.840 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.840 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 3, 2024-09-17T21:31:15.840 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "d6c34e3b-1ac6-415c-9c56-38ae77a5c9b0", 2024-09-17T21:31:15.840 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:31:15.840 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:31:15.840 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:31:15.840 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 0.989715576171875, 2024-09-17T21:31:15.840 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:31:15.840 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:31:15.840 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:31:15.841 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 104, 2024-09-17T21:31:15.841 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:31:15.841 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:31:15.841 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:31:15.841 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.841 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.841 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6826", 2024-09-17T21:31:15.841 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:31:15.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.841 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.841 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6827", 2024-09-17T21:31:15.841 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:31:15.842 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.842 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.842 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:31:15.842 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.842 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.842 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6828", 2024-09-17T21:31:15.842 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:31:15.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.842 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.842 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6829", 2024-09-17T21:31:15.843 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:31:15.843 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.843 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.843 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:31:15.843 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.843 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.843 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6832", 2024-09-17T21:31:15.843 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:31:15.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.843 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.843 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6833", 2024-09-17T21:31:15.844 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:31:15.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.844 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.844 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:31:15.844 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.844 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.844 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6830", 2024-09-17T21:31:15.844 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:31:15.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.844 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.845 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6831", 2024-09-17T21:31:15.845 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:31:15.845 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.845 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.845 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6827/393361453", 2024-09-17T21:31:15.845 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6829/393361453", 2024-09-17T21:31:15.845 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6833/393361453", 2024-09-17T21:31:15.845 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6831/393361453", 2024-09-17T21:31:15.845 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:31:15.845 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:31:15.845 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:31:15.846 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.846 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 4, 2024-09-17T21:31:15.846 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "7d722e8c-c497-4f0c-9f61-d9826f8c93d8", 2024-09-17T21:31:15.846 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:31:15.846 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:31:15.846 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:31:15.846 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:31:15.846 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:31:15.846 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:31:15.846 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:31:15.846 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 103, 2024-09-17T21:31:15.846 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:31:15.847 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:31:15.847 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:31:15.847 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.847 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.847 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6808", 2024-09-17T21:31:15.847 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:31:15.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.847 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.847 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6809", 2024-09-17T21:31:15.847 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:31:15.847 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.847 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.848 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:31:15.848 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.848 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.848 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6810", 2024-09-17T21:31:15.848 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:31:15.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.848 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.848 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6811", 2024-09-17T21:31:15.848 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:31:15.848 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.848 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.849 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:31:15.849 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.849 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.849 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6814", 2024-09-17T21:31:15.849 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:31:15.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.849 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.849 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6815", 2024-09-17T21:31:15.849 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:31:15.849 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.849 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.850 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:31:15.850 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.850 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.850 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6812", 2024-09-17T21:31:15.850 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:31:15.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.850 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.850 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6813", 2024-09-17T21:31:15.850 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:31:15.850 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.850 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.851 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6809/2337546183", 2024-09-17T21:31:15.851 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6811/2337546183", 2024-09-17T21:31:15.851 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6815/2337546183", 2024-09-17T21:31:15.851 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6813/2337546183", 2024-09-17T21:31:15.851 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:31:15.851 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:31:15.851 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:31:15.851 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.851 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 5, 2024-09-17T21:31:15.851 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "954f801b-3ac9-410a-a375-eb1c533744de", 2024-09-17T21:31:15.851 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:31:15.851 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:31:15.852 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:31:15.852 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:31:15.852 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:31:15.852 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:31:15.852 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:31:15.852 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 105, 2024-09-17T21:31:15.852 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:31:15.852 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:31:15.852 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:31:15.852 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.852 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.852 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6816", 2024-09-17T21:31:15.852 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:31:15.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.853 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.853 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6817", 2024-09-17T21:31:15.853 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:31:15.853 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.853 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.853 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:31:15.853 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.853 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.853 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6818", 2024-09-17T21:31:15.853 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:31:15.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.854 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.854 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6819", 2024-09-17T21:31:15.854 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:31:15.854 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.854 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.854 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:31:15.854 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.854 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.854 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6822", 2024-09-17T21:31:15.854 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:31:15.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.855 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.855 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6823", 2024-09-17T21:31:15.855 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:31:15.855 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.855 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.855 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:31:15.855 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.855 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.855 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6820", 2024-09-17T21:31:15.855 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:31:15.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.856 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.856 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6821", 2024-09-17T21:31:15.856 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:31:15.856 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.856 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.856 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6817/1510320351", 2024-09-17T21:31:15.856 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6819/1510320351", 2024-09-17T21:31:15.856 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6823/1510320351", 2024-09-17T21:31:15.856 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6821/1510320351", 2024-09-17T21:31:15.856 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:31:15.856 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:31:15.856 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:31:15.856 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.857 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 6, 2024-09-17T21:31:15.857 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "21837515-65a2-4f21-96d6-67cfc4589358", 2024-09-17T21:31:15.857 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:31:15.857 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:31:15.857 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:31:15.857 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:31:15.857 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:31:15.857 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:31:15.857 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:31:15.857 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 48, 2024-09-17T21:31:15.857 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:31:15.857 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:31:15.858 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:31:15.858 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.858 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.858 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6824", 2024-09-17T21:31:15.858 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:31:15.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.858 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.858 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6825", 2024-09-17T21:31:15.858 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:31:15.858 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.858 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.859 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:31:15.859 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.859 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.859 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6826", 2024-09-17T21:31:15.859 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:31:15.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.859 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.859 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6827", 2024-09-17T21:31:15.859 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:31:15.859 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.859 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.860 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:31:15.860 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.860 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.860 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6830", 2024-09-17T21:31:15.860 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:31:15.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.860 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.860 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6831", 2024-09-17T21:31:15.860 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:31:15.860 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.860 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.861 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:31:15.861 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.861 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.861 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6828", 2024-09-17T21:31:15.861 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:31:15.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.861 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.861 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6829", 2024-09-17T21:31:15.861 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:31:15.861 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.861 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.862 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6825/471043930", 2024-09-17T21:31:15.862 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6827/471043930", 2024-09-17T21:31:15.862 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6831/471043930", 2024-09-17T21:31:15.862 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6829/471043930", 2024-09-17T21:31:15.862 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:31:15.862 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:31:15.862 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:31:15.862 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.862 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 7, 2024-09-17T21:31:15.862 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "67af7eea-8803-425d-ae50-30a6ab91a711", 2024-09-17T21:31:15.862 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:31:15.862 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:31:15.863 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:31:15.863 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:31:15.863 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:31:15.863 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:31:15.863 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:31:15.863 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 104, 2024-09-17T21:31:15.863 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:31:15.863 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:31:15.863 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:31:15.863 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.863 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.863 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6800", 2024-09-17T21:31:15.863 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:31:15.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.864 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.864 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6801", 2024-09-17T21:31:15.864 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:31:15.864 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.864 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.864 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:31:15.864 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.864 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.864 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6802", 2024-09-17T21:31:15.864 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:31:15.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.865 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.865 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6803", 2024-09-17T21:31:15.865 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:31:15.865 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.865 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.865 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:31:15.865 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.865 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.865 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6806", 2024-09-17T21:31:15.866 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:31:15.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.866 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.866 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6807", 2024-09-17T21:31:15.866 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:31:15.866 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.866 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.866 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:31:15.866 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:15.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.866 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:15.866 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6804", 2024-09-17T21:31:15.866 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:31:15.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.867 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:15.867 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6805", 2024-09-17T21:31:15.867 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:31:15.867 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.867 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.867 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6801/1770619177", 2024-09-17T21:31:15.867 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6803/1770619177", 2024-09-17T21:31:15.867 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6807/1770619177", 2024-09-17T21:31:15.867 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6805/1770619177", 2024-09-17T21:31:15.867 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:31:15.867 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:31:15.867 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:31:15.868 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.868 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.868 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:31:15.868 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_xinfo": [ 2024-09-17T21:31:15.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.868 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 0, 2024-09-17T21:31:15.868 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:31:15.868 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:31:15.868 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:31:15.868 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:31:15.868 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:31:15.868 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:21.445879+0000", 2024-09-17T21:31:15.868 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:31:15.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.869 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 1, 2024-09-17T21:31:15.869 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:31:15.869 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:31:15.869 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:31:15.869 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:31:15.869 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:31:15.869 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.240967+0000", 2024-09-17T21:31:15.869 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 48 2024-09-17T21:31:15.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.869 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 2, 2024-09-17T21:31:15.869 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:31:15.869 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:31:15.870 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:31:15.870 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:31:15.870 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:31:15.870 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.039097+0000", 2024-09-17T21:31:15.870 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:31:15.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.870 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 3, 2024-09-17T21:31:15.870 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:31:15.870 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:31:15.870 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:31:15.870 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:31:15.870 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:31:15.870 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.372014+0000", 2024-09-17T21:31:15.871 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:31:15.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.871 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 4, 2024-09-17T21:31:15.871 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:31:15.871 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:31:15.871 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:31:15.871 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:31:15.871 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:31:15.871 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.049543+0000", 2024-09-17T21:31:15.871 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:31:15.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.871 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 5, 2024-09-17T21:31:15.871 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:31:15.872 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:31:15.872 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:31:15.872 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:31:15.872 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:31:15.872 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.009384+0000", 2024-09-17T21:31:15.872 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:31:15.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.872 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 6, 2024-09-17T21:31:15.872 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:31:15.872 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:31:15.873 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:31:15.873 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:31:15.873 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:31:15.873 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.048598+0000", 2024-09-17T21:31:15.873 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:31:15.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.873 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 7, 2024-09-17T21:31:15.873 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:31:15.873 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:31:15.873 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:31:15.873 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:31:15.873 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:31:15.874 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.046345+0000", 2024-09-17T21:31:15.874 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:31:15.874 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.874 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:31:15.874 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_upmap": [], 2024-09-17T21:31:15.874 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_upmap_items": [], 2024-09-17T21:31:15.874 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_upmap_primaries": [], 2024-09-17T21:31:15.874 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_temp": [], 2024-09-17T21:31:15.874 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_temp": [], 2024-09-17T21:31:15.874 INFO:teuthology.orchestra.run.smithi129.stdout: "blocklist": {}, 2024-09-17T21:31:15.874 INFO:teuthology.orchestra.run.smithi129.stdout: "range_blocklist": {}, 2024-09-17T21:31:15.874 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profiles": { 2024-09-17T21:31:15.874 INFO:teuthology.orchestra.run.smithi129.stdout: "default": { 2024-09-17T21:31:15.874 INFO:teuthology.orchestra.run.smithi129.stdout: "crush-failure-domain": "osd", 2024-09-17T21:31:15.875 INFO:teuthology.orchestra.run.smithi129.stdout: "k": "2", 2024-09-17T21:31:15.875 INFO:teuthology.orchestra.run.smithi129.stdout: "m": "1", 2024-09-17T21:31:15.875 INFO:teuthology.orchestra.run.smithi129.stdout: "plugin": "jerasure", 2024-09-17T21:31:15.875 INFO:teuthology.orchestra.run.smithi129.stdout: "technique": "reed_sol_van" 2024-09-17T21:31:15.875 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:15.875 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps_queue": [ 2024-09-17T21:31:15.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.875 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 3, 2024-09-17T21:31:15.875 INFO:teuthology.orchestra.run.smithi129.stdout: "snaps": [ 2024-09-17T21:31:15.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:15.875 INFO:teuthology.orchestra.run.smithi129.stdout: "begin": 33, 2024-09-17T21:31:15.875 INFO:teuthology.orchestra.run.smithi129.stdout: "length": 7 2024-09-17T21:31:15.875 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.876 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:15.876 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.876 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:31:15.876 INFO:teuthology.orchestra.run.smithi129.stdout: "new_removed_snaps": [], 2024-09-17T21:31:15.876 INFO:teuthology.orchestra.run.smithi129.stdout: "new_purged_snaps": [], 2024-09-17T21:31:15.876 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_node_flags": {}, 2024-09-17T21:31:15.876 INFO:teuthology.orchestra.run.smithi129.stdout: "device_class_flags": {}, 2024-09-17T21:31:15.876 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_mode": { 2024-09-17T21:31:15.876 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_mode_enabled": false, 2024-09-17T21:31:15.876 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_bucket_count": 0, 2024-09-17T21:31:15.876 INFO:teuthology.orchestra.run.smithi129.stdout: "degraded_stretch_mode": 0, 2024-09-17T21:31:15.876 INFO:teuthology.orchestra.run.smithi129.stdout: "recovering_stretch_mode": 0, 2024-09-17T21:31:15.876 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_mode_bucket": 0 2024-09-17T21:31:15.876 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:15.876 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:15.878 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:15.878 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:15.879 INFO:tasks.thrashosds.thrasher:j is {'epoch': 106, 'fsid': 'f6d3a80c-8a1f-4716-b25d-e97b28890c0d', 'created': '2024-09-17T21:29:12.606637+0000', 'modified': '2024-09-17T21:31:15.444452+0000', 'last_up_change': '2024-09-17T21:30:27.721456+0000', 'last_in_change': '2024-09-17T21:31:10.410190+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T21:29:26.876763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9987144470214844, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T21:29:31.078466+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9987144470214844, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T21:29:39.892418+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 47, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9987144470214844, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'ad067b51-9402-4f69-a0f0-08fbf068dcfc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 105, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6800', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6801', 'nonce': 3635367421}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6802', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6803', 'nonce': 3635367421}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6806', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6807', 'nonce': 3635367421}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6804', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6805', 'nonce': 3635367421}]}, 'public_addr': '172.21.15.129:6801/3635367421', 'cluster_addr': '172.21.15.129:6803/3635367421', 'heartbeat_back_addr': '172.21.15.129:6807/3635367421', 'heartbeat_front_addr': '172.21.15.129:6805/3635367421', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '30385fc0-ef5f-45f5-a3ec-37e890d4826e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 47, 'up_from': 63, 'up_thru': 104, 'down_at': 48, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6818', 'nonce': 842542464}, {'type': 'v1', 'addr': '172.21.15.129:6819', 'nonce': 842542464}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6820', 'nonce': 842542464}, {'type': 'v1', 'addr': '172.21.15.129:6821', 'nonce': 842542464}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6824', 'nonce': 842542464}, {'type': 'v1', 'addr': '172.21.15.129:6825', 'nonce': 842542464}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6822', 'nonce': 842542464}, {'type': 'v1', 'addr': '172.21.15.129:6823', 'nonce': 842542464}]}, 'public_addr': '172.21.15.129:6819/842542464', 'cluster_addr': '172.21.15.129:6821/842542464', 'heartbeat_back_addr': '172.21.15.129:6825/842542464', 'heartbeat_front_addr': '172.21.15.129:6823/842542464', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '76be9b2b-b63f-402a-9ab5-2564bd671ac3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6808', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6809', 'nonce': 1790706405}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6810', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6811', 'nonce': 1790706405}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6814', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6815', 'nonce': 1790706405}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6812', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6813', 'nonce': 1790706405}]}, 'public_addr': '172.21.15.129:6809/1790706405', 'cluster_addr': '172.21.15.129:6811/1790706405', 'heartbeat_back_addr': '172.21.15.129:6815/1790706405', 'heartbeat_front_addr': '172.21.15.129:6813/1790706405', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'd6c34e3b-1ac6-415c-9c56-38ae77a5c9b0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.989715576171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 104, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6826', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6827', 'nonce': 393361453}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6828', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6829', 'nonce': 393361453}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6832', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6833', 'nonce': 393361453}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6830', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6831', 'nonce': 393361453}]}, 'public_addr': '172.21.15.129:6827/393361453', 'cluster_addr': '172.21.15.129:6829/393361453', 'heartbeat_back_addr': '172.21.15.129:6833/393361453', 'heartbeat_front_addr': '172.21.15.129:6831/393361453', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '7d722e8c-c497-4f0c-9f61-d9826f8c93d8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6808', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6809', 'nonce': 2337546183}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6810', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6811', 'nonce': 2337546183}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6814', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6815', 'nonce': 2337546183}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6812', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6813', 'nonce': 2337546183}]}, 'public_addr': '172.21.15.171:6809/2337546183', 'cluster_addr': '172.21.15.171:6811/2337546183', 'heartbeat_back_addr': '172.21.15.171:6815/2337546183', 'heartbeat_front_addr': '172.21.15.171:6813/2337546183', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '954f801b-3ac9-410a-a375-eb1c533744de', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 105, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6816', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6817', 'nonce': 1510320351}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6818', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6819', 'nonce': 1510320351}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6822', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6823', 'nonce': 1510320351}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6820', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6821', 'nonce': 1510320351}]}, 'public_addr': '172.21.15.171:6817/1510320351', 'cluster_addr': '172.21.15.171:6819/1510320351', 'heartbeat_back_addr': '172.21.15.171:6823/1510320351', 'heartbeat_front_addr': '172.21.15.171:6821/1510320351', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '21837515-65a2-4f21-96d6-67cfc4589358', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6824', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6825', 'nonce': 471043930}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6826', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6827', 'nonce': 471043930}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6830', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6831', 'nonce': 471043930}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6828', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6829', 'nonce': 471043930}]}, 'public_addr': '172.21.15.171:6825/471043930', 'cluster_addr': '172.21.15.171:6827/471043930', 'heartbeat_back_addr': '172.21.15.171:6831/471043930', 'heartbeat_front_addr': '172.21.15.171:6829/471043930', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '67af7eea-8803-425d-ae50-30a6ab91a711', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 104, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6800', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6801', 'nonce': 1770619177}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6802', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6803', 'nonce': 1770619177}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6806', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6807', 'nonce': 1770619177}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6804', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6805', 'nonce': 1770619177}]}, 'public_addr': '172.21.15.171:6801/1770619177', 'cluster_addr': '172.21.15.171:6803/1770619177', 'heartbeat_back_addr': '172.21.15.171:6807/1770619177', 'heartbeat_front_addr': '172.21.15.171:6805/1770619177', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:21.445879+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.240967+0000', 'dead_epoch': 48}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.039097+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.372014+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.049543+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.009384+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.048598+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.046345+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 33, 'length': 7}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T21:31:15.879 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-17T21:31:15.880 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:31:15.880 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:15.875+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:15.918 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:15.919 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:15.935 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:16.292 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:16.293 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:16.293 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:16.293 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:16.294 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:16.294 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:16.294 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:16.295 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:16.295 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:16.295 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:16.295 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:16.296 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:16.297 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:16.298 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:16.298 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:16.298 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:31:16.301 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:16.298+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:16.398 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:16.396+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:16.403 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:31:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:31:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 108.46024186299999, 2024-09-17T21:31:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61572000699999996, 2024-09-17T21:31:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:31:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:31:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T21:31:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:31:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:31:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939745+0000", 2024-09-17T21:31:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6899999999999999e-06 2024-09-17T21:31:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939753+0000", 2024-09-17T21:31:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1950000000000003e-06 2024-09-17T21:31:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939767+0000", 2024-09-17T21:31:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3429e-05 2024-09-17T21:31:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939776+0000", 2024-09-17T21:31:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:31:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939858+0000", 2024-09-17T21:31:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2496000000000004e-05 2024-09-17T21:31:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T21:31:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939868+0000", 2024-09-17T21:31:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1910000000000006e-06 2024-09-17T21:31:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940612+0000", 2024-09-17T21:31:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074429599999999998 2024-09-17T21:31:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:31:16.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940621+0000", 2024-09-17T21:31:16.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9139999999999997e-06 2024-09-17T21:31:16.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554006+0000", 2024-09-17T21:31:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61338505099999996 2024-09-17T21:31:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554166+0000", 2024-09-17T21:31:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015982600000000001 2024-09-17T21:31:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:31:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554296+0000", 2024-09-17T21:31:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013002399999999999 2024-09-17T21:31:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554936+0000", 2024-09-17T21:31:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064009400000000001 2024-09-17T21:31:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555389+0000", 2024-09-17T21:31:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045355200000000002 2024-09-17T21:31:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555432+0000", 2024-09-17T21:31:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2477e-05 2024-09-17T21:31:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555463+0000", 2024-09-17T21:31:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1612000000000002e-05 2024-09-17T21:31:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17527 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1005568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:31:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.995014032, 2024-09-17T21:31:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046452090000000001, 2024-09-17T21:31:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17527 2024-09-17T21:31:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:31:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:31:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405496+0000", 2024-09-17T21:31:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052498400000000004 2024-09-17T21:31:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405500+0000", 2024-09-17T21:31:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8410000000000002e-06 2024-09-17T21:31:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405511+0000", 2024-09-17T21:31:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0749e-05 2024-09-17T21:31:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405517+0000", 2024-09-17T21:31:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:31:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451275+0000", 2024-09-17T21:31:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045758134999999998 2024-09-17T21:31:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451355+0000", 2024-09-17T21:31:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9453e-05 2024-09-17T21:31:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451423+0000", 2024-09-17T21:31:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8782999999999995e-05 2024-09-17T21:31:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17615 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.989773544000002, 2024-09-17T21:31:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046741309000000002, 2024-09-17T21:31:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17615 2024-09-17T21:31:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:31:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410213+0000", 2024-09-17T21:31:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:31:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410217+0000", 2024-09-17T21:31:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8199999999999998e-06 2024-09-17T21:31:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410220+0000", 2024-09-17T21:31:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6900000000000001e-06 2024-09-17T21:31:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455264+0000", 2024-09-17T21:31:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045044002999999999 2024-09-17T21:31:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455338+0000", 2024-09-17T21:31:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4182000000000002e-05 2024-09-17T21:31:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.456953+0000", 2024-09-17T21:31:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016150089999999999 2024-09-17T21:31:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17616 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 872448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.989761492, 2024-09-17T21:31:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946039000000002, 2024-09-17T21:31:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17616 2024-09-17T21:31:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T21:31:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410225+0000", 2024-09-17T21:31:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3459999999999999e-06 2024-09-17T21:31:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410228+0000", 2024-09-17T21:31:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.083e-06 2024-09-17T21:31:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410232+0000", 2024-09-17T21:31:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4419999999999998e-06 2024-09-17T21:31:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457004+0000", 2024-09-17T21:31:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046771873999999998 2024-09-17T21:31:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457084+0000", 2024-09-17T21:31:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0506000000000005e-05 2024-09-17T21:31:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457170+0000", 2024-09-17T21:31:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5494000000000005e-05 2024-09-17T21:31:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17617 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.989750090000001, 2024-09-17T21:31:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047084168000000003, 2024-09-17T21:31:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17617 2024-09-17T21:31:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-17T21:31:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410237+0000", 2024-09-17T21:31:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1960000000000001e-06 2024-09-17T21:31:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410246+0000", 2024-09-17T21:31:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7880000000000001e-06 2024-09-17T21:31:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410250+0000", 2024-09-17T21:31:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.518e-06 2024-09-17T21:31:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457209+0000", 2024-09-17T21:31:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046959283999999997 2024-09-17T21:31:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457264+0000", 2024-09-17T21:31:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673999999999999e-05 2024-09-17T21:31:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457319+0000", 2024-09-17T21:31:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5497000000000001e-05 2024-09-17T21:31:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17618 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 899072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.989730645999998, 2024-09-17T21:31:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047204208999999997, 2024-09-17T21:31:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17618 2024-09-17T21:31:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8799999999999998e-07 2024-09-17T21:31:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410256+0000", 2024-09-17T21:31:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T21:31:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410260+0000", 2024-09-17T21:31:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6500000000000002e-06 2024-09-17T21:31:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410263+0000", 2024-09-17T21:31:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7089999999999998e-06 2024-09-17T21:31:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457355+0000", 2024-09-17T21:31:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047092756999999999 2024-09-17T21:31:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457407+0000", 2024-09-17T21:31:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1808999999999998e-05 2024-09-17T21:31:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457459+0000", 2024-09-17T21:31:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1684000000000002e-05 2024-09-17T21:31:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17619 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 913408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.989719407999999, 2024-09-17T21:31:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047293443999999997, 2024-09-17T21:31:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17619 2024-09-17T21:31:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:31:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410267+0000", 2024-09-17T21:31:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:31:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410270+0000", 2024-09-17T21:31:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.01e-06 2024-09-17T21:31:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410273+0000", 2024-09-17T21:31:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6410000000000002e-06 2024-09-17T21:31:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457473+0000", 2024-09-17T21:31:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047199829999999998 2024-09-17T21:31:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457517+0000", 2024-09-17T21:31:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4058999999999999e-05 2024-09-17T21:31:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457559+0000", 2024-09-17T21:31:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2404999999999998e-05 2024-09-17T21:31:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17620 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 927744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.989708235999998, 2024-09-17T21:31:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047402409, 2024-09-17T21:31:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17620 2024-09-17T21:31:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:31:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410278+0000", 2024-09-17T21:31:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1790000000000001e-06 2024-09-17T21:31:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410281+0000", 2024-09-17T21:31:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.061e-06 2024-09-17T21:31:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410284+0000", 2024-09-17T21:31:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.65e-06 2024-09-17T21:31:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457587+0000", 2024-09-17T21:31:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047303206 2024-09-17T21:31:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457633+0000", 2024-09-17T21:31:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6063999999999998e-05 2024-09-17T21:31:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457679+0000", 2024-09-17T21:31:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6007000000000002e-05 2024-09-17T21:31:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17621 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 940032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:31:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.989697820000004, 2024-09-17T21:31:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047486550000000002, 2024-09-17T21:31:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17621 2024-09-17T21:31:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:31:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:31:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410288+0000", 2024-09-17T21:31:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:31:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410289+0000", 2024-09-17T21:31:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1179999999999999e-06 2024-09-17T21:31:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410292+0000", 2024-09-17T21:31:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:31:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410294+0000", 2024-09-17T21:31:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T21:31:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457694+0000", 2024-09-17T21:31:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047399416999999999 2024-09-17T21:31:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457734+0000", 2024-09-17T21:31:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0528999999999998e-05 2024-09-17T21:31:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457774+0000", 2024-09-17T21:31:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9529e-05 2024-09-17T21:31:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17622 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.989687638000007, 2024-09-17T21:31:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047596619, 2024-09-17T21:31:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17622 2024-09-17T21:31:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:31:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410303+0000", 2024-09-17T21:31:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2979999999999999e-06 2024-09-17T21:31:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410307+0000", 2024-09-17T21:31:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:31:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410310+0000", 2024-09-17T21:31:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.146e-06 2024-09-17T21:31:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457801+0000", 2024-09-17T21:31:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047491322000000002 2024-09-17T21:31:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457850+0000", 2024-09-17T21:31:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8585000000000003e-05 2024-09-17T21:31:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457894+0000", 2024-09-17T21:31:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4455e-05 2024-09-17T21:31:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17623 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 968704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.989671254000001, 2024-09-17T21:31:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047689377999999998, 2024-09-17T21:31:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17623 2024-09-17T21:31:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:31:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410315+0000", 2024-09-17T21:31:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0499999999999999e-06 2024-09-17T21:31:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410319+0000", 2024-09-17T21:31:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.517e-06 2024-09-17T21:31:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410321+0000", 2024-09-17T21:31:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:31:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457922+0000", 2024-09-17T21:31:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047600638000000001 2024-09-17T21:31:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457964+0000", 2024-09-17T21:31:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2126000000000003e-05 2024-09-17T21:31:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458003+0000", 2024-09-17T21:31:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9138999999999999e-05 2024-09-17T21:31:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17624 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.989660670000006, 2024-09-17T21:31:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047798292999999999, 2024-09-17T21:31:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17624 2024-09-17T21:31:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:31:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410326+0000", 2024-09-17T21:31:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0610000000000001e-06 2024-09-17T21:31:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410329+0000", 2024-09-17T21:31:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0319999999999999e-06 2024-09-17T21:31:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410331+0000", 2024-09-17T21:31:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5749999999999999e-06 2024-09-17T21:31:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458033+0000", 2024-09-17T21:31:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047701371999999999 2024-09-17T21:31:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458079+0000", 2024-09-17T21:31:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:31:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458123+0000", 2024-09-17T21:31:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850999999999997e-05 2024-09-17T21:31:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17625 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.989650510000004, 2024-09-17T21:31:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047883051000000003, 2024-09-17T21:31:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17625 2024-09-17T21:31:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:31:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410336+0000", 2024-09-17T21:31:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1310000000000001e-06 2024-09-17T21:31:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410339+0000", 2024-09-17T21:31:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000001e-06 2024-09-17T21:31:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410342+0000", 2024-09-17T21:31:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7460000000000001e-06 2024-09-17T21:31:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458137+0000", 2024-09-17T21:31:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047794931999999998 2024-09-17T21:31:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458178+0000", 2024-09-17T21:31:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1013999999999997e-05 2024-09-17T21:31:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458218+0000", 2024-09-17T21:31:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9675999999999998e-05 2024-09-17T21:31:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17626 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1009664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:31:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.989639889000003, 2024-09-17T21:31:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047991601000000002, 2024-09-17T21:31:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17626 2024-09-17T21:31:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:31:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:31:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410346+0000", 2024-09-17T21:31:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:31:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410347+0000", 2024-09-17T21:31:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.114e-06 2024-09-17T21:31:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410350+0000", 2024-09-17T21:31:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.101e-06 2024-09-17T21:31:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410353+0000", 2024-09-17T21:31:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T21:31:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458245+0000", 2024-09-17T21:31:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0478921 2024-09-17T21:31:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458292+0000", 2024-09-17T21:31:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397999999999999e-05 2024-09-17T21:31:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458337+0000", 2024-09-17T21:31:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5221e-05 2024-09-17T21:31:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17627 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:31:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.989602787999999, 2024-09-17T21:31:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048047383999999999, 2024-09-17T21:31:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17627 2024-09-17T21:31:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:31:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:31:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410383+0000", 2024-09-17T21:31:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:31:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410385+0000", 2024-09-17T21:31:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:31:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410391+0000", 2024-09-17T21:31:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:31:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410395+0000", 2024-09-17T21:31:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6539999999999998e-06 2024-09-17T21:31:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458350+0000", 2024-09-17T21:31:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047954855999999997 2024-09-17T21:31:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458392+0000", 2024-09-17T21:31:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2094000000000001e-05 2024-09-17T21:31:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458430+0000", 2024-09-17T21:31:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7474000000000001e-05 2024-09-17T21:31:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17628 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.989583226999997, 2024-09-17T21:31:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048142676000000002, 2024-09-17T21:31:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17628 2024-09-17T21:31:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:31:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410404+0000", 2024-09-17T21:31:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-06 2024-09-17T21:31:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410411+0000", 2024-09-17T21:31:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6850000000000004e-06 2024-09-17T21:31:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410415+0000", 2024-09-17T21:31:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6750000000000001e-06 2024-09-17T21:31:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458457+0000", 2024-09-17T21:31:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048042045999999998 2024-09-17T21:31:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458501+0000", 2024-09-17T21:31:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3661000000000001e-05 2024-09-17T21:31:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458545+0000", 2024-09-17T21:31:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3529000000000003e-05 2024-09-17T21:31:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17687 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:31:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.987138274000003, 2024-09-17T21:31:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050670448, 2024-09-17T21:31:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17687 2024-09-17T21:31:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:31:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:31:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412848+0000", 2024-09-17T21:31:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9100000000000003e-07 2024-09-17T21:31:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413102+0000", 2024-09-17T21:31:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025415600000000001 2024-09-17T21:31:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413111+0000", 2024-09-17T21:31:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2010000000000005e-06 2024-09-17T21:31:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413116+0000", 2024-09-17T21:31:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:31:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458559+0000", 2024-09-17T21:31:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045442914000000001 2024-09-17T21:31:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458632+0000", 2024-09-17T21:31:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3207000000000006e-05 2024-09-17T21:31:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:31:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458785+0000", 2024-09-17T21:31:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015226700000000001 2024-09-17T21:31:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.461648+0000", 2024-09-17T21:31:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028633069999999998 2024-09-17T21:31:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463416+0000", 2024-09-17T21:31:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017680230000000001 2024-09-17T21:31:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463475+0000", 2024-09-17T21:31:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9333000000000003e-05 2024-09-17T21:31:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463517+0000", 2024-09-17T21:31:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2162000000000001e-05 2024-09-17T21:31:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17688 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [write 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.986859405999994, 2024-09-17T21:31:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050625098, 2024-09-17T21:31:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17688 2024-09-17T21:31:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:31:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413128+0000", 2024-09-17T21:31:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.813e-06 2024-09-17T21:31:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413132+0000", 2024-09-17T21:31:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:31:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413135+0000", 2024-09-17T21:31:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:31:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459756+0000", 2024-09-17T21:31:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046621293000000001 2024-09-17T21:31:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459816+0000", 2024-09-17T21:31:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9156000000000001e-05 2024-09-17T21:31:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:31:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459912+0000", 2024-09-17T21:31:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6302000000000001e-05 2024-09-17T21:31:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463231+0000", 2024-09-17T21:31:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033188290000000001 2024-09-17T21:31:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463701+0000", 2024-09-17T21:31:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047068199999999999 2024-09-17T21:31:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463728+0000", 2024-09-17T21:31:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6726000000000001e-05 2024-09-17T21:31:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463751+0000", 2024-09-17T21:31:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2824e-05 2024-09-17T21:31:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17691 3.d 3:b3a055f3:::rbd_data.11207744cc2c.0000000000000083:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:31:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.983314503000003, 2024-09-17T21:31:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049309370999999998, 2024-09-17T21:31:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17691 2024-09-17T21:31:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:31:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:31:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416672+0000", 2024-09-17T21:31:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.47e-07 2024-09-17T21:31:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416915+0000", 2024-09-17T21:31:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024344500000000001 2024-09-17T21:31:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416924+0000", 2024-09-17T21:31:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7739999999999993e-06 2024-09-17T21:31:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416929+0000", 2024-09-17T21:31:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6990000000000002e-06 2024-09-17T21:31:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460414+0000", 2024-09-17T21:31:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043485466 2024-09-17T21:31:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460540+0000", 2024-09-17T21:31:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012608899999999999 2024-09-17T21:31:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:31:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460746+0000", 2024-09-17T21:31:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000205577 2024-09-17T21:31:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463800+0000", 2024-09-17T21:31:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030542030000000001 2024-09-17T21:31:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465867+0000", 2024-09-17T21:31:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020665929999999998 2024-09-17T21:31:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465914+0000", 2024-09-17T21:31:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497000000000003e-05 2024-09-17T21:31:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465980+0000", 2024-09-17T21:31:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6081000000000006e-05 2024-09-17T21:31:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:70218 3.16 3:68692d8f:::rbd_data.1120b9193b9f.000000000000001b:head [stat,delete] snapc 1d=[1d] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-17T21:31:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.594347413999998, 2024-09-17T21:31:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.951400786, 2024-09-17T21:31:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 70218 2024-09-17T21:31:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4799999999999998e-07 2024-09-17T21:31:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805642+0000", 2024-09-17T21:31:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2009999999999999e-06 2024-09-17T21:31:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805650+0000", 2024-09-17T21:31:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1000000000000004e-06 2024-09-17T21:31:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805655+0000", 2024-09-17T21:31:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:31:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805722+0000", 2024-09-17T21:31:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6878000000000006e-05 2024-09-17T21:31:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:31:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805733+0000", 2024-09-17T21:31:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0895e-05 2024-09-17T21:31:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755252+0000", 2024-09-17T21:31:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.94951843800000002 2024-09-17T21:31:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755398+0000", 2024-09-17T21:31:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014622600000000001 2024-09-17T21:31:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.757039+0000", 2024-09-17T21:31:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001640819 2024-09-17T21:31:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:16.505 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:16.507 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is set 2024-09-17T21:31:16.507 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:31:16.508 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:16.504+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:16.607 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:16.605+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:16.658 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:16.658 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:16.658 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:16.658 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:16.667 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:31:16.709 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:16.707+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:16.770 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:16.770 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:16.770 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:16.770 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:16.771 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:16.779 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:31:16.810 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:16.808+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:16.886 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:16.887 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:16.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:16.887 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:16.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.887 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97778 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 245760~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.887 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:31:16.887 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916857712999999, 2024-09-17T21:31:16.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050933728999999997, 2024-09-17T21:31:16.887 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.887 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.887 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.887 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.888 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97778 2024-09-17T21:31:16.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.888 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:31:16.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:31:16.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966899+0000", 2024-09-17T21:31:16.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:31:16.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966900+0000", 2024-09-17T21:31:16.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.384e-06 2024-09-17T21:31:16.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966905+0000", 2024-09-17T21:31:16.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6530000000000002e-06 2024-09-17T21:31:16.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966909+0000", 2024-09-17T21:31:16.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2889999999999999e-06 2024-09-17T21:31:16.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017585+0000", 2024-09-17T21:31:16.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050675691000000002 2024-09-17T21:31:16.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017642+0000", 2024-09-17T21:31:16.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7167999999999999e-05 2024-09-17T21:31:16.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017832+0000", 2024-09-17T21:31:16.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019022 2024-09-17T21:31:16.891 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.891 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.891 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.891 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97779 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.891 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:31:16.891 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916840645000001, 2024-09-17T21:31:16.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051245027999999998, 2024-09-17T21:31:16.892 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.892 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.892 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.892 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97779 2024-09-17T21:31:16.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.892 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:31:16.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:31:16.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966916+0000", 2024-09-17T21:31:16.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:31:16.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966918+0000", 2024-09-17T21:31:16.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8369999999999999e-06 2024-09-17T21:31:16.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966924+0000", 2024-09-17T21:31:16.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2260000000000002e-06 2024-09-17T21:31:16.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966928+0000", 2024-09-17T21:31:16.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3819999999999997e-06 2024-09-17T21:31:16.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017866+0000", 2024-09-17T21:31:16.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050938079999999997 2024-09-17T21:31:16.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017945+0000", 2024-09-17T21:31:16.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8275000000000005e-05 2024-09-17T21:31:16.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018161+0000", 2024-09-17T21:31:16.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021579499999999999 2024-09-17T21:31:16.895 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.895 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.896 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.896 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97780 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 344064~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.896 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:16.896 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916816366999999, 2024-09-17T21:31:16.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051535301999999998, 2024-09-17T21:31:16.896 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.896 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.896 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.896 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97780 2024-09-17T21:31:16.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.897 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:16.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:16.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:16.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:31:16.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966942+0000", 2024-09-17T21:31:16.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7439999999999999e-06 2024-09-17T21:31:16.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966946+0000", 2024-09-17T21:31:16.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5510000000000001e-06 2024-09-17T21:31:16.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966951+0000", 2024-09-17T21:31:16.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:31:16.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018191+0000", 2024-09-17T21:31:16.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051240290000000001 2024-09-17T21:31:16.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018269+0000", 2024-09-17T21:31:16.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7416999999999994e-05 2024-09-17T21:31:16.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018475+0000", 2024-09-17T21:31:16.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206486 2024-09-17T21:31:16.900 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.900 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.900 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.900 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97789 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 40960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.900 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:16.900 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916796198, 2024-09-17T21:31:16.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051862143999999999, 2024-09-17T21:31:16.901 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.901 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.901 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.901 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97789 2024-09-17T21:31:16.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.901 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:16.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:16.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:16.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5899999999999996e-07 2024-09-17T21:31:16.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966962+0000", 2024-09-17T21:31:16.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9700000000000002e-06 2024-09-17T21:31:16.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966969+0000", 2024-09-17T21:31:16.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6939999999999998e-06 2024-09-17T21:31:16.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966980+0000", 2024-09-17T21:31:16.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0679999999999999e-05 2024-09-17T21:31:16.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018513+0000", 2024-09-17T21:31:16.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051533420000000003 2024-09-17T21:31:16.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018595+0000", 2024-09-17T21:31:16.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1260999999999993e-05 2024-09-17T21:31:16.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018822+0000", 2024-09-17T21:31:16.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022756 2024-09-17T21:31:16.904 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.904 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.904 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.905 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97790 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 90112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.905 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:31:16.905 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916765699999999, 2024-09-17T21:31:16.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052173479000000002, 2024-09-17T21:31:16.905 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.905 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.905 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.905 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97790 2024-09-17T21:31:16.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.905 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:31:16.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:31:16.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966991+0000", 2024-09-17T21:31:16.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9500000000000003e-07 2024-09-17T21:31:16.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966993+0000", 2024-09-17T21:31:16.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.238e-06 2024-09-17T21:31:16.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967000+0000", 2024-09-17T21:31:16.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9500000000000004e-06 2024-09-17T21:31:16.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967005+0000", 2024-09-17T21:31:16.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:31:16.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018852+0000", 2024-09-17T21:31:16.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051847250999999997 2024-09-17T21:31:16.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018927+0000", 2024-09-17T21:31:16.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4944000000000006e-05 2024-09-17T21:31:16.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019164+0000", 2024-09-17T21:31:16.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000236803 2024-09-17T21:31:16.909 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.909 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.909 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.909 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97791 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 139264~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.909 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:31:16.909 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916743956000001, 2024-09-17T21:31:16.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052464594000000003, 2024-09-17T21:31:16.909 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.910 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.910 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.910 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97791 2024-09-17T21:31:16.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.910 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:31:16.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:31:16.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967013+0000", 2024-09-17T21:31:16.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:31:16.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967014+0000", 2024-09-17T21:31:16.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8679999999999999e-06 2024-09-17T21:31:16.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967020+0000", 2024-09-17T21:31:16.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.322e-06 2024-09-17T21:31:16.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967024+0000", 2024-09-17T21:31:16.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2660000000000003e-06 2024-09-17T21:31:16.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019193+0000", 2024-09-17T21:31:16.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052169010000000002 2024-09-17T21:31:16.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019269+0000", 2024-09-17T21:31:16.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6149999999999994e-05 2024-09-17T21:31:16.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019477+0000", 2024-09-17T21:31:16.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020753699999999999 2024-09-17T21:31:16.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.913 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.914 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97792 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 188416~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.914 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:16.914 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916726217000001, 2024-09-17T21:31:16.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052764958000000001, 2024-09-17T21:31:16.914 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.914 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.914 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.914 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97792 2024-09-17T21:31:16.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.914 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:16.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:16.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:16.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0900000000000002e-07 2024-09-17T21:31:16.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967032+0000", 2024-09-17T21:31:16.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9740000000000001e-06 2024-09-17T21:31:16.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967039+0000", 2024-09-17T21:31:16.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3679999999999998e-06 2024-09-17T21:31:16.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967043+0000", 2024-09-17T21:31:16.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3379999999999998e-06 2024-09-17T21:31:16.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019505+0000", 2024-09-17T21:31:16.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052462264000000002 2024-09-17T21:31:16.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019582+0000", 2024-09-17T21:31:16.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7146999999999998e-05 2024-09-17T21:31:16.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019795+0000", 2024-09-17T21:31:16.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000212458 2024-09-17T21:31:16.917 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.918 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.918 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.918 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97793 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 237568~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.918 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:31:16.918 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916704717000002, 2024-09-17T21:31:16.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053070436999999998, 2024-09-17T21:31:16.918 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.918 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.918 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.918 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97793 2024-09-17T21:31:16.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.919 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:31:16.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:31:16.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967052+0000", 2024-09-17T21:31:16.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7500000000000001e-07 2024-09-17T21:31:16.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967054+0000", 2024-09-17T21:31:16.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:31:16.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967059+0000", 2024-09-17T21:31:16.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5840000000000002e-06 2024-09-17T21:31:16.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967064+0000", 2024-09-17T21:31:16.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:31:16.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019826+0000", 2024-09-17T21:31:16.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052762396000000003 2024-09-17T21:31:16.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019900+0000", 2024-09-17T21:31:16.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3594e-05 2024-09-17T21:31:16.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020122+0000", 2024-09-17T21:31:16.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022185999999999999 2024-09-17T21:31:16.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.922 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.922 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97794 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 286720~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.922 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:16.922 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916684580999998, 2024-09-17T21:31:16.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053453734000000003, 2024-09-17T21:31:16.923 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.923 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.923 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.923 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97794 2024-09-17T21:31:16.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.923 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:16.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:16.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:16.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2199999999999999e-07 2024-09-17T21:31:16.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967074+0000", 2024-09-17T21:31:16.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8419999999999999e-06 2024-09-17T21:31:16.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967080+0000", 2024-09-17T21:31:16.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5660000000000003e-06 2024-09-17T21:31:16.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967120+0000", 2024-09-17T21:31:16.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9504e-05 2024-09-17T21:31:16.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020189+0000", 2024-09-17T21:31:16.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053069144999999998 2024-09-17T21:31:16.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020287+0000", 2024-09-17T21:31:16.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8259999999999998e-05 2024-09-17T21:31:16.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020525+0000", 2024-09-17T21:31:16.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023799499999999999 2024-09-17T21:31:16.926 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.926 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.926 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.926 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97795 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 335872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.927 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:16.927 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916626446999999, 2024-09-17T21:31:16.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053707207999999999, 2024-09-17T21:31:16.927 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.927 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.927 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.927 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97795 2024-09-17T21:31:16.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.927 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:16.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:16.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:16.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:31:16.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967139+0000", 2024-09-17T21:31:16.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5790000000000004e-06 2024-09-17T21:31:16.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967145+0000", 2024-09-17T21:31:16.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3550000000000001e-06 2024-09-17T21:31:16.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967155+0000", 2024-09-17T21:31:16.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8859999999999999e-06 2024-09-17T21:31:16.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020555+0000", 2024-09-17T21:31:16.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053400359000000001 2024-09-17T21:31:16.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020630+0000", 2024-09-17T21:31:16.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4894999999999994e-05 2024-09-17T21:31:16.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020837+0000", 2024-09-17T21:31:16.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020670500000000001 2024-09-17T21:31:16.931 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.931 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.931 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.931 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97796 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 385024~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.931 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:16.931 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916594187000001, 2024-09-17T21:31:16.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053990227000000002, 2024-09-17T21:31:16.931 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.931 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.931 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.932 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97796 2024-09-17T21:31:16.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.932 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:16.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:16.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:16.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:31:16.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967164+0000", 2024-09-17T21:31:16.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0370000000000001e-06 2024-09-17T21:31:16.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967171+0000", 2024-09-17T21:31:16.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2190000000000001e-06 2024-09-17T21:31:16.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967175+0000", 2024-09-17T21:31:16.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2599999999999999e-06 2024-09-17T21:31:16.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020866+0000", 2024-09-17T21:31:16.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053691539000000003 2024-09-17T21:31:16.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020942+0000", 2024-09-17T21:31:16.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5487000000000004e-05 2024-09-17T21:31:16.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021152+0000", 2024-09-17T21:31:16.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021021599999999999 2024-09-17T21:31:16.935 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.935 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.935 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.935 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97805 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 8192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.935 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:16.935 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916575507000001, 2024-09-17T21:31:16.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054331771000000001, 2024-09-17T21:31:16.935 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.935 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.935 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.935 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.935 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.936 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97805 2024-09-17T21:31:16.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.936 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:16.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:16.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:16.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:31:16.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967183+0000", 2024-09-17T21:31:16.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6840000000000001e-06 2024-09-17T21:31:16.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967189+0000", 2024-09-17T21:31:16.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0299999999999999e-06 2024-09-17T21:31:16.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967193+0000", 2024-09-17T21:31:16.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-17T21:31:16.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021182+0000", 2024-09-17T21:31:16.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053989319000000001 2024-09-17T21:31:16.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021261+0000", 2024-09-17T21:31:16.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8460000000000004e-05 2024-09-17T21:31:16.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021512+0000", 2024-09-17T21:31:16.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025158500000000002 2024-09-17T21:31:16.939 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.939 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.939 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.939 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97806 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.939 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:16.939 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916557504, 2024-09-17T21:31:16.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054654343000000001, 2024-09-17T21:31:16.939 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.939 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.940 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.940 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97806 2024-09-17T21:31:16.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.940 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:16.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:16.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:16.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:31:16.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967201+0000", 2024-09-17T21:31:16.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.691e-06 2024-09-17T21:31:16.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967206+0000", 2024-09-17T21:31:16.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7780000000000004e-06 2024-09-17T21:31:16.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967224+0000", 2024-09-17T21:31:16.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7357999999999999e-05 2024-09-17T21:31:16.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021546+0000", 2024-09-17T21:31:16.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054322237000000002 2024-09-17T21:31:16.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021618+0000", 2024-09-17T21:31:16.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2007000000000004e-05 2024-09-17T21:31:16.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021853+0000", 2024-09-17T21:31:16.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023495800000000001 2024-09-17T21:31:16.943 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.943 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.943 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.943 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97807 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 106496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.943 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:31:16.943 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916525104000002, 2024-09-17T21:31:16.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055014183000000001, 2024-09-17T21:31:16.944 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.944 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.944 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.944 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.944 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.944 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.944 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97807 2024-09-17T21:31:16.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.944 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:31:16.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:31:16.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967232+0000", 2024-09-17T21:31:16.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:31:16.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967234+0000", 2024-09-17T21:31:16.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.283e-06 2024-09-17T21:31:16.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967240+0000", 2024-09-17T21:31:16.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.144e-06 2024-09-17T21:31:16.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967252+0000", 2024-09-17T21:31:16.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2322e-05 2024-09-17T21:31:16.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021882+0000", 2024-09-17T21:31:16.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054629952000000002 2024-09-17T21:31:16.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021952+0000", 2024-09-17T21:31:16.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9919999999999995e-05 2024-09-17T21:31:16.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022245+0000", 2024-09-17T21:31:16.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00029302100000000003 2024-09-17T21:31:16.947 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.947 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.947 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.948 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97808 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 155648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.948 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:16.948 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916496505000001, 2024-09-17T21:31:16.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055447485999999997, 2024-09-17T21:31:16.948 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.948 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.948 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.948 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.948 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.948 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.948 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97808 2024-09-17T21:31:16.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.948 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:16.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:16.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:16.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4299999999999998e-07 2024-09-17T21:31:16.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967267+0000", 2024-09-17T21:31:16.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9820000000000002e-06 2024-09-17T21:31:16.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967273+0000", 2024-09-17T21:31:16.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:31:16.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967277+0000", 2024-09-17T21:31:16.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5050000000000001e-06 2024-09-17T21:31:16.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022297+0000", 2024-09-17T21:31:16.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055020131999999999 2024-09-17T21:31:16.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022379+0000", 2024-09-17T21:31:16.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1810000000000004e-05 2024-09-17T21:31:16.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022707+0000", 2024-09-17T21:31:16.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00032784300000000003 2024-09-17T21:31:16.951 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.952 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.952 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.952 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97809 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.952 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:31:16.952 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916472989999999, 2024-09-17T21:31:16.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056012109999999997, 2024-09-17T21:31:16.952 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.952 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.952 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.952 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.952 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.952 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.953 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97809 2024-09-17T21:31:16.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.953 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:31:16.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:31:16.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967284+0000", 2024-09-17T21:31:16.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:31:16.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967285+0000", 2024-09-17T21:31:16.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.826e-06 2024-09-17T21:31:16.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967291+0000", 2024-09-17T21:31:16.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:31:16.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967302+0000", 2024-09-17T21:31:16.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1114999999999999e-05 2024-09-17T21:31:16.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022901+0000", 2024-09-17T21:31:16.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055599335 2024-09-17T21:31:16.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022990+0000", 2024-09-17T21:31:16.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8671999999999994e-05 2024-09-17T21:31:16.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023295+0000", 2024-09-17T21:31:16.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030554900000000001 2024-09-17T21:31:16.956 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.956 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.956 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.956 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97810 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 253952~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.956 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:16.956 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916446562000001, 2024-09-17T21:31:16.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056355668999999997, 2024-09-17T21:31:16.956 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.956 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.956 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.956 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.957 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.957 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.957 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97810 2024-09-17T21:31:16.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.957 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:16.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:16.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:16.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0200000000000002e-07 2024-09-17T21:31:16.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967313+0000", 2024-09-17T21:31:16.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4549999999999998e-06 2024-09-17T21:31:16.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967319+0000", 2024-09-17T21:31:16.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3080000000000004e-06 2024-09-17T21:31:16.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967324+0000", 2024-09-17T21:31:16.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0159999999999999e-06 2024-09-17T21:31:16.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023342+0000", 2024-09-17T21:31:16.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056018635999999997 2024-09-17T21:31:16.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023423+0000", 2024-09-17T21:31:16.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0473e-05 2024-09-17T21:31:16.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023665+0000", 2024-09-17T21:31:16.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242279 2024-09-17T21:31:16.960 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.960 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.960 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.960 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97811 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 303104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.960 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:16.960 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916420303999999, 2024-09-17T21:31:16.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056743978, 2024-09-17T21:31:16.960 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.961 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.961 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.961 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.961 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.961 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.961 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97811 2024-09-17T21:31:16.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.961 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:16.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:16.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:16.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1300000000000001e-07 2024-09-17T21:31:16.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967338+0000", 2024-09-17T21:31:16.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9740000000000001e-06 2024-09-17T21:31:16.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967346+0000", 2024-09-17T21:31:16.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8129999999999996e-06 2024-09-17T21:31:16.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967351+0000", 2024-09-17T21:31:16.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6129999999999997e-06 2024-09-17T21:31:16.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023750+0000", 2024-09-17T21:31:16.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056399796000000002 2024-09-17T21:31:16.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023839+0000", 2024-09-17T21:31:16.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.852e-05 2024-09-17T21:31:16.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024080+0000", 2024-09-17T21:31:16.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024084900000000001 2024-09-17T21:31:16.964 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.964 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.964 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.965 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97812 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 352256~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:16.965 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:31:16.965 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.916399687999998, 2024-09-17T21:31:16.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.057024455000000002, 2024-09-17T21:31:16.965 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.965 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.965 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.965 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.965 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.965 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.965 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97812 2024-09-17T21:31:16.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.966 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:31:16.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:31:16.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967357+0000", 2024-09-17T21:31:16.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:31:16.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967359+0000", 2024-09-17T21:31:16.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:31:16.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967365+0000", 2024-09-17T21:31:16.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:31:16.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967369+0000", 2024-09-17T21:31:16.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5029999999999999e-06 2024-09-17T21:31:16.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024109+0000", 2024-09-17T21:31:16.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056740038 2024-09-17T21:31:16.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024175+0000", 2024-09-17T21:31:16.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5106999999999999e-05 2024-09-17T21:31:16.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024381+0000", 2024-09-17T21:31:16.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020629700000000001 2024-09-17T21:31:16.969 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.969 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.969 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.969 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:112033 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-09-17T21:31:16.969 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:31:11.421210+0000", 2024-09-17T21:31:16.969 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.4625459220000003, 2024-09-17T21:31:16.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0104142869999999, 2024-09-17T21:31:16.969 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:16.969 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:16.969 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:16.969 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 112033 2024-09-17T21:31:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421210+0000", 2024-09-17T21:31:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421210+0000", 2024-09-17T21:31:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421212+0000", 2024-09-17T21:31:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.376e-06 2024-09-17T21:31:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421217+0000", 2024-09-17T21:31:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.259e-06 2024-09-17T21:31:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421228+0000", 2024-09-17T21:31:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0944000000000001e-05 2024-09-17T21:31:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:16.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421238+0000", 2024-09-17T21:31:16.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:31:16.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421626+0000", 2024-09-17T21:31:16.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00038804099999999999 2024-09-17T21:31:16.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:31:16.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421637+0000", 2024-09-17T21:31:16.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1783000000000001e-05 2024-09-17T21:31:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:12.431245+0000", 2024-09-17T21:31:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.00960765 2024-09-17T21:31:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:12.431413+0000", 2024-09-17T21:31:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000167784 2024-09-17T21:31:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:12.431624+0000", 2024-09-17T21:31:16.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000211704 2024-09-17T21:31:16.974 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.974 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.974 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.974 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:16.974 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:16.974 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:16.975 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:31:16.976 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:16.973+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:17.076 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:17.074+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:17.079 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:17.079 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:17.079 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:17.079 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:17.088 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:31:17.177 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:17.175+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:17.191 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:17.191 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:17.191 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:17.191 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:17.191 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:17.200 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:31:17.284 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:17.282+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:17.344 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:17.344 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:17.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:17.344 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:17.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.344 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:37817 3.17 3:eab85a8d:::rbd_header.1120e62eea89:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:31:17.344 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:31:17.344 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 75.645993795999999, 2024-09-17T21:31:17.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86635970399999995, 2024-09-17T21:31:17.345 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.345 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.345 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.345 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.345 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.345 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.345 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37817 2024-09-17T21:31:17.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.345 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:31:17.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:31:17.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695457+0000", 2024-09-17T21:31:17.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5799999999999998e-07 2024-09-17T21:31:17.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695461+0000", 2024-09-17T21:31:17.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1110000000000001e-06 2024-09-17T21:31:17.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695471+0000", 2024-09-17T21:31:17.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0441e-05 2024-09-17T21:31:17.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695477+0000", 2024-09-17T21:31:17.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7599999999999999e-06 2024-09-17T21:31:17.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695533+0000", 2024-09-17T21:31:17.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6570999999999998e-05 2024-09-17T21:31:17.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:31:17.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695543+0000", 2024-09-17T21:31:17.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9979999999999998e-06 2024-09-17T21:31:17.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561472+0000", 2024-09-17T21:31:17.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86592841899999995 2024-09-17T21:31:17.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561610+0000", 2024-09-17T21:31:17.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013854699999999999 2024-09-17T21:31:17.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561816+0000", 2024-09-17T21:31:17.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020509900000000001 2024-09-17T21:31:17.349 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.349 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.349 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.349 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50202 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.349 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:31:17.349 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.233823005999994, 2024-09-17T21:31:17.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.076620261999999995, 2024-09-17T21:31:17.349 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.349 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.350 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:17.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.350 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50202 2024-09-17T21:31:17.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.350 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:31:17.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:31:17.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107629+0000", 2024-09-17T21:31:17.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3709999999999998e-06 2024-09-17T21:31:17.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119765+0000", 2024-09-17T21:31:17.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012135683 2024-09-17T21:31:17.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119787+0000", 2024-09-17T21:31:17.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2748999999999999e-05 2024-09-17T21:31:17.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119797+0000", 2024-09-17T21:31:17.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8169999999999994e-06 2024-09-17T21:31:17.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119872+0000", 2024-09-17T21:31:17.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270999999999994e-05 2024-09-17T21:31:17.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119987+0000", 2024-09-17T21:31:17.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011557699999999999 2024-09-17T21:31:17.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:17.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.121766+0000", 2024-09-17T21:31:17.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00177853 2024-09-17T21:31:17.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:17.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178307+0000", 2024-09-17T21:31:17.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056541559999999998 2024-09-17T21:31:17.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:17.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184097+0000", 2024-09-17T21:31:17.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057899029999999999 2024-09-17T21:31:17.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:17.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184172+0000", 2024-09-17T21:31:17.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5226000000000001e-05 2024-09-17T21:31:17.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184247+0000", 2024-09-17T21:31:17.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4574999999999997e-05 2024-09-17T21:31:17.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.355 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.355 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.355 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50203 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [truncate 20971520] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.355 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:31:17.355 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.221623101000006, 2024-09-17T21:31:17.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065009779000000004, 2024-09-17T21:31:17.355 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.355 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.355 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:17.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.355 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50203 2024-09-17T21:31:17.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.356 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:31:17.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:31:17.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119828+0000", 2024-09-17T21:31:17.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6390000000000001e-06 2024-09-17T21:31:17.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119832+0000", 2024-09-17T21:31:17.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7699999999999999e-06 2024-09-17T21:31:17.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119845+0000", 2024-09-17T21:31:17.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3093e-05 2024-09-17T21:31:17.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119852+0000", 2024-09-17T21:31:17.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6950000000000003e-06 2024-09-17T21:31:17.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.160945+0000", 2024-09-17T21:31:17.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041093076999999999 2024-09-17T21:31:17.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161071+0000", 2024-09-17T21:31:17.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012569799999999999 2024-09-17T21:31:17.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:17.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161283+0000", 2024-09-17T21:31:17.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021201999999999999 2024-09-17T21:31:17.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:17.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.179360+0000", 2024-09-17T21:31:17.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018077784999999999 2024-09-17T21:31:17.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:17.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184726+0000", 2024-09-17T21:31:17.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053652960000000003 2024-09-17T21:31:17.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:17.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184787+0000", 2024-09-17T21:31:17.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1402000000000006e-05 2024-09-17T21:31:17.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184836+0000", 2024-09-17T21:31:17.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9304000000000001e-05 2024-09-17T21:31:17.360 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.360 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.360 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.360 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50206 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.360 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:31:17.360 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.184541561000003, 2024-09-17T21:31:17.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.090361968000000001, 2024-09-17T21:31:17.360 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.360 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.361 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:17.361 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.361 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.361 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.361 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50206 2024-09-17T21:31:17.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.361 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:31:17.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:31:17.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156911+0000", 2024-09-17T21:31:17.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-17T21:31:17.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173670+0000", 2024-09-17T21:31:17.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016758907 2024-09-17T21:31:17.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173696+0000", 2024-09-17T21:31:17.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6329000000000002e-05 2024-09-17T21:31:17.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173708+0000", 2024-09-17T21:31:17.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2512e-05 2024-09-17T21:31:17.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173788+0000", 2024-09-17T21:31:17.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9332999999999994e-05 2024-09-17T21:31:17.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173918+0000", 2024-09-17T21:31:17.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013063300000000001 2024-09-17T21:31:17.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:17.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.176228+0000", 2024-09-17T21:31:17.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002309757 2024-09-17T21:31:17.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:17.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.245164+0000", 2024-09-17T21:31:17.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068936301000000005 2024-09-17T21:31:17.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:17.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247176+0000", 2024-09-17T21:31:17.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020117590000000001 2024-09-17T21:31:17.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:17.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247215+0000", 2024-09-17T21:31:17.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9094000000000002e-05 2024-09-17T21:31:17.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247270+0000", 2024-09-17T21:31:17.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4793999999999997e-05 2024-09-17T21:31:17.366 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.366 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.366 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.366 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50207 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [truncate 21556327] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.366 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:31:17.366 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.167692580999997, 2024-09-17T21:31:17.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.074055363999999999, 2024-09-17T21:31:17.366 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.366 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.366 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:17.366 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.367 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.367 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.367 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50207 2024-09-17T21:31:17.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.367 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:31:17.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:31:17.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173759+0000", 2024-09-17T21:31:17.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8160000000000001e-06 2024-09-17T21:31:17.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173763+0000", 2024-09-17T21:31:17.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5360000000000003e-06 2024-09-17T21:31:17.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173771+0000", 2024-09-17T21:31:17.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8849999999999999e-06 2024-09-17T21:31:17.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173777+0000", 2024-09-17T21:31:17.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7799999999999997e-06 2024-09-17T21:31:17.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221363+0000", 2024-09-17T21:31:17.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047585493 2024-09-17T21:31:17.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221430+0000", 2024-09-17T21:31:17.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7129000000000001e-05 2024-09-17T21:31:17.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:17.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221550+0000", 2024-09-17T21:31:17.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120658 2024-09-17T21:31:17.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:17.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246113+0000", 2024-09-17T21:31:17.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02456262 2024-09-17T21:31:17.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:17.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247744+0000", 2024-09-17T21:31:17.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001630692 2024-09-17T21:31:17.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:17.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247777+0000", 2024-09-17T21:31:17.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3015000000000002e-05 2024-09-17T21:31:17.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247812+0000", 2024-09-17T21:31:17.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.574e-05 2024-09-17T21:31:17.371 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.371 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.371 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.372 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50828 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.372 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:17.372 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.859580648999994, 2024-09-17T21:31:17.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047324185999999997, 2024-09-17T21:31:17.372 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.372 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.372 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.372 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.372 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.372 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.372 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50828 2024-09-17T21:31:17.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.372 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:17.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:17.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:17.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.46e-07 2024-09-17T21:31:17.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481871+0000", 2024-09-17T21:31:17.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T21:31:17.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481878+0000", 2024-09-17T21:31:17.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:31:17.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481882+0000", 2024-09-17T21:31:17.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:31:17.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529038+0000", 2024-09-17T21:31:17.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047156034999999999 2024-09-17T21:31:17.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529099+0000", 2024-09-17T21:31:17.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0316000000000003e-05 2024-09-17T21:31:17.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529193+0000", 2024-09-17T21:31:17.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4369000000000004e-05 2024-09-17T21:31:17.375 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.375 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.376 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.376 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50829 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 12845056~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.376 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:17.376 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.859561854999995, 2024-09-17T21:31:17.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047887596999999997, 2024-09-17T21:31:17.376 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.376 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.376 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.376 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50829 2024-09-17T21:31:17.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.376 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:17.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:17.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:17.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:31:17.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481890+0000", 2024-09-17T21:31:17.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.877e-06 2024-09-17T21:31:17.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481895+0000", 2024-09-17T21:31:17.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:31:17.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481900+0000", 2024-09-17T21:31:17.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4440000000000001e-06 2024-09-17T21:31:17.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529225+0000", 2024-09-17T21:31:17.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047325117999999999 2024-09-17T21:31:17.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529292+0000", 2024-09-17T21:31:17.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7100000000000005e-05 2024-09-17T21:31:17.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529775+0000", 2024-09-17T21:31:17.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048322799999999999 2024-09-17T21:31:17.379 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.379 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.379 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.380 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50830 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.380 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:31:17.380 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.859543453000001, 2024-09-17T21:31:17.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048062123999999998, 2024-09-17T21:31:17.380 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.380 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.380 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.380 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.380 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.380 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.380 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50830 2024-09-17T21:31:17.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.380 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:31:17.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:31:17.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481907+0000", 2024-09-17T21:31:17.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:31:17.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481909+0000", 2024-09-17T21:31:17.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T21:31:17.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481915+0000", 2024-09-17T21:31:17.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.02e-06 2024-09-17T21:31:17.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481920+0000", 2024-09-17T21:31:17.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6799999999999998e-06 2024-09-17T21:31:17.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529793+0000", 2024-09-17T21:31:17.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047873042999999997 2024-09-17T21:31:17.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529870+0000", 2024-09-17T21:31:17.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6967999999999999e-05 2024-09-17T21:31:17.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529968+0000", 2024-09-17T21:31:17.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8085999999999996e-05 2024-09-17T21:31:17.383 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.384 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.384 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.384 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50831 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 18087936~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.384 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:17.384 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.859523107000001, 2024-09-17T21:31:17.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048663304999999997, 2024-09-17T21:31:17.384 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.384 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.384 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.384 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.384 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.384 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.385 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50831 2024-09-17T21:31:17.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.385 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:17.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:17.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:17.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:31:17.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481929+0000", 2024-09-17T21:31:17.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2189999999999998e-06 2024-09-17T21:31:17.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481935+0000", 2024-09-17T21:31:17.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.383e-06 2024-09-17T21:31:17.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481939+0000", 2024-09-17T21:31:17.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2479999999999998e-06 2024-09-17T21:31:17.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530095+0000", 2024-09-17T21:31:17.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048156617999999998 2024-09-17T21:31:17.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530159+0000", 2024-09-17T21:31:17.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3394999999999999e-05 2024-09-17T21:31:17.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530590+0000", 2024-09-17T21:31:17.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043101199999999999 2024-09-17T21:31:17.388 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.388 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.388 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.388 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50832 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 19660800~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.388 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:31:17.388 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.859504459999997, 2024-09-17T21:31:17.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049339466999999998, 2024-09-17T21:31:17.388 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.388 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.388 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.389 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.389 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.389 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50832 2024-09-17T21:31:17.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.389 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:31:17.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:31:17.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481946+0000", 2024-09-17T21:31:17.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:17.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481951+0000", 2024-09-17T21:31:17.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1479999999999997e-06 2024-09-17T21:31:17.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481956+0000", 2024-09-17T21:31:17.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7339999999999996e-06 2024-09-17T21:31:17.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481961+0000", 2024-09-17T21:31:17.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4889999999999997e-06 2024-09-17T21:31:17.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530630+0000", 2024-09-17T21:31:17.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048668596000000001 2024-09-17T21:31:17.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530720+0000", 2024-09-17T21:31:17.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0163999999999999e-05 2024-09-17T21:31:17.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531285+0000", 2024-09-17T21:31:17.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056493199999999998 2024-09-17T21:31:17.392 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.392 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.392 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.392 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50833 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.392 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:31:17.392 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.859482315999998, 2024-09-17T21:31:17.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049908069999999999, 2024-09-17T21:31:17.392 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.392 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.392 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.393 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.393 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.393 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.393 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50833 2024-09-17T21:31:17.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.393 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:31:17.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:31:17.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481968+0000", 2024-09-17T21:31:17.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5600000000000001e-07 2024-09-17T21:31:17.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481970+0000", 2024-09-17T21:31:17.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:31:17.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481975+0000", 2024-09-17T21:31:17.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8499999999999999e-06 2024-09-17T21:31:17.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481980+0000", 2024-09-17T21:31:17.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-17T21:31:17.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531337+0000", 2024-09-17T21:31:17.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049356874000000002 2024-09-17T21:31:17.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531392+0000", 2024-09-17T21:31:17.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4826e-05 2024-09-17T21:31:17.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531875+0000", 2024-09-17T21:31:17.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048345099999999998 2024-09-17T21:31:17.396 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.396 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.396 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.396 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50834 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 24903680~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.396 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:31:17.396 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.859457547999995, 2024-09-17T21:31:17.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050534015000000002, 2024-09-17T21:31:17.397 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.397 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.397 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.397 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.397 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.397 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.397 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50834 2024-09-17T21:31:17.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.397 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:31:17.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:31:17.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481993+0000", 2024-09-17T21:31:17.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:31:17.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481995+0000", 2024-09-17T21:31:17.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.199e-06 2024-09-17T21:31:17.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482001+0000", 2024-09-17T21:31:17.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.772e-06 2024-09-17T21:31:17.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482006+0000", 2024-09-17T21:31:17.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0660000000000003e-06 2024-09-17T21:31:17.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531923+0000", 2024-09-17T21:31:17.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049917200000000002 2024-09-17T21:31:17.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532004+0000", 2024-09-17T21:31:17.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0758000000000002e-05 2024-09-17T21:31:17.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532526+0000", 2024-09-17T21:31:17.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00052258899999999995 2024-09-17T21:31:17.400 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.400 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.400 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.400 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50835 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 28573696~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.400 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:31:17.400 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.859437564999993, 2024-09-17T21:31:17.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051204701999999998, 2024-09-17T21:31:17.401 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.401 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.401 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.401 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50835 2024-09-17T21:31:17.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.401 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:31:17.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:31:17.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482013+0000", 2024-09-17T21:31:17.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:31:17.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482015+0000", 2024-09-17T21:31:17.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.272e-06 2024-09-17T21:31:17.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482020+0000", 2024-09-17T21:31:17.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3040000000000004e-06 2024-09-17T21:31:17.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482025+0000", 2024-09-17T21:31:17.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5650000000000003e-06 2024-09-17T21:31:17.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532569+0000", 2024-09-17T21:31:17.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050544661999999997 2024-09-17T21:31:17.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532649+0000", 2024-09-17T21:31:17.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9640000000000003e-05 2024-09-17T21:31:17.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533217+0000", 2024-09-17T21:31:17.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056784100000000003 2024-09-17T21:31:17.404 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.404 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.404 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.404 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50836 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 30146560~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.405 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:17.405 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.859417948000001, 2024-09-17T21:31:17.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051875589, 2024-09-17T21:31:17.405 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.405 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.405 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.405 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.405 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.405 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.405 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50836 2024-09-17T21:31:17.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.405 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:17.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:17.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:17.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3700000000000001e-07 2024-09-17T21:31:17.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482034+0000", 2024-09-17T21:31:17.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0059999999999999e-06 2024-09-17T21:31:17.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482040+0000", 2024-09-17T21:31:17.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:31:17.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482044+0000", 2024-09-17T21:31:17.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:31:17.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533260+0000", 2024-09-17T21:31:17.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051215416 2024-09-17T21:31:17.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533348+0000", 2024-09-17T21:31:17.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8695000000000004e-05 2024-09-17T21:31:17.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533907+0000", 2024-09-17T21:31:17.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055902199999999995 2024-09-17T21:31:17.408 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.408 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.409 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.409 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50844 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.409 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:31:17.409 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.859254522000001, 2024-09-17T21:31:17.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0475644, 2024-09-17T21:31:17.409 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.409 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.409 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.409 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.409 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.409 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.409 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50844 2024-09-17T21:31:17.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.409 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:31:17.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:31:17.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482196+0000", 2024-09-17T21:31:17.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.39e-07 2024-09-17T21:31:17.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482198+0000", 2024-09-17T21:31:17.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0099999999999998e-06 2024-09-17T21:31:17.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482204+0000", 2024-09-17T21:31:17.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.93e-06 2024-09-17T21:31:17.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482208+0000", 2024-09-17T21:31:17.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7439999999999998e-06 2024-09-17T21:31:17.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529318+0000", 2024-09-17T21:31:17.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047110051999999999 2024-09-17T21:31:17.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529396+0000", 2024-09-17T21:31:17.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7712000000000004e-05 2024-09-17T21:31:17.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529760+0000", 2024-09-17T21:31:17.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00036351299999999999 2024-09-17T21:31:17.413 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.413 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.413 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.413 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50845 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 21757952~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.413 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:31:17.413 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.859235419000001, 2024-09-17T21:31:17.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048084625999999998, 2024-09-17T21:31:17.414 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.414 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.414 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.414 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50845 2024-09-17T21:31:17.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.414 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:31:17.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:31:17.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482215+0000", 2024-09-17T21:31:17.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:31:17.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482216+0000", 2024-09-17T21:31:17.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.823e-06 2024-09-17T21:31:17.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482222+0000", 2024-09-17T21:31:17.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.507e-06 2024-09-17T21:31:17.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482227+0000", 2024-09-17T21:31:17.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6609999999999999e-06 2024-09-17T21:31:17.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529792+0000", 2024-09-17T21:31:17.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047565075999999998 2024-09-17T21:31:17.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529871+0000", 2024-09-17T21:31:17.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8855000000000003e-05 2024-09-17T21:31:17.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530299+0000", 2024-09-17T21:31:17.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000428307 2024-09-17T21:31:17.417 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.417 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.417 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.417 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50846 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 25427968~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.417 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:17.417 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.859213032, 2024-09-17T21:31:17.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048735142000000002, 2024-09-17T21:31:17.418 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.418 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.418 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.418 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.418 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.418 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.418 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50846 2024-09-17T21:31:17.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.418 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:17.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:17.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:17.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:31:17.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482239+0000", 2024-09-17T21:31:17.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.871e-06 2024-09-17T21:31:17.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482244+0000", 2024-09-17T21:31:17.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.49e-06 2024-09-17T21:31:17.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482249+0000", 2024-09-17T21:31:17.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6269999999999999e-06 2024-09-17T21:31:17.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530347+0000", 2024-09-17T21:31:17.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048097721000000003 2024-09-17T21:31:17.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530437+0000", 2024-09-17T21:31:17.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0604000000000002e-05 2024-09-17T21:31:17.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530972+0000", 2024-09-17T21:31:17.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053445000000000001 2024-09-17T21:31:17.421 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.421 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.421 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.422 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50847 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 27000832~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.422 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:17.422 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.859195135999997, 2024-09-17T21:31:17.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049308668, 2024-09-17T21:31:17.422 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.422 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.422 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.422 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.422 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.422 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.422 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50847 2024-09-17T21:31:17.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.423 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:17.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:17.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:17.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:31:17.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482257+0000", 2024-09-17T21:31:17.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.846e-06 2024-09-17T21:31:17.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482264+0000", 2024-09-17T21:31:17.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8170000000000001e-06 2024-09-17T21:31:17.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482283+0000", 2024-09-17T21:31:17.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9718e-05 2024-09-17T21:31:17.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531018+0000", 2024-09-17T21:31:17.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048734497000000002 2024-09-17T21:31:17.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531080+0000", 2024-09-17T21:31:17.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1795e-05 2024-09-17T21:31:17.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531563+0000", 2024-09-17T21:31:17.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048366499999999998 2024-09-17T21:31:17.426 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.426 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.426 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.426 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50848 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 30670848~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.426 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:31:17.426 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.859158593999993, 2024-09-17T21:31:17.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049893337000000003, 2024-09-17T21:31:17.426 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.426 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.426 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.426 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.426 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.427 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.427 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50848 2024-09-17T21:31:17.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.427 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:31:17.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:31:17.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482292+0000", 2024-09-17T21:31:17.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T21:31:17.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482294+0000", 2024-09-17T21:31:17.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3209999999999999e-06 2024-09-17T21:31:17.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482300+0000", 2024-09-17T21:31:17.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7189999999999998e-06 2024-09-17T21:31:17.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482305+0000", 2024-09-17T21:31:17.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T21:31:17.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531606+0000", 2024-09-17T21:31:17.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049301273999999999 2024-09-17T21:31:17.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531696+0000", 2024-09-17T21:31:17.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0161999999999995e-05 2024-09-17T21:31:17.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532184+0000", 2024-09-17T21:31:17.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048806400000000002 2024-09-17T21:31:17.430 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.430 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.430 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.430 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50849 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 32243712~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:17.430 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:31:17.430 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.859139153000001, 2024-09-17T21:31:17.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050444607000000002, 2024-09-17T21:31:17.430 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.430 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.430 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.430 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.431 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.431 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.431 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50849 2024-09-17T21:31:17.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.431 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:31:17.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:31:17.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482311+0000", 2024-09-17T21:31:17.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:17.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482316+0000", 2024-09-17T21:31:17.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1209999999999998e-06 2024-09-17T21:31:17.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482321+0000", 2024-09-17T21:31:17.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4779999999999999e-06 2024-09-17T21:31:17.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482326+0000", 2024-09-17T21:31:17.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7099999999999998e-06 2024-09-17T21:31:17.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532221+0000", 2024-09-17T21:31:17.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049894936000000001 2024-09-17T21:31:17.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532287+0000", 2024-09-17T21:31:17.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5914000000000006e-05 2024-09-17T21:31:17.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532755+0000", 2024-09-17T21:31:17.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000468107 2024-09-17T21:31:17.434 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.434 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.434 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.434 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.434 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.434 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:17.436 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:31:17.436 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:17.434+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:17.537 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:17.534+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:17.590 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:17.590 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:17.590 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:17.590 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:17.600 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:31:17.643 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:17.641+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:17.704 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:17.704 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:17.704 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:17.704 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:17.705 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:17.713 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:31:17.738 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:17.737+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:17.839 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:17.836+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:17.858 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:17.858 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:17.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:17.858 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:17.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.858 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "pg_update_log_missing(3.5 epoch 45/37 rep_tid 844 entries 45'73 (0'0) error 3:a65c40ae:::rbd_data.1120b21c3309.0000000000000126:head by client.4384.0:48900 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 45'70)", 2024-09-17T21:31:17.858 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:31:17.858 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.808251702999996, 2024-09-17T21:31:17.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019554365000000001, 2024-09-17T21:31:17.858 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.859 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.859 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T21:31:17.859 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:31:17.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:31:17.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047112+0000", 2024-09-17T21:31:17.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:31:17.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047121+0000", 2024-09-17T21:31:17.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4379999999999995e-06 2024-09-17T21:31:17.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047139+0000", 2024-09-17T21:31:17.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8235000000000001e-05 2024-09-17T21:31:17.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047303+0000", 2024-09-17T21:31:17.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016389400000000001 2024-09-17T21:31:17.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047396+0000", 2024-09-17T21:31:17.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3355000000000003e-05 2024-09-17T21:31:17.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066666+0000", 2024-09-17T21:31:17.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019269649999999999 2024-09-17T21:31:17.861 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.862 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.862 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.862 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48906 3.4 e45/25 3:2503fe27:::rbd_data.1120b21c3309.0000000000000127:head v 45'148, mlcod=45'146)", 2024-09-17T21:31:17.862 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:17.862 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.806125523999995, 2024-09-17T21:31:17.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018169739000000001, 2024-09-17T21:31:17.862 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.862 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.862 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:17.862 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:17.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:17.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:17.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:31:17.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049244+0000", 2024-09-17T21:31:17.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8390000000000003e-06 2024-09-17T21:31:17.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066411+0000", 2024-09-17T21:31:17.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017166818 2024-09-17T21:31:17.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066426+0000", 2024-09-17T21:31:17.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4888999999999999e-05 2024-09-17T21:31:17.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066611+0000", 2024-09-17T21:31:17.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018464499999999999 2024-09-17T21:31:17.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066630+0000", 2024-09-17T21:31:17.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9201000000000001e-05 2024-09-17T21:31:17.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:17.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067373+0000", 2024-09-17T21:31:17.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00074322600000000004 2024-09-17T21:31:17.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067407+0000", 2024-09-17T21:31:17.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4357999999999999e-05 2024-09-17T21:31:17.868 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.868 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.869 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.869 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50206 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'301, mlcod=45'299)", 2024-09-17T21:31:17.869 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:31:17.869 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.67664868, 2024-09-17T21:31:17.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068285810000000002, 2024-09-17T21:31:17.869 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.869 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.869 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:17.869 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:31:17.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:31:17.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178716+0000", 2024-09-17T21:31:17.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.438e-06 2024-09-17T21:31:17.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192044+0000", 2024-09-17T21:31:17.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013328370000000001 2024-09-17T21:31:17.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192069+0000", 2024-09-17T21:31:17.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4496000000000001e-05 2024-09-17T21:31:17.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192085+0000", 2024-09-17T21:31:17.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5758999999999999e-05 2024-09-17T21:31:17.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192167+0000", 2024-09-17T21:31:17.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2842000000000004e-05 2024-09-17T21:31:17.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192201+0000", 2024-09-17T21:31:17.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3127999999999999e-05 2024-09-17T21:31:17.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:17.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246938+0000", 2024-09-17T21:31:17.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054736982000000003 2024-09-17T21:31:17.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247000+0000", 2024-09-17T21:31:17.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2794999999999998e-05 2024-09-17T21:31:17.873 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.873 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.873 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.873 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50207 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'303, mlcod=45'299)", 2024-09-17T21:31:17.873 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:31:17.873 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.633701630999994, 2024-09-17T21:31:17.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026003433999999999, 2024-09-17T21:31:17.874 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.874 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.874 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:17.874 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:31:17.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:31:17.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221663+0000", 2024-09-17T21:31:17.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.271e-06 2024-09-17T21:31:17.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221672+0000", 2024-09-17T21:31:17.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6810000000000004e-06 2024-09-17T21:31:17.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221684+0000", 2024-09-17T21:31:17.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:31:17.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221691+0000", 2024-09-17T21:31:17.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:31:17.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221743+0000", 2024-09-17T21:31:17.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2868999999999998e-05 2024-09-17T21:31:17.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221765+0000", 2024-09-17T21:31:17.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1316999999999999e-05 2024-09-17T21:31:17.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:17.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247616+0000", 2024-09-17T21:31:17.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025851504000000001 2024-09-17T21:31:17.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247665+0000", 2024-09-17T21:31:17.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8739000000000002e-05 2024-09-17T21:31:17.877 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.877 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.877 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.878 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57808 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 196608~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:17.878 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:17.878 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 56.077402591000002, 2024-09-17T21:31:17.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017751475999999999, 2024-09-17T21:31:17.878 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.878 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.878 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.878 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.878 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.878 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.878 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57808 2024-09-17T21:31:17.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.879 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:17.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:17.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:17.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:31:17.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777968+0000", 2024-09-17T21:31:17.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8530000000000003e-06 2024-09-17T21:31:17.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777975+0000", 2024-09-17T21:31:17.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4410000000000004e-06 2024-09-17T21:31:17.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777980+0000", 2024-09-17T21:31:17.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7210000000000002e-06 2024-09-17T21:31:17.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795323+0000", 2024-09-17T21:31:17.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017343319999999999 2024-09-17T21:31:17.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795410+0000", 2024-09-17T21:31:17.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6567e-05 2024-09-17T21:31:17.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795712+0000", 2024-09-17T21:31:17.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030219200000000001 2024-09-17T21:31:17.882 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.882 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.882 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.882 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57809 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 458752~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:17.882 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:31:17.882 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 56.077376739000002, 2024-09-17T21:31:17.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018092869000000001, 2024-09-17T21:31:17.882 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.882 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.882 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.883 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.883 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.883 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.883 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57809 2024-09-17T21:31:17.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.883 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:31:17.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:31:17.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777987+0000", 2024-09-17T21:31:17.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T21:31:17.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777989+0000", 2024-09-17T21:31:17.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8649999999999999e-06 2024-09-17T21:31:17.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777994+0000", 2024-09-17T21:31:17.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4199999999999998e-06 2024-09-17T21:31:17.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778004+0000", 2024-09-17T21:31:17.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6900000000000004e-06 2024-09-17T21:31:17.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795745+0000", 2024-09-17T21:31:17.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017740859000000001 2024-09-17T21:31:17.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795815+0000", 2024-09-17T21:31:17.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0219e-05 2024-09-17T21:31:17.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796079+0000", 2024-09-17T21:31:17.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000264446 2024-09-17T21:31:17.886 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.886 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.886 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.886 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57810 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 720896~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:17.886 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:31:17.887 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 56.077353012000003, 2024-09-17T21:31:17.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018361253000000001, 2024-09-17T21:31:17.887 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.887 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.887 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.887 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.887 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.887 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.887 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57810 2024-09-17T21:31:17.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.887 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:31:17.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:31:17.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778011+0000", 2024-09-17T21:31:17.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:31:17.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778012+0000", 2024-09-17T21:31:17.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.872e-06 2024-09-17T21:31:17.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778018+0000", 2024-09-17T21:31:17.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8150000000000002e-06 2024-09-17T21:31:17.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778023+0000", 2024-09-17T21:31:17.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3939999999999998e-06 2024-09-17T21:31:17.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796106+0000", 2024-09-17T21:31:17.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018083446999999999 2024-09-17T21:31:17.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796159+0000", 2024-09-17T21:31:17.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3029999999999999e-05 2024-09-17T21:31:17.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796371+0000", 2024-09-17T21:31:17.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021231700000000001 2024-09-17T21:31:17.890 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.890 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.890 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.891 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57811 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:17.891 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:17.891 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 56.077334592, 2024-09-17T21:31:17.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018666035000000001, 2024-09-17T21:31:17.891 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.891 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.891 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.891 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.891 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.891 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.891 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57811 2024-09-17T21:31:17.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.891 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:17.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:17.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:17.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:17.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778031+0000", 2024-09-17T21:31:17.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.796e-06 2024-09-17T21:31:17.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778036+0000", 2024-09-17T21:31:17.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3290000000000001e-06 2024-09-17T21:31:17.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778041+0000", 2024-09-17T21:31:17.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5839999999999996e-06 2024-09-17T21:31:17.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796409+0000", 2024-09-17T21:31:17.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018368703 2024-09-17T21:31:17.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796483+0000", 2024-09-17T21:31:17.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3751999999999995e-05 2024-09-17T21:31:17.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796695+0000", 2024-09-17T21:31:17.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021153 2024-09-17T21:31:17.894 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.895 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.895 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.895 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57812 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1245184~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:17.895 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:17.895 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 56.077315695000003, 2024-09-17T21:31:17.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019000064000000001, 2024-09-17T21:31:17.895 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.895 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.895 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.895 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.895 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.895 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.895 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57812 2024-09-17T21:31:17.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.896 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:17.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:17.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:17.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:31:17.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778050+0000", 2024-09-17T21:31:17.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7e-06 2024-09-17T21:31:17.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778055+0000", 2024-09-17T21:31:17.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4049999999999999e-06 2024-09-17T21:31:17.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778059+0000", 2024-09-17T21:31:17.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1479999999999999e-06 2024-09-17T21:31:17.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796734+0000", 2024-09-17T21:31:17.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018674399000000001 2024-09-17T21:31:17.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796786+0000", 2024-09-17T21:31:17.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2879e-05 2024-09-17T21:31:17.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797048+0000", 2024-09-17T21:31:17.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026118400000000003 2024-09-17T21:31:17.899 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.899 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.899 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.899 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57813 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1507328~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:17.899 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:31:17.899 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 56.077297803, 2024-09-17T21:31:17.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019279392999999999, 2024-09-17T21:31:17.899 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.899 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.899 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.899 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.899 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.900 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.900 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57813 2024-09-17T21:31:17.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.900 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:31:17.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:31:17.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778066+0000", 2024-09-17T21:31:17.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6199999999999999e-07 2024-09-17T21:31:17.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778068+0000", 2024-09-17T21:31:17.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7990000000000001e-06 2024-09-17T21:31:17.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778074+0000", 2024-09-17T21:31:17.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:31:17.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778078+0000", 2024-09-17T21:31:17.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2579999999999997e-06 2024-09-17T21:31:17.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797077+0000", 2024-09-17T21:31:17.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018999180000000001 2024-09-17T21:31:17.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797123+0000", 2024-09-17T21:31:17.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5834999999999997e-05 2024-09-17T21:31:17.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797345+0000", 2024-09-17T21:31:17.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022191100000000001 2024-09-17T21:31:17.903 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.903 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.903 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.903 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57814 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:17.903 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:31:17.903 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 56.077278931000002, 2024-09-17T21:31:17.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019604947000000001, 2024-09-17T21:31:17.903 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.903 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.903 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.904 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57814 2024-09-17T21:31:17.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.904 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:31:17.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:31:17.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778085+0000", 2024-09-17T21:31:17.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:31:17.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778086+0000", 2024-09-17T21:31:17.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7850000000000001e-06 2024-09-17T21:31:17.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778093+0000", 2024-09-17T21:31:17.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6449999999999999e-06 2024-09-17T21:31:17.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778098+0000", 2024-09-17T21:31:17.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:31:17.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797383+0000", 2024-09-17T21:31:17.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019284879000000001 2024-09-17T21:31:17.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797431+0000", 2024-09-17T21:31:17.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8856000000000001e-05 2024-09-17T21:31:17.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797689+0000", 2024-09-17T21:31:17.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025785800000000002 2024-09-17T21:31:17.907 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.907 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.907 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.907 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57815 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2031616~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:17.907 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:17.907 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 56.077258262999997, 2024-09-17T21:31:17.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019910231, 2024-09-17T21:31:17.908 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.908 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.908 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.908 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57815 2024-09-17T21:31:17.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.908 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:17.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:17.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:17.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:31:17.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778118+0000", 2024-09-17T21:31:17.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2967e-05 2024-09-17T21:31:17.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778125+0000", 2024-09-17T21:31:17.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.601e-06 2024-09-17T21:31:17.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778130+0000", 2024-09-17T21:31:17.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1839999999999998e-06 2024-09-17T21:31:17.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797720+0000", 2024-09-17T21:31:17.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019589766000000002 2024-09-17T21:31:17.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797765+0000", 2024-09-17T21:31:17.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5281999999999998e-05 2024-09-17T21:31:17.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798015+0000", 2024-09-17T21:31:17.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025006200000000001 2024-09-17T21:31:17.911 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.911 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.912 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.912 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57816 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2293760~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:17.912 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:17.912 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 56.077225669000001, 2024-09-17T21:31:17.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020175322999999998, 2024-09-17T21:31:17.912 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.912 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.912 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.912 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.912 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.912 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.912 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57816 2024-09-17T21:31:17.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.913 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:17.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:17.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:17.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:31:17.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778140+0000", 2024-09-17T21:31:17.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T21:31:17.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778145+0000", 2024-09-17T21:31:17.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7320000000000003e-06 2024-09-17T21:31:17.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778154+0000", 2024-09-17T21:31:17.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6969999999999999e-06 2024-09-17T21:31:17.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798049+0000", 2024-09-17T21:31:17.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019894603 2024-09-17T21:31:17.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798106+0000", 2024-09-17T21:31:17.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7207000000000003e-05 2024-09-17T21:31:17.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798313+0000", 2024-09-17T21:31:17.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206861 2024-09-17T21:31:17.916 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.916 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.916 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.916 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57817 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2555904~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:17.916 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:17.916 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 56.077202423000003, 2024-09-17T21:31:17.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020466314999999999, 2024-09-17T21:31:17.916 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57817 2024-09-17T21:31:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2399999999999999e-07 2024-09-17T21:31:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778163+0000", 2024-09-17T21:31:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.204e-06 2024-09-17T21:31:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778169+0000", 2024-09-17T21:31:17.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:31:17.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778173+0000", 2024-09-17T21:31:17.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:31:17.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798342+0000", 2024-09-17T21:31:17.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020168416000000002 2024-09-17T21:31:17.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798388+0000", 2024-09-17T21:31:17.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6171000000000002e-05 2024-09-17T21:31:17.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798627+0000", 2024-09-17T21:31:17.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000239058 2024-09-17T21:31:17.920 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.920 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.920 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.920 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57818 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2818048~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:17.920 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:31:17.920 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 56.077183726000001, 2024-09-17T21:31:17.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020767240999999999, 2024-09-17T21:31:17.921 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.921 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.921 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.921 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.921 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.921 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.921 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57818 2024-09-17T21:31:17.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.921 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:31:17.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:31:17.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778180+0000", 2024-09-17T21:31:17.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:31:17.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778182+0000", 2024-09-17T21:31:17.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7269999999999999e-06 2024-09-17T21:31:17.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778188+0000", 2024-09-17T21:31:17.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6080000000000001e-06 2024-09-17T21:31:17.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778202+0000", 2024-09-17T21:31:17.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3369e-05 2024-09-17T21:31:17.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798660+0000", 2024-09-17T21:31:17.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020458171000000001 2024-09-17T21:31:17.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798704+0000", 2024-09-17T21:31:17.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4273000000000001e-05 2024-09-17T21:31:17.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798947+0000", 2024-09-17T21:31:17.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242651 2024-09-17T21:31:17.924 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.924 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.925 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.925 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57819 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:17.925 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:31:17.925 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 56.077152857000002, 2024-09-17T21:31:17.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020892340999999998, 2024-09-17T21:31:17.925 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.925 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.925 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.925 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57819 2024-09-17T21:31:17.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.926 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:31:17.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:31:17.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778211+0000", 2024-09-17T21:31:17.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:31:17.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778217+0000", 2024-09-17T21:31:17.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T21:31:17.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778224+0000", 2024-09-17T21:31:17.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3520000000000003e-06 2024-09-17T21:31:17.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778228+0000", 2024-09-17T21:31:17.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7480000000000001e-06 2024-09-17T21:31:17.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798977+0000", 2024-09-17T21:31:17.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020748571 2024-09-17T21:31:17.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799035+0000", 2024-09-17T21:31:17.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7908999999999997e-05 2024-09-17T21:31:17.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799103+0000", 2024-09-17T21:31:17.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7794999999999997e-05 2024-09-17T21:31:17.929 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.929 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.929 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.929 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57820 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:17.929 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:17.929 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 56.077127664999999, 2024-09-17T21:31:17.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021016658000000001, 2024-09-17T21:31:17.929 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.929 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.930 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.930 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.930 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.930 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.930 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57820 2024-09-17T21:31:17.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.930 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:17.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:17.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:17.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:31:17.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778238+0000", 2024-09-17T21:31:17.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.945e-06 2024-09-17T21:31:17.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778244+0000", 2024-09-17T21:31:17.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6520000000000003e-06 2024-09-17T21:31:17.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778248+0000", 2024-09-17T21:31:17.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5920000000000002e-06 2024-09-17T21:31:17.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799131+0000", 2024-09-17T21:31:17.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020883288999999999 2024-09-17T21:31:17.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799185+0000", 2024-09-17T21:31:17.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3223000000000002e-05 2024-09-17T21:31:17.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799252+0000", 2024-09-17T21:31:17.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7576999999999994e-05 2024-09-17T21:31:17.933 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.933 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.933 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.933 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57821 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:17.933 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:17.934 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 56.077109319999998, 2024-09-17T21:31:17.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021153528000000001, 2024-09-17T21:31:17.934 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.934 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.934 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.934 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.934 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.934 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.934 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57821 2024-09-17T21:31:17.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.934 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:17.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:17.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:17.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-07 2024-09-17T21:31:17.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778256+0000", 2024-09-17T21:31:17.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.604e-06 2024-09-17T21:31:17.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778262+0000", 2024-09-17T21:31:17.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7629999999999997e-06 2024-09-17T21:31:17.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778266+0000", 2024-09-17T21:31:17.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:31:17.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799284+0000", 2024-09-17T21:31:17.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021017312 2024-09-17T21:31:17.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799326+0000", 2024-09-17T21:31:17.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2639000000000003e-05 2024-09-17T21:31:17.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799407+0000", 2024-09-17T21:31:17.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1077999999999999e-05 2024-09-17T21:31:17.937 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.937 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.937 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.938 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57822 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:17.938 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:31:17.938 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 56.077091009999997, 2024-09-17T21:31:17.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021268966, 2024-09-17T21:31:17.938 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.938 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.938 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.938 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.938 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.938 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.938 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57822 2024-09-17T21:31:17.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.938 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:31:17.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:31:17.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778273+0000", 2024-09-17T21:31:17.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:31:17.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778274+0000", 2024-09-17T21:31:17.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.598e-06 2024-09-17T21:31:17.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778283+0000", 2024-09-17T21:31:17.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8279999999999992e-06 2024-09-17T21:31:17.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778289+0000", 2024-09-17T21:31:17.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5609999999999998e-06 2024-09-17T21:31:17.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799432+0000", 2024-09-17T21:31:17.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021143862999999999 2024-09-17T21:31:17.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799473+0000", 2024-09-17T21:31:17.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1075999999999998e-05 2024-09-17T21:31:17.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799541+0000", 2024-09-17T21:31:17.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7657999999999995e-05 2024-09-17T21:31:17.942 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.942 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.942 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.942 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57823 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:17.942 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:17.942 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 56.077068048999998, 2024-09-17T21:31:17.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021379770999999999, 2024-09-17T21:31:17.942 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:17.942 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:17.942 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:17.943 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:17.943 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:17.943 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:17.943 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57823 2024-09-17T21:31:17.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.943 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:17.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:17.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:17.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:17.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:17.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:17.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:17.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:17.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:31:17.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:17.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778297+0000", 2024-09-17T21:31:17.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:31:17.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:17.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778303+0000", 2024-09-17T21:31:17.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4559999999999999e-06 2024-09-17T21:31:17.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:17.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778307+0000", 2024-09-17T21:31:17.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6229999999999996e-06 2024-09-17T21:31:17.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:17.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799560+0000", 2024-09-17T21:31:17.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021252931999999999 2024-09-17T21:31:17.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:17.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799609+0000", 2024-09-17T21:31:17.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8301999999999999e-05 2024-09-17T21:31:17.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:17.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:17.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:17.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799675+0000", 2024-09-17T21:31:17.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6205999999999996e-05 2024-09-17T21:31:17.946 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.946 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.946 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.946 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:17.946 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:17.946 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:17.948 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:31:17.948 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:17.945+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:18.048 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:18.045+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:18.091 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:18.091 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:18.091 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:18.091 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:18.100 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:31:18.183 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:18.181+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:18.241 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:18.241 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:18.242 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:18.242 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:18.242 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:18.249 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:18.246+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:18.250 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:31:18.298 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:31:18.350 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:18.348+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:18.397 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:18.397 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:18.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:18.397 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:18.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.397 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:49361 3.19 e46/25 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:head v 46'135, mlcod=45'133)", 2024-09-17T21:31:18.397 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:18.397 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.640660858999993, 2024-09-17T21:31:18.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043477744999999998, 2024-09-17T21:31:18.398 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.398 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.398 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:18.398 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:18.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:18.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:18.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-07 2024-09-17T21:31:18.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769703+0000", 2024-09-17T21:31:18.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016134381 2024-09-17T21:31:18.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769724+0000", 2024-09-17T21:31:18.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0781999999999999e-05 2024-09-17T21:31:18.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769734+0000", 2024-09-17T21:31:18.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6870000000000006e-06 2024-09-17T21:31:18.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769797+0000", 2024-09-17T21:31:18.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3418999999999997e-05 2024-09-17T21:31:18.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769822+0000", 2024-09-17T21:31:18.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4652999999999999e-05 2024-09-17T21:31:18.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:18.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.796982+0000", 2024-09-17T21:31:18.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027160205 2024-09-17T21:31:18.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.797047+0000", 2024-09-17T21:31:18.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4455999999999994e-05 2024-09-17T21:31:18.401 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.401 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.401 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.402 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50805 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 17825792~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:18.402 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:31:18.402 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.899077364999997, 2024-09-17T21:31:18.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035591579999999998, 2024-09-17T21:31:18.402 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.402 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.402 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.402 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.402 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.402 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.402 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50805 2024-09-17T21:31:18.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.402 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:31:18.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:31:18.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495153+0000", 2024-09-17T21:31:18.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:31:18.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495155+0000", 2024-09-17T21:31:18.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1849999999999998e-06 2024-09-17T21:31:18.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495162+0000", 2024-09-17T21:31:18.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.64e-06 2024-09-17T21:31:18.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495166+0000", 2024-09-17T21:31:18.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:31:18.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530020+0000", 2024-09-17T21:31:18.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.034853824999999998 2024-09-17T21:31:18.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530076+0000", 2024-09-17T21:31:18.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5235999999999998e-05 2024-09-17T21:31:18.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530744+0000", 2024-09-17T21:31:18.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066835500000000003 2024-09-17T21:31:18.405 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.405 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.405 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.406 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50806 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19398656~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:18.406 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:18.406 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.899056896999994, 2024-09-17T21:31:18.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036408459999999997, 2024-09-17T21:31:18.406 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.406 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.406 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.406 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50806 2024-09-17T21:31:18.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.406 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:18.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:18.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:18.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.72e-07 2024-09-17T21:31:18.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495175+0000", 2024-09-17T21:31:18.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:31:18.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495182+0000", 2024-09-17T21:31:18.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8079999999999999e-06 2024-09-17T21:31:18.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495186+0000", 2024-09-17T21:31:18.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5549999999999996e-06 2024-09-17T21:31:18.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530791+0000", 2024-09-17T21:31:18.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035604963000000003 2024-09-17T21:31:18.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530855+0000", 2024-09-17T21:31:18.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3286000000000004e-05 2024-09-17T21:31:18.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531581+0000", 2024-09-17T21:31:18.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00072655699999999996 2024-09-17T21:31:18.409 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.409 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.409 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.410 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50807 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23068672~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:18.410 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:18.410 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.899038012000005, 2024-09-17T21:31:18.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037183314000000002, 2024-09-17T21:31:18.410 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.410 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.410 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.410 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.410 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.410 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.410 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50807 2024-09-17T21:31:18.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.410 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:18.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:18.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:18.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9999999999999999e-07 2024-09-17T21:31:18.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495194+0000", 2024-09-17T21:31:18.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.779e-06 2024-09-17T21:31:18.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495199+0000", 2024-09-17T21:31:18.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.468e-06 2024-09-17T21:31:18.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495203+0000", 2024-09-17T21:31:18.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0010000000000003e-06 2024-09-17T21:31:18.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531636+0000", 2024-09-17T21:31:18.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036432665000000003 2024-09-17T21:31:18.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531727+0000", 2024-09-17T21:31:18.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1607999999999994e-05 2024-09-17T21:31:18.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532375+0000", 2024-09-17T21:31:18.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00064749299999999996 2024-09-17T21:31:18.414 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.414 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.414 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.414 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50808 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24641536~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:18.414 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:31:18.414 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.899021269000002, 2024-09-17T21:31:18.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037870190999999997, 2024-09-17T21:31:18.414 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.414 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.414 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.415 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.415 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50808 2024-09-17T21:31:18.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.415 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:31:18.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:31:18.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495209+0000", 2024-09-17T21:31:18.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:31:18.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495215+0000", 2024-09-17T21:31:18.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1630000000000001e-06 2024-09-17T21:31:18.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495221+0000", 2024-09-17T21:31:18.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-17T21:31:18.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495226+0000", 2024-09-17T21:31:18.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8189999999999999e-06 2024-09-17T21:31:18.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532425+0000", 2024-09-17T21:31:18.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037199135000000001 2024-09-17T21:31:18.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532531+0000", 2024-09-17T21:31:18.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106314 2024-09-17T21:31:18.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533079+0000", 2024-09-17T21:31:18.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00054739700000000003 2024-09-17T21:31:18.418 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.418 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.418 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.418 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50809 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28311552~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:18.418 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:31:18.418 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.898998302999999, 2024-09-17T21:31:18.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038597605, 2024-09-17T21:31:18.418 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.418 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.418 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.419 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.419 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.419 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.419 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50809 2024-09-17T21:31:18.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.419 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:31:18.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:31:18.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495232+0000", 2024-09-17T21:31:18.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.39e-07 2024-09-17T21:31:18.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495234+0000", 2024-09-17T21:31:18.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T21:31:18.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495239+0000", 2024-09-17T21:31:18.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7969999999999997e-06 2024-09-17T21:31:18.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495244+0000", 2024-09-17T21:31:18.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999996e-06 2024-09-17T21:31:18.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533125+0000", 2024-09-17T21:31:18.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037881523 2024-09-17T21:31:18.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533237+0000", 2024-09-17T21:31:18.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000111879 2024-09-17T21:31:18.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533829+0000", 2024-09-17T21:31:18.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000591933 2024-09-17T21:31:18.422 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.422 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.422 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.422 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50810 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 29884416~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:18.422 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:31:18.422 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.898980385000002, 2024-09-17T21:31:18.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039149809000000001, 2024-09-17T21:31:18.422 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.423 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.423 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.423 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.423 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.423 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.423 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50810 2024-09-17T21:31:18.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.423 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:31:18.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:31:18.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495250+0000", 2024-09-17T21:31:18.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:31:18.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495251+0000", 2024-09-17T21:31:18.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.81e-06 2024-09-17T21:31:18.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495257+0000", 2024-09-17T21:31:18.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1399999999999999e-06 2024-09-17T21:31:18.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495261+0000", 2024-09-17T21:31:18.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4229999999999998e-06 2024-09-17T21:31:18.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533879+0000", 2024-09-17T21:31:18.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038618292999999998 2024-09-17T21:31:18.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533939+0000", 2024-09-17T21:31:18.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.012e-05 2024-09-17T21:31:18.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534399+0000", 2024-09-17T21:31:18.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00045968099999999998 2024-09-17T21:31:18.426 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.426 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.426 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.426 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50850 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 2359296~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:18.426 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:18.426 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.898962010000005, 2024-09-17T21:31:18.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039731491000000001, 2024-09-17T21:31:18.427 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.427 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.427 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.427 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.427 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.427 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.427 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50850 2024-09-17T21:31:18.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.427 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:18.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:18.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:18.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3200000000000001e-07 2024-09-17T21:31:18.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495270+0000", 2024-09-17T21:31:18.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.694e-06 2024-09-17T21:31:18.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495275+0000", 2024-09-17T21:31:18.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5890000000000002e-06 2024-09-17T21:31:18.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495279+0000", 2024-09-17T21:31:18.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-17T21:31:18.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534440+0000", 2024-09-17T21:31:18.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039160805999999999 2024-09-17T21:31:18.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534526+0000", 2024-09-17T21:31:18.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5327999999999993e-05 2024-09-17T21:31:18.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534999+0000", 2024-09-17T21:31:18.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00047353400000000001 2024-09-17T21:31:18.430 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.430 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.430 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.430 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50851 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 3932160~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:18.430 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:31:18.431 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.898944397999998, 2024-09-17T21:31:18.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040211991000000002, 2024-09-17T21:31:18.431 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.431 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.431 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.431 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.431 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.431 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.431 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50851 2024-09-17T21:31:18.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.431 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:31:18.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:31:18.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495286+0000", 2024-09-17T21:31:18.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:31:18.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495287+0000", 2024-09-17T21:31:18.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6640000000000001e-06 2024-09-17T21:31:18.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495293+0000", 2024-09-17T21:31:18.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6710000000000004e-06 2024-09-17T21:31:18.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495298+0000", 2024-09-17T21:31:18.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6700000000000002e-06 2024-09-17T21:31:18.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535017+0000", 2024-09-17T21:31:18.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039719059000000001 2024-09-17T21:31:18.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535067+0000", 2024-09-17T21:31:18.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0046000000000002e-05 2024-09-17T21:31:18.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535497+0000", 2024-09-17T21:31:18.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043050299999999999 2024-09-17T21:31:18.434 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.434 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.434 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.434 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50852 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 7602176~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:18.434 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:18.434 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.898925481000006, 2024-09-17T21:31:18.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041166337999999997, 2024-09-17T21:31:18.435 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.435 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.435 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.435 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.435 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.435 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.435 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50852 2024-09-17T21:31:18.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.435 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:18.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:18.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495305+0000", 2024-09-17T21:31:18.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1600000000000002e-07 2024-09-17T21:31:18.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495307+0000", 2024-09-17T21:31:18.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0980000000000001e-06 2024-09-17T21:31:18.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495312+0000", 2024-09-17T21:31:18.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6210000000000001e-06 2024-09-17T21:31:18.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495317+0000", 2024-09-17T21:31:18.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T21:31:18.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535531+0000", 2024-09-17T21:31:18.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040214580999999999 2024-09-17T21:31:18.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535618+0000", 2024-09-17T21:31:18.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6648000000000001e-05 2024-09-17T21:31:18.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536471+0000", 2024-09-17T21:31:18.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00085251599999999999 2024-09-17T21:31:18.438 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.438 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.438 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.438 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50853 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 9175040~204800] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:18.439 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:18.439 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.898907149999999, 2024-09-17T21:31:18.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041673496999999997, 2024-09-17T21:31:18.439 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.439 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.439 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.439 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.439 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.439 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.439 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50853 2024-09-17T21:31:18.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.439 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:18.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:18.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:18.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:31:18.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495334+0000", 2024-09-17T21:31:18.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.076e-05 2024-09-17T21:31:18.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495340+0000", 2024-09-17T21:31:18.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:31:18.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495345+0000", 2024-09-17T21:31:18.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:31:18.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536514+0000", 2024-09-17T21:31:18.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041168534999999999 2024-09-17T21:31:18.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536572+0000", 2024-09-17T21:31:18.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8742000000000001e-05 2024-09-17T21:31:18.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536996+0000", 2024-09-17T21:31:18.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042368899999999999 2024-09-17T21:31:18.442 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.442 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.442 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.442 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50854 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 12845056~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:18.443 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:18.443 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.898878091, 2024-09-17T21:31:18.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042777397000000002, 2024-09-17T21:31:18.443 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.443 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.443 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.443 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50854 2024-09-17T21:31:18.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.443 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:18.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:18.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:18.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.82e-07 2024-09-17T21:31:18.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495354+0000", 2024-09-17T21:31:18.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.091e-06 2024-09-17T21:31:18.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495360+0000", 2024-09-17T21:31:18.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.716e-06 2024-09-17T21:31:18.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495365+0000", 2024-09-17T21:31:18.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2939999999999996e-06 2024-09-17T21:31:18.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537020+0000", 2024-09-17T21:31:18.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041654419999999998 2024-09-17T21:31:18.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537105+0000", 2024-09-17T21:31:18.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5857000000000001e-05 2024-09-17T21:31:18.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538129+0000", 2024-09-17T21:31:18.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010235369999999999 2024-09-17T21:31:18.446 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.446 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.446 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.446 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50855 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 14417920~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:18.447 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:18.447 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.898858125999993, 2024-09-17T21:31:18.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043254325000000003, 2024-09-17T21:31:18.447 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.447 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.447 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.447 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50855 2024-09-17T21:31:18.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.447 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:18.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:18.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:18.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:31:18.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495374+0000", 2024-09-17T21:31:18.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.922e-06 2024-09-17T21:31:18.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495379+0000", 2024-09-17T21:31:18.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-17T21:31:18.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495384+0000", 2024-09-17T21:31:18.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-17T21:31:18.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538167+0000", 2024-09-17T21:31:18.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042782845 2024-09-17T21:31:18.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538218+0000", 2024-09-17T21:31:18.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1335000000000002e-05 2024-09-17T21:31:18.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538626+0000", 2024-09-17T21:31:18.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000407656 2024-09-17T21:31:18.450 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.450 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.450 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.451 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50856 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 18087936~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:18.451 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:18.451 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.898839671000005, 2024-09-17T21:31:18.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043911444000000001, 2024-09-17T21:31:18.451 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.451 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.451 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.451 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.451 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.451 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.451 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50856 2024-09-17T21:31:18.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.451 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:18.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:18.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:18.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0499999999999999e-07 2024-09-17T21:31:18.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495392+0000", 2024-09-17T21:31:18.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7430000000000001e-06 2024-09-17T21:31:18.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495398+0000", 2024-09-17T21:31:18.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3040000000000001e-06 2024-09-17T21:31:18.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495403+0000", 2024-09-17T21:31:18.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5600000000000004e-06 2024-09-17T21:31:18.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538660+0000", 2024-09-17T21:31:18.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043256832000000002 2024-09-17T21:31:18.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538746+0000", 2024-09-17T21:31:18.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6448999999999999e-05 2024-09-17T21:31:18.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539301+0000", 2024-09-17T21:31:18.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055525099999999999 2024-09-17T21:31:18.454 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.455 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.455 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.455 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50857 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19660800~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:18.455 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:18.455 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.898820266000001, 2024-09-17T21:31:18.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044408156999999997, 2024-09-17T21:31:18.455 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.455 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.455 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.455 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.455 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.455 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.456 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50857 2024-09-17T21:31:18.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.456 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:18.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:18.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495410+0000", 2024-09-17T21:31:18.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:31:18.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495412+0000", 2024-09-17T21:31:18.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9580000000000001e-06 2024-09-17T21:31:18.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495418+0000", 2024-09-17T21:31:18.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1909999999999997e-06 2024-09-17T21:31:18.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495423+0000", 2024-09-17T21:31:18.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2429999999999996e-06 2024-09-17T21:31:18.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539338+0000", 2024-09-17T21:31:18.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043914478 2024-09-17T21:31:18.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539393+0000", 2024-09-17T21:31:18.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5226999999999998e-05 2024-09-17T21:31:18.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539818+0000", 2024-09-17T21:31:18.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042469300000000002 2024-09-17T21:31:18.459 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.459 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.459 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.459 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50858 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23330816~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:18.459 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:18.459 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.898799169, 2024-09-17T21:31:18.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045010798999999997, 2024-09-17T21:31:18.459 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.459 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.459 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.459 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.459 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.460 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.460 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50858 2024-09-17T21:31:18.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.460 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:18.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:18.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:18.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:31:18.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495433+0000", 2024-09-17T21:31:18.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.787e-06 2024-09-17T21:31:18.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495438+0000", 2024-09-17T21:31:18.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:31:18.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495442+0000", 2024-09-17T21:31:18.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2640000000000002e-06 2024-09-17T21:31:18.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539852+0000", 2024-09-17T21:31:18.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044409592999999997 2024-09-17T21:31:18.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539940+0000", 2024-09-17T21:31:18.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7696999999999997e-05 2024-09-17T21:31:18.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540441+0000", 2024-09-17T21:31:18.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050164299999999997 2024-09-17T21:31:18.463 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.463 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.463 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.463 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50859 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24903680~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:18.463 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:18.463 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.898781490999994, 2024-09-17T21:31:18.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045571569999999999, 2024-09-17T21:31:18.463 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.463 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.464 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.464 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.464 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.464 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.464 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50859 2024-09-17T21:31:18.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.464 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:18.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:18.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495449+0000", 2024-09-17T21:31:18.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:18.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495450+0000", 2024-09-17T21:31:18.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7889999999999999e-06 2024-09-17T21:31:18.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495456+0000", 2024-09-17T21:31:18.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5679999999999999e-06 2024-09-17T21:31:18.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495460+0000", 2024-09-17T21:31:18.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4989999999999996e-06 2024-09-17T21:31:18.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540472+0000", 2024-09-17T21:31:18.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045011585999999999 2024-09-17T21:31:18.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540554+0000", 2024-09-17T21:31:18.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2443999999999999e-05 2024-09-17T21:31:18.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541020+0000", 2024-09-17T21:31:18.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00046534300000000001 2024-09-17T21:31:18.467 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.467 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.467 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.467 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50860 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28573696~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:18.468 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:18.468 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.898763138000007, 2024-09-17T21:31:18.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046167899999999998, 2024-09-17T21:31:18.468 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.468 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.468 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.468 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.468 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.468 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.468 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50860 2024-09-17T21:31:18.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.468 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:18.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:18.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:18.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:31:18.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495469+0000", 2024-09-17T21:31:18.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.99e-06 2024-09-17T21:31:18.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495475+0000", 2024-09-17T21:31:18.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:31:18.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495480+0000", 2024-09-17T21:31:18.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:31:18.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541050+0000", 2024-09-17T21:31:18.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045569853 2024-09-17T21:31:18.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541138+0000", 2024-09-17T21:31:18.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7997000000000005e-05 2024-09-17T21:31:18.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541634+0000", 2024-09-17T21:31:18.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00049685500000000004 2024-09-17T21:31:18.471 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.471 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.471 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.472 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50861 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 30146560~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:18.472 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:18.472 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.898743960999994, 2024-09-17T21:31:18.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046690553000000003, 2024-09-17T21:31:18.472 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.472 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.472 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.472 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.472 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.472 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.472 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50861 2024-09-17T21:31:18.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.472 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:18.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:18.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:18.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:31:18.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495488+0000", 2024-09-17T21:31:18.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:31:18.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495495+0000", 2024-09-17T21:31:18.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:31:18.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495500+0000", 2024-09-17T21:31:18.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8620000000000002e-06 2024-09-17T21:31:18.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541687+0000", 2024-09-17T21:31:18.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046186434999999998 2024-09-17T21:31:18.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541741+0000", 2024-09-17T21:31:18.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4228999999999999e-05 2024-09-17T21:31:18.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.542176+0000", 2024-09-17T21:31:18.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043531100000000002 2024-09-17T21:31:18.475 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.475 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.475 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.476 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97804 3.7 3:e725aa05:::rbd_data.1120c318df8c.0000000000000003:24 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:18.476 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:18.476 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.432797860000001, 2024-09-17T21:31:18.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035514416, 2024-09-17T21:31:18.476 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:18.476 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:18.476 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:18.476 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:18.476 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:18.476 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:18.476 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97804 2024-09-17T21:31:18.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.476 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:18.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:18.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:18.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:18.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:18.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:18.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:18.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:18.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:31:18.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:18.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961434+0000", 2024-09-17T21:31:18.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8959999999999999e-06 2024-09-17T21:31:18.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:18.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961439+0000", 2024-09-17T21:31:18.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T21:31:18.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:18.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961444+0000", 2024-09-17T21:31:18.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4970000000000003e-06 2024-09-17T21:31:18.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:18.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996466+0000", 2024-09-17T21:31:18.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035022438000000003 2024-09-17T21:31:18.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:18.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996556+0000", 2024-09-17T21:31:18.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9561000000000005e-05 2024-09-17T21:31:18.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:18.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:18.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:18.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996946+0000", 2024-09-17T21:31:18.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039028700000000001 2024-09-17T21:31:18.482 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.482 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.483 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.483 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:18.483 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:18.483 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:18.484 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:31:18.484 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:18.485 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:18.485 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:18.485 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:18.485 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:18.482+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:18.495 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:18.495 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:18.508 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:31:18.516 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:18.516 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:18.536 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:18.536 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:18.557 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:18.557 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:18.576 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:18.576 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:18.585 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:18.582+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:18.593 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:18.593 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:18.606 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:18.606 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:18.616 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:31:18.686 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:18.684+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:18.718 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:18.718 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:18.727 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:31:18.812 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:18.809+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:18.870 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:18.871 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:18.879 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:31:18.921 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:18.918+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:18.990 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:18.987+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:19.023 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:19.023 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:19.033 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:31:19.090 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:19.088+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:19.136 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:19.136 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:19.144 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:31:19.191 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:19.189+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:19.248 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:19.248 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:19.257 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:31:19.300 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:19.298+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:19.359 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:19.359 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:19.368 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:31:19.410 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:19.408+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:19.479 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is set 2024-09-17T21:31:19.494 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:19.491+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:19.517 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:19.517 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:19.526 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:31:19.594 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:19.592+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:19.635 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:19.636 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:19.644 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:31:19.696 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:19.693+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:19.747 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:19.747 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:19.755 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:31:19.796 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:19.794+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:19.859 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:19.859 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:19.868 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:31:19.897 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:19.894+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:19.970 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:19.970 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:19.979 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:31:20.023 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:20.021+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:20.091 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:20.091 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:20.099 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:31:20.100 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:20.097+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:20.201 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:20.199+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:20.243 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:20.244 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:20.252 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:31:20.302 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:20.300+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:20.356 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:20.356 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:20.365 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:31:20.409 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:20.407+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:20.469 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:20.469 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:20.478 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:31:20.503 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:20.500+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:20.603 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:20.601+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:20.606 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:31:20.633 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:20.634 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:20.643 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:31:20.727 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:20.725+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:20.729 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:20.729 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:20.746 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:20.746 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:20.763 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:20.763 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:20.779 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:20.779 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:20.791 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:20.791 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:20.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:20.791 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:20.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.791 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:37817 3.17 3:eab85a8d:::rbd_header.1120e62eea89:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:31:20.791 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:31:20.791 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 79.093060433000005, 2024-09-17T21:31:20.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86635970399999995, 2024-09-17T21:31:20.792 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.792 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.792 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:20.792 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.792 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.792 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.792 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37817 2024-09-17T21:31:20.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.792 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:31:20.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:31:20.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695457+0000", 2024-09-17T21:31:20.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5799999999999998e-07 2024-09-17T21:31:20.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695461+0000", 2024-09-17T21:31:20.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1110000000000001e-06 2024-09-17T21:31:20.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695471+0000", 2024-09-17T21:31:20.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0441e-05 2024-09-17T21:31:20.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695477+0000", 2024-09-17T21:31:20.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7599999999999999e-06 2024-09-17T21:31:20.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695533+0000", 2024-09-17T21:31:20.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6570999999999998e-05 2024-09-17T21:31:20.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:31:20.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695543+0000", 2024-09-17T21:31:20.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9979999999999998e-06 2024-09-17T21:31:20.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561472+0000", 2024-09-17T21:31:20.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86592841899999995 2024-09-17T21:31:20.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561610+0000", 2024-09-17T21:31:20.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013854699999999999 2024-09-17T21:31:20.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561816+0000", 2024-09-17T21:31:20.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020509900000000001 2024-09-17T21:31:20.796 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.796 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.796 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.797 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50202 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.797 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:31:20.797 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.680889643, 2024-09-17T21:31:20.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.076620261999999995, 2024-09-17T21:31:20.797 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.797 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.797 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:20.797 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.797 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.797 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.797 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50202 2024-09-17T21:31:20.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.798 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:31:20.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:31:20.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107629+0000", 2024-09-17T21:31:20.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3709999999999998e-06 2024-09-17T21:31:20.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119765+0000", 2024-09-17T21:31:20.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012135683 2024-09-17T21:31:20.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119787+0000", 2024-09-17T21:31:20.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2748999999999999e-05 2024-09-17T21:31:20.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119797+0000", 2024-09-17T21:31:20.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8169999999999994e-06 2024-09-17T21:31:20.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119872+0000", 2024-09-17T21:31:20.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270999999999994e-05 2024-09-17T21:31:20.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119987+0000", 2024-09-17T21:31:20.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011557699999999999 2024-09-17T21:31:20.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:20.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.121766+0000", 2024-09-17T21:31:20.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00177853 2024-09-17T21:31:20.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:20.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178307+0000", 2024-09-17T21:31:20.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056541559999999998 2024-09-17T21:31:20.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:20.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184097+0000", 2024-09-17T21:31:20.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057899029999999999 2024-09-17T21:31:20.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:20.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184172+0000", 2024-09-17T21:31:20.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5226000000000001e-05 2024-09-17T21:31:20.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184247+0000", 2024-09-17T21:31:20.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4574999999999997e-05 2024-09-17T21:31:20.802 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.802 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.802 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.803 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50203 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [truncate 20971520] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.803 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:31:20.803 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.668689737999998, 2024-09-17T21:31:20.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065009779000000004, 2024-09-17T21:31:20.803 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.803 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.803 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:20.803 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.803 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.803 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.803 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50203 2024-09-17T21:31:20.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.803 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:31:20.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:31:20.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119828+0000", 2024-09-17T21:31:20.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6390000000000001e-06 2024-09-17T21:31:20.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119832+0000", 2024-09-17T21:31:20.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7699999999999999e-06 2024-09-17T21:31:20.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119845+0000", 2024-09-17T21:31:20.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3093e-05 2024-09-17T21:31:20.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119852+0000", 2024-09-17T21:31:20.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6950000000000003e-06 2024-09-17T21:31:20.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.160945+0000", 2024-09-17T21:31:20.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041093076999999999 2024-09-17T21:31:20.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161071+0000", 2024-09-17T21:31:20.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012569799999999999 2024-09-17T21:31:20.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:20.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161283+0000", 2024-09-17T21:31:20.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021201999999999999 2024-09-17T21:31:20.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:20.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.179360+0000", 2024-09-17T21:31:20.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018077784999999999 2024-09-17T21:31:20.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:20.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184726+0000", 2024-09-17T21:31:20.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053652960000000003 2024-09-17T21:31:20.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:20.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184787+0000", 2024-09-17T21:31:20.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1402000000000006e-05 2024-09-17T21:31:20.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184836+0000", 2024-09-17T21:31:20.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9304000000000001e-05 2024-09-17T21:31:20.808 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.808 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.808 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.809 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50206 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.809 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:31:20.809 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.631608197999995, 2024-09-17T21:31:20.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.090361968000000001, 2024-09-17T21:31:20.809 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.809 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.809 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:20.809 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.809 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.809 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.809 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50206 2024-09-17T21:31:20.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.809 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:31:20.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:31:20.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156911+0000", 2024-09-17T21:31:20.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-17T21:31:20.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173670+0000", 2024-09-17T21:31:20.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016758907 2024-09-17T21:31:20.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173696+0000", 2024-09-17T21:31:20.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6329000000000002e-05 2024-09-17T21:31:20.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173708+0000", 2024-09-17T21:31:20.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2512e-05 2024-09-17T21:31:20.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173788+0000", 2024-09-17T21:31:20.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9332999999999994e-05 2024-09-17T21:31:20.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173918+0000", 2024-09-17T21:31:20.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013063300000000001 2024-09-17T21:31:20.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:20.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.176228+0000", 2024-09-17T21:31:20.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002309757 2024-09-17T21:31:20.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:20.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.245164+0000", 2024-09-17T21:31:20.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068936301000000005 2024-09-17T21:31:20.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:20.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247176+0000", 2024-09-17T21:31:20.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020117590000000001 2024-09-17T21:31:20.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:20.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247215+0000", 2024-09-17T21:31:20.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9094000000000002e-05 2024-09-17T21:31:20.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247270+0000", 2024-09-17T21:31:20.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4793999999999997e-05 2024-09-17T21:31:20.814 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.814 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.814 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.814 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50207 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [truncate 21556327] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.814 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:31:20.815 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.614759218000003, 2024-09-17T21:31:20.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.074055363999999999, 2024-09-17T21:31:20.815 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.815 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.815 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:20.815 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.815 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.815 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.815 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50207 2024-09-17T21:31:20.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.815 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:31:20.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:31:20.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173759+0000", 2024-09-17T21:31:20.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8160000000000001e-06 2024-09-17T21:31:20.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173763+0000", 2024-09-17T21:31:20.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5360000000000003e-06 2024-09-17T21:31:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173771+0000", 2024-09-17T21:31:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8849999999999999e-06 2024-09-17T21:31:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173777+0000", 2024-09-17T21:31:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7799999999999997e-06 2024-09-17T21:31:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221363+0000", 2024-09-17T21:31:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047585493 2024-09-17T21:31:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221430+0000", 2024-09-17T21:31:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7129000000000001e-05 2024-09-17T21:31:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221550+0000", 2024-09-17T21:31:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120658 2024-09-17T21:31:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246113+0000", 2024-09-17T21:31:20.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02456262 2024-09-17T21:31:20.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:20.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247744+0000", 2024-09-17T21:31:20.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001630692 2024-09-17T21:31:20.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:20.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247777+0000", 2024-09-17T21:31:20.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3015000000000002e-05 2024-09-17T21:31:20.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247812+0000", 2024-09-17T21:31:20.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.574e-05 2024-09-17T21:31:20.820 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.820 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.820 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.820 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50828 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.820 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:20.820 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.306647286, 2024-09-17T21:31:20.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047324185999999997, 2024-09-17T21:31:20.820 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.820 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.820 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:20.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.821 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50828 2024-09-17T21:31:20.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.821 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:20.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:20.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:20.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.46e-07 2024-09-17T21:31:20.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481871+0000", 2024-09-17T21:31:20.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T21:31:20.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481878+0000", 2024-09-17T21:31:20.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:31:20.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481882+0000", 2024-09-17T21:31:20.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:31:20.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529038+0000", 2024-09-17T21:31:20.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047156034999999999 2024-09-17T21:31:20.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529099+0000", 2024-09-17T21:31:20.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0316000000000003e-05 2024-09-17T21:31:20.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529193+0000", 2024-09-17T21:31:20.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4369000000000004e-05 2024-09-17T21:31:20.824 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.824 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.824 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.824 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50829 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 12845056~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.824 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:20.824 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.306628492000002, 2024-09-17T21:31:20.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047887596999999997, 2024-09-17T21:31:20.824 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.824 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.825 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:20.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.825 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50829 2024-09-17T21:31:20.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.825 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:31:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481890+0000", 2024-09-17T21:31:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.877e-06 2024-09-17T21:31:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481895+0000", 2024-09-17T21:31:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:31:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481900+0000", 2024-09-17T21:31:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4440000000000001e-06 2024-09-17T21:31:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529225+0000", 2024-09-17T21:31:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047325117999999999 2024-09-17T21:31:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529292+0000", 2024-09-17T21:31:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7100000000000005e-05 2024-09-17T21:31:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529775+0000", 2024-09-17T21:31:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048322799999999999 2024-09-17T21:31:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50830 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:31:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.306610090000007, 2024-09-17T21:31:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048062123999999998, 2024-09-17T21:31:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50830 2024-09-17T21:31:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:31:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:31:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481907+0000", 2024-09-17T21:31:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:31:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481909+0000", 2024-09-17T21:31:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T21:31:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481915+0000", 2024-09-17T21:31:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.02e-06 2024-09-17T21:31:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481920+0000", 2024-09-17T21:31:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6799999999999998e-06 2024-09-17T21:31:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529793+0000", 2024-09-17T21:31:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047873042999999997 2024-09-17T21:31:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529870+0000", 2024-09-17T21:31:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6967999999999999e-05 2024-09-17T21:31:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529968+0000", 2024-09-17T21:31:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8085999999999996e-05 2024-09-17T21:31:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50831 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 18087936~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.306589743999993, 2024-09-17T21:31:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048663304999999997, 2024-09-17T21:31:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50831 2024-09-17T21:31:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:31:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481929+0000", 2024-09-17T21:31:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2189999999999998e-06 2024-09-17T21:31:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481935+0000", 2024-09-17T21:31:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.383e-06 2024-09-17T21:31:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481939+0000", 2024-09-17T21:31:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2479999999999998e-06 2024-09-17T21:31:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530095+0000", 2024-09-17T21:31:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048156617999999998 2024-09-17T21:31:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530159+0000", 2024-09-17T21:31:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3394999999999999e-05 2024-09-17T21:31:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530590+0000", 2024-09-17T21:31:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043101199999999999 2024-09-17T21:31:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50832 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 19660800~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:31:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.306571097000003, 2024-09-17T21:31:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049339466999999998, 2024-09-17T21:31:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50832 2024-09-17T21:31:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:31:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:31:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481946+0000", 2024-09-17T21:31:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481951+0000", 2024-09-17T21:31:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1479999999999997e-06 2024-09-17T21:31:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481956+0000", 2024-09-17T21:31:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7339999999999996e-06 2024-09-17T21:31:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481961+0000", 2024-09-17T21:31:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4889999999999997e-06 2024-09-17T21:31:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530630+0000", 2024-09-17T21:31:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048668596000000001 2024-09-17T21:31:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530720+0000", 2024-09-17T21:31:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0163999999999999e-05 2024-09-17T21:31:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531285+0000", 2024-09-17T21:31:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056493199999999998 2024-09-17T21:31:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50833 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:31:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.306548953000004, 2024-09-17T21:31:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049908069999999999, 2024-09-17T21:31:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50833 2024-09-17T21:31:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:31:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:31:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481968+0000", 2024-09-17T21:31:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5600000000000001e-07 2024-09-17T21:31:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481970+0000", 2024-09-17T21:31:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:31:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481975+0000", 2024-09-17T21:31:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8499999999999999e-06 2024-09-17T21:31:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481980+0000", 2024-09-17T21:31:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-17T21:31:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531337+0000", 2024-09-17T21:31:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049356874000000002 2024-09-17T21:31:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531392+0000", 2024-09-17T21:31:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4826e-05 2024-09-17T21:31:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531875+0000", 2024-09-17T21:31:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048345099999999998 2024-09-17T21:31:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50834 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 24903680~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:31:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.306524185000001, 2024-09-17T21:31:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050534015000000002, 2024-09-17T21:31:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50834 2024-09-17T21:31:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:31:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:31:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481993+0000", 2024-09-17T21:31:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:31:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481995+0000", 2024-09-17T21:31:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.199e-06 2024-09-17T21:31:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482001+0000", 2024-09-17T21:31:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.772e-06 2024-09-17T21:31:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482006+0000", 2024-09-17T21:31:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0660000000000003e-06 2024-09-17T21:31:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531923+0000", 2024-09-17T21:31:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049917200000000002 2024-09-17T21:31:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532004+0000", 2024-09-17T21:31:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0758000000000002e-05 2024-09-17T21:31:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532526+0000", 2024-09-17T21:31:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00052258899999999995 2024-09-17T21:31:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50835 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 28573696~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:31:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.306504201999999, 2024-09-17T21:31:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051204701999999998, 2024-09-17T21:31:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50835 2024-09-17T21:31:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:31:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:31:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482013+0000", 2024-09-17T21:31:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:31:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482015+0000", 2024-09-17T21:31:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.272e-06 2024-09-17T21:31:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482020+0000", 2024-09-17T21:31:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3040000000000004e-06 2024-09-17T21:31:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482025+0000", 2024-09-17T21:31:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5650000000000003e-06 2024-09-17T21:31:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532569+0000", 2024-09-17T21:31:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050544661999999997 2024-09-17T21:31:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532649+0000", 2024-09-17T21:31:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9640000000000003e-05 2024-09-17T21:31:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533217+0000", 2024-09-17T21:31:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056784100000000003 2024-09-17T21:31:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50836 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 30146560~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.306484585000007, 2024-09-17T21:31:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051875589, 2024-09-17T21:31:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50836 2024-09-17T21:31:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3700000000000001e-07 2024-09-17T21:31:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482034+0000", 2024-09-17T21:31:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0059999999999999e-06 2024-09-17T21:31:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482040+0000", 2024-09-17T21:31:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:31:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482044+0000", 2024-09-17T21:31:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:31:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533260+0000", 2024-09-17T21:31:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051215416 2024-09-17T21:31:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533348+0000", 2024-09-17T21:31:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8695000000000004e-05 2024-09-17T21:31:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533907+0000", 2024-09-17T21:31:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055902199999999995 2024-09-17T21:31:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50844 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:31:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.306321159000007, 2024-09-17T21:31:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0475644, 2024-09-17T21:31:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50844 2024-09-17T21:31:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:31:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:31:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482196+0000", 2024-09-17T21:31:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.39e-07 2024-09-17T21:31:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482198+0000", 2024-09-17T21:31:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0099999999999998e-06 2024-09-17T21:31:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482204+0000", 2024-09-17T21:31:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.93e-06 2024-09-17T21:31:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482208+0000", 2024-09-17T21:31:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7439999999999998e-06 2024-09-17T21:31:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529318+0000", 2024-09-17T21:31:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047110051999999999 2024-09-17T21:31:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529396+0000", 2024-09-17T21:31:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7712000000000004e-05 2024-09-17T21:31:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529760+0000", 2024-09-17T21:31:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00036351299999999999 2024-09-17T21:31:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50845 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 21757952~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:31:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.306302056000007, 2024-09-17T21:31:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048084625999999998, 2024-09-17T21:31:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50845 2024-09-17T21:31:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:31:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:31:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482215+0000", 2024-09-17T21:31:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:31:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482216+0000", 2024-09-17T21:31:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.823e-06 2024-09-17T21:31:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482222+0000", 2024-09-17T21:31:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.507e-06 2024-09-17T21:31:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482227+0000", 2024-09-17T21:31:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6609999999999999e-06 2024-09-17T21:31:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529792+0000", 2024-09-17T21:31:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047565075999999998 2024-09-17T21:31:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529871+0000", 2024-09-17T21:31:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8855000000000003e-05 2024-09-17T21:31:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530299+0000", 2024-09-17T21:31:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000428307 2024-09-17T21:31:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50846 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 25427968~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.306279669000006, 2024-09-17T21:31:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048735142000000002, 2024-09-17T21:31:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50846 2024-09-17T21:31:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:31:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482239+0000", 2024-09-17T21:31:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.871e-06 2024-09-17T21:31:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482244+0000", 2024-09-17T21:31:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.49e-06 2024-09-17T21:31:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482249+0000", 2024-09-17T21:31:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6269999999999999e-06 2024-09-17T21:31:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530347+0000", 2024-09-17T21:31:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048097721000000003 2024-09-17T21:31:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530437+0000", 2024-09-17T21:31:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0604000000000002e-05 2024-09-17T21:31:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530972+0000", 2024-09-17T21:31:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053445000000000001 2024-09-17T21:31:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50847 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 27000832~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.306261773000003, 2024-09-17T21:31:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049308668, 2024-09-17T21:31:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50847 2024-09-17T21:31:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:31:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482257+0000", 2024-09-17T21:31:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.846e-06 2024-09-17T21:31:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482264+0000", 2024-09-17T21:31:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8170000000000001e-06 2024-09-17T21:31:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482283+0000", 2024-09-17T21:31:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9718e-05 2024-09-17T21:31:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531018+0000", 2024-09-17T21:31:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048734497000000002 2024-09-17T21:31:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531080+0000", 2024-09-17T21:31:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1795e-05 2024-09-17T21:31:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531563+0000", 2024-09-17T21:31:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048366499999999998 2024-09-17T21:31:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50848 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 30670848~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:31:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.306225230999999, 2024-09-17T21:31:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049893337000000003, 2024-09-17T21:31:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50848 2024-09-17T21:31:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:31:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:31:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482292+0000", 2024-09-17T21:31:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T21:31:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482294+0000", 2024-09-17T21:31:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3209999999999999e-06 2024-09-17T21:31:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482300+0000", 2024-09-17T21:31:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7189999999999998e-06 2024-09-17T21:31:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482305+0000", 2024-09-17T21:31:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T21:31:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531606+0000", 2024-09-17T21:31:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049301273999999999 2024-09-17T21:31:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531696+0000", 2024-09-17T21:31:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0161999999999995e-05 2024-09-17T21:31:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532184+0000", 2024-09-17T21:31:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048806400000000002 2024-09-17T21:31:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50849 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 32243712~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:31:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 69.306205790000007, 2024-09-17T21:31:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050444607000000002, 2024-09-17T21:31:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50849 2024-09-17T21:31:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:31:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:31:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482311+0000", 2024-09-17T21:31:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482316+0000", 2024-09-17T21:31:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1209999999999998e-06 2024-09-17T21:31:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482321+0000", 2024-09-17T21:31:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4779999999999999e-06 2024-09-17T21:31:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482326+0000", 2024-09-17T21:31:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7099999999999998e-06 2024-09-17T21:31:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532221+0000", 2024-09-17T21:31:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049894936000000001 2024-09-17T21:31:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532287+0000", 2024-09-17T21:31:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5914000000000006e-05 2024-09-17T21:31:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532755+0000", 2024-09-17T21:31:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000468107 2024-09-17T21:31:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.885 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.885 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:20.885 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:20.885 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:20.885 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-17T21:31:20.885 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:31:20.885 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T21:31:20.885 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.496679 2024-09-17T21:31:20.886 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0.49667854644124154 2024-09-17T21:31:20.886 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:20.886 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:20.887 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:31:20.888 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:20.888 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:20.888 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:20.888 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:20.889 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:20.889 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:20.890 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:20.883+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:20.987 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:20.985+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:21.034 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:21.034 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:21.034 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:21.034 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:21.044 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:31:21.087 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:21.086+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:21.150 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:21.150 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:21.150 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:21.150 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:21.150 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:21.160 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:31:21.204 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:21.202+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:21.263 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:21.263 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:21.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:21.264 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:21.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.264 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "pg_update_log_missing(3.5 epoch 45/37 rep_tid 844 entries 45'73 (0'0) error 3:a65c40ae:::rbd_data.1120b21c3309.0000000000000126:head by client.4384.0:48900 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 45'70)", 2024-09-17T21:31:21.264 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:31:21.264 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 74.213798109999999, 2024-09-17T21:31:21.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019554365000000001, 2024-09-17T21:31:21.264 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.264 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.264 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T21:31:21.264 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:31:21.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:31:21.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047112+0000", 2024-09-17T21:31:21.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:31:21.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047121+0000", 2024-09-17T21:31:21.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4379999999999995e-06 2024-09-17T21:31:21.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047139+0000", 2024-09-17T21:31:21.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8235000000000001e-05 2024-09-17T21:31:21.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047303+0000", 2024-09-17T21:31:21.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016389400000000001 2024-09-17T21:31:21.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047396+0000", 2024-09-17T21:31:21.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3355000000000003e-05 2024-09-17T21:31:21.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066666+0000", 2024-09-17T21:31:21.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019269649999999999 2024-09-17T21:31:21.267 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.267 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.267 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.268 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48906 3.4 e45/25 3:2503fe27:::rbd_data.1120b21c3309.0000000000000127:head v 45'148, mlcod=45'146)", 2024-09-17T21:31:21.268 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:21.268 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 74.211671930999998, 2024-09-17T21:31:21.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018169739000000001, 2024-09-17T21:31:21.268 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.268 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.268 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:21.268 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:21.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:21.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:21.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:31:21.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049244+0000", 2024-09-17T21:31:21.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8390000000000003e-06 2024-09-17T21:31:21.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066411+0000", 2024-09-17T21:31:21.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017166818 2024-09-17T21:31:21.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066426+0000", 2024-09-17T21:31:21.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4888999999999999e-05 2024-09-17T21:31:21.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066611+0000", 2024-09-17T21:31:21.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018464499999999999 2024-09-17T21:31:21.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066630+0000", 2024-09-17T21:31:21.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9201000000000001e-05 2024-09-17T21:31:21.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:21.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067373+0000", 2024-09-17T21:31:21.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00074322600000000004 2024-09-17T21:31:21.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067407+0000", 2024-09-17T21:31:21.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4357999999999999e-05 2024-09-17T21:31:21.272 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.272 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.272 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.272 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50206 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'301, mlcod=45'299)", 2024-09-17T21:31:21.272 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:31:21.272 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.082195087000002, 2024-09-17T21:31:21.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068285810000000002, 2024-09-17T21:31:21.272 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.273 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.273 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:21.273 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:31:21.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:31:21.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178716+0000", 2024-09-17T21:31:21.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.438e-06 2024-09-17T21:31:21.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192044+0000", 2024-09-17T21:31:21.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013328370000000001 2024-09-17T21:31:21.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192069+0000", 2024-09-17T21:31:21.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4496000000000001e-05 2024-09-17T21:31:21.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192085+0000", 2024-09-17T21:31:21.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5758999999999999e-05 2024-09-17T21:31:21.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192167+0000", 2024-09-17T21:31:21.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2842000000000004e-05 2024-09-17T21:31:21.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192201+0000", 2024-09-17T21:31:21.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3127999999999999e-05 2024-09-17T21:31:21.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:21.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246938+0000", 2024-09-17T21:31:21.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054736982000000003 2024-09-17T21:31:21.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247000+0000", 2024-09-17T21:31:21.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2794999999999998e-05 2024-09-17T21:31:21.276 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.276 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.277 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.277 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50207 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'303, mlcod=45'299)", 2024-09-17T21:31:21.277 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:31:21.277 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.039248037999997, 2024-09-17T21:31:21.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026003433999999999, 2024-09-17T21:31:21.277 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.277 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.277 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:21.277 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:31:21.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:31:21.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221663+0000", 2024-09-17T21:31:21.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.271e-06 2024-09-17T21:31:21.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221672+0000", 2024-09-17T21:31:21.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6810000000000004e-06 2024-09-17T21:31:21.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221684+0000", 2024-09-17T21:31:21.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:31:21.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221691+0000", 2024-09-17T21:31:21.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:31:21.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221743+0000", 2024-09-17T21:31:21.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2868999999999998e-05 2024-09-17T21:31:21.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221765+0000", 2024-09-17T21:31:21.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1316999999999999e-05 2024-09-17T21:31:21.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:21.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247616+0000", 2024-09-17T21:31:21.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025851504000000001 2024-09-17T21:31:21.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247665+0000", 2024-09-17T21:31:21.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8739000000000002e-05 2024-09-17T21:31:21.281 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.282 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.282 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.282 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57808 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 196608~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:21.282 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:21.282 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.482948997999998, 2024-09-17T21:31:21.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017751475999999999, 2024-09-17T21:31:21.282 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.282 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.282 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.282 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.282 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.283 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57808 2024-09-17T21:31:21.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.283 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:21.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:21.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:21.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:31:21.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777968+0000", 2024-09-17T21:31:21.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8530000000000003e-06 2024-09-17T21:31:21.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777975+0000", 2024-09-17T21:31:21.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4410000000000004e-06 2024-09-17T21:31:21.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777980+0000", 2024-09-17T21:31:21.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7210000000000002e-06 2024-09-17T21:31:21.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795323+0000", 2024-09-17T21:31:21.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017343319999999999 2024-09-17T21:31:21.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795410+0000", 2024-09-17T21:31:21.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6567e-05 2024-09-17T21:31:21.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795712+0000", 2024-09-17T21:31:21.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030219200000000001 2024-09-17T21:31:21.286 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.286 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.286 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.286 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57809 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 458752~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:21.286 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:31:21.286 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.482923145999997, 2024-09-17T21:31:21.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018092869000000001, 2024-09-17T21:31:21.287 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.287 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.287 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.287 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57809 2024-09-17T21:31:21.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.287 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:31:21.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:31:21.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777987+0000", 2024-09-17T21:31:21.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T21:31:21.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777989+0000", 2024-09-17T21:31:21.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8649999999999999e-06 2024-09-17T21:31:21.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777994+0000", 2024-09-17T21:31:21.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4199999999999998e-06 2024-09-17T21:31:21.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778004+0000", 2024-09-17T21:31:21.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6900000000000004e-06 2024-09-17T21:31:21.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795745+0000", 2024-09-17T21:31:21.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017740859000000001 2024-09-17T21:31:21.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795815+0000", 2024-09-17T21:31:21.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0219e-05 2024-09-17T21:31:21.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796079+0000", 2024-09-17T21:31:21.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000264446 2024-09-17T21:31:21.290 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.290 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.291 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.291 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57810 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 720896~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:21.291 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:31:21.291 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.482899418999999, 2024-09-17T21:31:21.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018361253000000001, 2024-09-17T21:31:21.291 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.291 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.291 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.291 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.291 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.291 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.291 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57810 2024-09-17T21:31:21.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.292 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:31:21.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:31:21.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778011+0000", 2024-09-17T21:31:21.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:31:21.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778012+0000", 2024-09-17T21:31:21.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.872e-06 2024-09-17T21:31:21.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778018+0000", 2024-09-17T21:31:21.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8150000000000002e-06 2024-09-17T21:31:21.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778023+0000", 2024-09-17T21:31:21.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3939999999999998e-06 2024-09-17T21:31:21.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796106+0000", 2024-09-17T21:31:21.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018083446999999999 2024-09-17T21:31:21.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796159+0000", 2024-09-17T21:31:21.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3029999999999999e-05 2024-09-17T21:31:21.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796371+0000", 2024-09-17T21:31:21.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021231700000000001 2024-09-17T21:31:21.295 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.295 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.295 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.295 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57811 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:21.295 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:21.295 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.482880999000002, 2024-09-17T21:31:21.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018666035000000001, 2024-09-17T21:31:21.296 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.296 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.296 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.296 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.296 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.296 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.296 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57811 2024-09-17T21:31:21.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.296 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:21.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:21.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:21.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:21.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778031+0000", 2024-09-17T21:31:21.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.796e-06 2024-09-17T21:31:21.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778036+0000", 2024-09-17T21:31:21.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3290000000000001e-06 2024-09-17T21:31:21.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778041+0000", 2024-09-17T21:31:21.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5839999999999996e-06 2024-09-17T21:31:21.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796409+0000", 2024-09-17T21:31:21.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018368703 2024-09-17T21:31:21.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796483+0000", 2024-09-17T21:31:21.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3751999999999995e-05 2024-09-17T21:31:21.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796695+0000", 2024-09-17T21:31:21.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021153 2024-09-17T21:31:21.299 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.299 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.299 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.300 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57812 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1245184~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:21.300 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:21.300 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.482862101999999, 2024-09-17T21:31:21.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019000064000000001, 2024-09-17T21:31:21.300 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.300 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.300 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.300 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.300 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.300 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.300 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57812 2024-09-17T21:31:21.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.301 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:21.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:21.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:21.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:31:21.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778050+0000", 2024-09-17T21:31:21.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7e-06 2024-09-17T21:31:21.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778055+0000", 2024-09-17T21:31:21.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4049999999999999e-06 2024-09-17T21:31:21.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778059+0000", 2024-09-17T21:31:21.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1479999999999999e-06 2024-09-17T21:31:21.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796734+0000", 2024-09-17T21:31:21.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018674399000000001 2024-09-17T21:31:21.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796786+0000", 2024-09-17T21:31:21.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2879e-05 2024-09-17T21:31:21.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797048+0000", 2024-09-17T21:31:21.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026118400000000003 2024-09-17T21:31:21.304 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.304 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.304 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.304 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57813 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1507328~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:21.304 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:31:21.304 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.482844210000003, 2024-09-17T21:31:21.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019279392999999999, 2024-09-17T21:31:21.304 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.304 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.305 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.305 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57813 2024-09-17T21:31:21.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.305 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:31:21.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:31:21.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778066+0000", 2024-09-17T21:31:21.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6199999999999999e-07 2024-09-17T21:31:21.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778068+0000", 2024-09-17T21:31:21.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7990000000000001e-06 2024-09-17T21:31:21.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778074+0000", 2024-09-17T21:31:21.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:31:21.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778078+0000", 2024-09-17T21:31:21.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2579999999999997e-06 2024-09-17T21:31:21.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797077+0000", 2024-09-17T21:31:21.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018999180000000001 2024-09-17T21:31:21.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797123+0000", 2024-09-17T21:31:21.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5834999999999997e-05 2024-09-17T21:31:21.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797345+0000", 2024-09-17T21:31:21.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022191100000000001 2024-09-17T21:31:21.308 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.308 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.308 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.309 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57814 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:21.309 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:31:21.309 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.482825337999998, 2024-09-17T21:31:21.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019604947000000001, 2024-09-17T21:31:21.309 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.309 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.309 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.309 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.309 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.309 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.309 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57814 2024-09-17T21:31:21.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.309 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:31:21.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:31:21.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778085+0000", 2024-09-17T21:31:21.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:31:21.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778086+0000", 2024-09-17T21:31:21.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7850000000000001e-06 2024-09-17T21:31:21.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778093+0000", 2024-09-17T21:31:21.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6449999999999999e-06 2024-09-17T21:31:21.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778098+0000", 2024-09-17T21:31:21.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:31:21.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797383+0000", 2024-09-17T21:31:21.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019284879000000001 2024-09-17T21:31:21.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797431+0000", 2024-09-17T21:31:21.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8856000000000001e-05 2024-09-17T21:31:21.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797689+0000", 2024-09-17T21:31:21.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025785800000000002 2024-09-17T21:31:21.313 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.313 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.313 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.313 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57815 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2031616~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:21.313 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:21.313 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.48280467, 2024-09-17T21:31:21.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019910231, 2024-09-17T21:31:21.313 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.313 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.313 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.313 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.313 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.314 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.314 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57815 2024-09-17T21:31:21.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.314 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:21.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:21.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:21.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:31:21.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778118+0000", 2024-09-17T21:31:21.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2967e-05 2024-09-17T21:31:21.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778125+0000", 2024-09-17T21:31:21.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.601e-06 2024-09-17T21:31:21.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778130+0000", 2024-09-17T21:31:21.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1839999999999998e-06 2024-09-17T21:31:21.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797720+0000", 2024-09-17T21:31:21.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019589766000000002 2024-09-17T21:31:21.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797765+0000", 2024-09-17T21:31:21.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5281999999999998e-05 2024-09-17T21:31:21.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798015+0000", 2024-09-17T21:31:21.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025006200000000001 2024-09-17T21:31:21.317 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.317 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.317 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.317 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57816 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2293760~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:21.317 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:21.317 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.482772076000003, 2024-09-17T21:31:21.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020175322999999998, 2024-09-17T21:31:21.318 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.318 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.318 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.318 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.318 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.318 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.318 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57816 2024-09-17T21:31:21.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.318 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:21.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:21.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:21.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:31:21.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778140+0000", 2024-09-17T21:31:21.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T21:31:21.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778145+0000", 2024-09-17T21:31:21.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7320000000000003e-06 2024-09-17T21:31:21.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778154+0000", 2024-09-17T21:31:21.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6969999999999999e-06 2024-09-17T21:31:21.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798049+0000", 2024-09-17T21:31:21.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019894603 2024-09-17T21:31:21.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798106+0000", 2024-09-17T21:31:21.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7207000000000003e-05 2024-09-17T21:31:21.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798313+0000", 2024-09-17T21:31:21.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206861 2024-09-17T21:31:21.322 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.322 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.322 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.322 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57817 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2555904~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:21.322 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:21.322 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.482748829999998, 2024-09-17T21:31:21.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020466314999999999, 2024-09-17T21:31:21.322 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.322 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.322 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.322 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.323 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.323 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.323 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57817 2024-09-17T21:31:21.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.323 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:21.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:21.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:21.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2399999999999999e-07 2024-09-17T21:31:21.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778163+0000", 2024-09-17T21:31:21.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.204e-06 2024-09-17T21:31:21.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778169+0000", 2024-09-17T21:31:21.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:31:21.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778173+0000", 2024-09-17T21:31:21.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:31:21.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798342+0000", 2024-09-17T21:31:21.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020168416000000002 2024-09-17T21:31:21.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798388+0000", 2024-09-17T21:31:21.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6171000000000002e-05 2024-09-17T21:31:21.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798627+0000", 2024-09-17T21:31:21.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000239058 2024-09-17T21:31:21.326 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.326 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.326 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.326 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57818 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2818048~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:21.327 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:31:21.327 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.482730132999997, 2024-09-17T21:31:21.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020767240999999999, 2024-09-17T21:31:21.327 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.327 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.327 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.327 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.327 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.327 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.327 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57818 2024-09-17T21:31:21.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.327 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:31:21.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:31:21.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778180+0000", 2024-09-17T21:31:21.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:31:21.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778182+0000", 2024-09-17T21:31:21.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7269999999999999e-06 2024-09-17T21:31:21.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778188+0000", 2024-09-17T21:31:21.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6080000000000001e-06 2024-09-17T21:31:21.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778202+0000", 2024-09-17T21:31:21.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3369e-05 2024-09-17T21:31:21.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798660+0000", 2024-09-17T21:31:21.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020458171000000001 2024-09-17T21:31:21.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798704+0000", 2024-09-17T21:31:21.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4273000000000001e-05 2024-09-17T21:31:21.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798947+0000", 2024-09-17T21:31:21.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242651 2024-09-17T21:31:21.331 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.331 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.331 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.331 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57819 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:21.331 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:31:21.331 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.482699263999997, 2024-09-17T21:31:21.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020892340999999998, 2024-09-17T21:31:21.331 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.331 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.331 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.331 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.331 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.332 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.332 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57819 2024-09-17T21:31:21.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.332 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:31:21.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:31:21.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778211+0000", 2024-09-17T21:31:21.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:31:21.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778217+0000", 2024-09-17T21:31:21.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T21:31:21.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778224+0000", 2024-09-17T21:31:21.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3520000000000003e-06 2024-09-17T21:31:21.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778228+0000", 2024-09-17T21:31:21.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7480000000000001e-06 2024-09-17T21:31:21.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798977+0000", 2024-09-17T21:31:21.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020748571 2024-09-17T21:31:21.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799035+0000", 2024-09-17T21:31:21.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7908999999999997e-05 2024-09-17T21:31:21.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799103+0000", 2024-09-17T21:31:21.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7794999999999997e-05 2024-09-17T21:31:21.339 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.339 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.339 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.339 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57820 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:21.339 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:21.339 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.482674072000002, 2024-09-17T21:31:21.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021016658000000001, 2024-09-17T21:31:21.339 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.339 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.339 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.339 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.340 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.340 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.340 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57820 2024-09-17T21:31:21.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.340 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:21.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:21.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:21.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:31:21.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778238+0000", 2024-09-17T21:31:21.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.945e-06 2024-09-17T21:31:21.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778244+0000", 2024-09-17T21:31:21.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6520000000000003e-06 2024-09-17T21:31:21.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778248+0000", 2024-09-17T21:31:21.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5920000000000002e-06 2024-09-17T21:31:21.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799131+0000", 2024-09-17T21:31:21.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020883288999999999 2024-09-17T21:31:21.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799185+0000", 2024-09-17T21:31:21.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3223000000000002e-05 2024-09-17T21:31:21.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799252+0000", 2024-09-17T21:31:21.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7576999999999994e-05 2024-09-17T21:31:21.343 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.343 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.343 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.344 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57821 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:21.344 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:21.344 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.482655727000001, 2024-09-17T21:31:21.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021153528000000001, 2024-09-17T21:31:21.344 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.344 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.344 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.344 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.344 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.344 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.344 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57821 2024-09-17T21:31:21.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.344 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:21.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:21.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:21.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-07 2024-09-17T21:31:21.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778256+0000", 2024-09-17T21:31:21.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.604e-06 2024-09-17T21:31:21.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778262+0000", 2024-09-17T21:31:21.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7629999999999997e-06 2024-09-17T21:31:21.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778266+0000", 2024-09-17T21:31:21.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:31:21.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799284+0000", 2024-09-17T21:31:21.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021017312 2024-09-17T21:31:21.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799326+0000", 2024-09-17T21:31:21.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2639000000000003e-05 2024-09-17T21:31:21.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799407+0000", 2024-09-17T21:31:21.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1077999999999999e-05 2024-09-17T21:31:21.348 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.348 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.348 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.348 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57822 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:21.348 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:31:21.348 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.482637416999999, 2024-09-17T21:31:21.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021268966, 2024-09-17T21:31:21.348 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.349 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.349 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.349 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.349 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.349 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.349 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57822 2024-09-17T21:31:21.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.349 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:31:21.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:31:21.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778273+0000", 2024-09-17T21:31:21.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:31:21.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778274+0000", 2024-09-17T21:31:21.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.598e-06 2024-09-17T21:31:21.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778283+0000", 2024-09-17T21:31:21.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8279999999999992e-06 2024-09-17T21:31:21.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778289+0000", 2024-09-17T21:31:21.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5609999999999998e-06 2024-09-17T21:31:21.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799432+0000", 2024-09-17T21:31:21.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021143862999999999 2024-09-17T21:31:21.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799473+0000", 2024-09-17T21:31:21.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1075999999999998e-05 2024-09-17T21:31:21.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799541+0000", 2024-09-17T21:31:21.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7657999999999995e-05 2024-09-17T21:31:21.352 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.353 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.353 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.353 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57823 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:21.353 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:21.353 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.482614456, 2024-09-17T21:31:21.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021379770999999999, 2024-09-17T21:31:21.353 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.353 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.353 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.353 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.353 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.353 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.354 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57823 2024-09-17T21:31:21.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.354 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:21.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:21.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:21.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:31:21.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778297+0000", 2024-09-17T21:31:21.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:31:21.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778303+0000", 2024-09-17T21:31:21.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4559999999999999e-06 2024-09-17T21:31:21.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778307+0000", 2024-09-17T21:31:21.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6229999999999996e-06 2024-09-17T21:31:21.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799560+0000", 2024-09-17T21:31:21.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021252931999999999 2024-09-17T21:31:21.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799609+0000", 2024-09-17T21:31:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8301999999999999e-05 2024-09-17T21:31:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799675+0000", 2024-09-17T21:31:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6205999999999996e-05 2024-09-17T21:31:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.357 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:21.359 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:31:21.359 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:21.356+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:21.460 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:21.457+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:21.492 INFO:teuthology.orchestra.run.smithi129.stderr:set osd.2 primary-affinity to 0.496679 (8325502) 2024-09-17T21:31:21.501 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:31:21.516 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:21.516 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:21.516 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:21.516 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:21.527 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:31:21.561 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:21.558+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:21.662 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:21.659+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:21.682 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:21.683 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:21.683 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:21.683 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:21.683 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:21.692 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:31:21.763 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:21.760+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:21.799 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:21.799 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:21.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:21.799 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:21.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.799 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:49361 3.19 e46/25 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:head v 46'135, mlcod=45'133)", 2024-09-17T21:31:21.799 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:21.800 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 74.042169463999997, 2024-09-17T21:31:21.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043477744999999998, 2024-09-17T21:31:21.800 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.800 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.800 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:21.800 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:21.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:21.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:21.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-07 2024-09-17T21:31:21.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769703+0000", 2024-09-17T21:31:21.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016134381 2024-09-17T21:31:21.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769724+0000", 2024-09-17T21:31:21.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0781999999999999e-05 2024-09-17T21:31:21.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769734+0000", 2024-09-17T21:31:21.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6870000000000006e-06 2024-09-17T21:31:21.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769797+0000", 2024-09-17T21:31:21.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3418999999999997e-05 2024-09-17T21:31:21.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769822+0000", 2024-09-17T21:31:21.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4652999999999999e-05 2024-09-17T21:31:21.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:21.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.796982+0000", 2024-09-17T21:31:21.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027160205 2024-09-17T21:31:21.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.797047+0000", 2024-09-17T21:31:21.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4455999999999994e-05 2024-09-17T21:31:21.804 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.804 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.804 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.805 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50805 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 17825792~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:21.805 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:31:21.805 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.30058597, 2024-09-17T21:31:21.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035591579999999998, 2024-09-17T21:31:21.805 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.805 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.805 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.805 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.805 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.805 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.805 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50805 2024-09-17T21:31:21.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.805 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:31:21.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:31:21.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495153+0000", 2024-09-17T21:31:21.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:31:21.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495155+0000", 2024-09-17T21:31:21.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1849999999999998e-06 2024-09-17T21:31:21.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495162+0000", 2024-09-17T21:31:21.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.64e-06 2024-09-17T21:31:21.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495166+0000", 2024-09-17T21:31:21.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:31:21.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530020+0000", 2024-09-17T21:31:21.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.034853824999999998 2024-09-17T21:31:21.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530076+0000", 2024-09-17T21:31:21.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5235999999999998e-05 2024-09-17T21:31:21.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530744+0000", 2024-09-17T21:31:21.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066835500000000003 2024-09-17T21:31:21.809 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.809 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.809 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.809 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50806 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19398656~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:21.810 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:21.810 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.300565501999998, 2024-09-17T21:31:21.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036408459999999997, 2024-09-17T21:31:21.810 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.810 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.810 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.810 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.810 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.810 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.810 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50806 2024-09-17T21:31:21.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.810 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:21.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:21.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:21.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.72e-07 2024-09-17T21:31:21.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495175+0000", 2024-09-17T21:31:21.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:31:21.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495182+0000", 2024-09-17T21:31:21.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8079999999999999e-06 2024-09-17T21:31:21.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495186+0000", 2024-09-17T21:31:21.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5549999999999996e-06 2024-09-17T21:31:21.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530791+0000", 2024-09-17T21:31:21.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035604963000000003 2024-09-17T21:31:21.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530855+0000", 2024-09-17T21:31:21.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3286000000000004e-05 2024-09-17T21:31:21.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531581+0000", 2024-09-17T21:31:21.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00072655699999999996 2024-09-17T21:31:21.815 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.815 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.815 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.815 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50807 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23068672~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:21.815 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:21.815 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.300546616999995, 2024-09-17T21:31:21.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037183314000000002, 2024-09-17T21:31:21.815 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.816 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.816 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.816 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50807 2024-09-17T21:31:21.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.816 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:21.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:21.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:21.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9999999999999999e-07 2024-09-17T21:31:21.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495194+0000", 2024-09-17T21:31:21.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.779e-06 2024-09-17T21:31:21.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495199+0000", 2024-09-17T21:31:21.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.468e-06 2024-09-17T21:31:21.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495203+0000", 2024-09-17T21:31:21.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0010000000000003e-06 2024-09-17T21:31:21.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531636+0000", 2024-09-17T21:31:21.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036432665000000003 2024-09-17T21:31:21.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531727+0000", 2024-09-17T21:31:21.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1607999999999994e-05 2024-09-17T21:31:21.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532375+0000", 2024-09-17T21:31:21.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00064749299999999996 2024-09-17T21:31:21.820 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.820 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.820 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.820 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50808 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24641536~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:21.820 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:31:21.820 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.300529874000006, 2024-09-17T21:31:21.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037870190999999997, 2024-09-17T21:31:21.821 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.821 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.821 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.821 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.821 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.821 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.821 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50808 2024-09-17T21:31:21.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.821 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:31:21.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:31:21.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495209+0000", 2024-09-17T21:31:21.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:31:21.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495215+0000", 2024-09-17T21:31:21.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1630000000000001e-06 2024-09-17T21:31:21.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495221+0000", 2024-09-17T21:31:21.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-17T21:31:21.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495226+0000", 2024-09-17T21:31:21.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8189999999999999e-06 2024-09-17T21:31:21.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532425+0000", 2024-09-17T21:31:21.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037199135000000001 2024-09-17T21:31:21.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532531+0000", 2024-09-17T21:31:21.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106314 2024-09-17T21:31:21.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533079+0000", 2024-09-17T21:31:21.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00054739700000000003 2024-09-17T21:31:21.825 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.825 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.825 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.825 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50809 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28311552~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:21.825 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:31:21.825 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.300506908000003, 2024-09-17T21:31:21.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038597605, 2024-09-17T21:31:21.825 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.826 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.826 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.826 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.826 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.826 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.826 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50809 2024-09-17T21:31:21.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.826 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:31:21.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:31:21.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495232+0000", 2024-09-17T21:31:21.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.39e-07 2024-09-17T21:31:21.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495234+0000", 2024-09-17T21:31:21.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T21:31:21.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495239+0000", 2024-09-17T21:31:21.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7969999999999997e-06 2024-09-17T21:31:21.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495244+0000", 2024-09-17T21:31:21.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999996e-06 2024-09-17T21:31:21.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533125+0000", 2024-09-17T21:31:21.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037881523 2024-09-17T21:31:21.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533237+0000", 2024-09-17T21:31:21.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000111879 2024-09-17T21:31:21.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533829+0000", 2024-09-17T21:31:21.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000591933 2024-09-17T21:31:21.829 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.829 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.830 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.830 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50810 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 29884416~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:21.830 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:31:21.830 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.300488990000005, 2024-09-17T21:31:21.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039149809000000001, 2024-09-17T21:31:21.830 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.830 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.830 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.830 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.830 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.830 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.831 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50810 2024-09-17T21:31:21.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.831 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:31:21.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:31:21.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495250+0000", 2024-09-17T21:31:21.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:31:21.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495251+0000", 2024-09-17T21:31:21.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.81e-06 2024-09-17T21:31:21.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495257+0000", 2024-09-17T21:31:21.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1399999999999999e-06 2024-09-17T21:31:21.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495261+0000", 2024-09-17T21:31:21.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4229999999999998e-06 2024-09-17T21:31:21.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533879+0000", 2024-09-17T21:31:21.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038618292999999998 2024-09-17T21:31:21.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533939+0000", 2024-09-17T21:31:21.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.012e-05 2024-09-17T21:31:21.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534399+0000", 2024-09-17T21:31:21.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00045968099999999998 2024-09-17T21:31:21.834 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.834 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.834 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.835 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50850 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 2359296~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:21.835 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:21.835 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.300470614999995, 2024-09-17T21:31:21.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039731491000000001, 2024-09-17T21:31:21.835 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.835 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.835 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.835 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.835 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.835 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.835 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50850 2024-09-17T21:31:21.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.836 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:21.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:21.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:21.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3200000000000001e-07 2024-09-17T21:31:21.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495270+0000", 2024-09-17T21:31:21.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.694e-06 2024-09-17T21:31:21.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495275+0000", 2024-09-17T21:31:21.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5890000000000002e-06 2024-09-17T21:31:21.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495279+0000", 2024-09-17T21:31:21.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-17T21:31:21.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534440+0000", 2024-09-17T21:31:21.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039160805999999999 2024-09-17T21:31:21.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534526+0000", 2024-09-17T21:31:21.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5327999999999993e-05 2024-09-17T21:31:21.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534999+0000", 2024-09-17T21:31:21.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00047353400000000001 2024-09-17T21:31:21.839 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.839 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.839 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.840 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50851 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 3932160~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:21.840 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:31:21.840 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.300453003000001, 2024-09-17T21:31:21.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040211991000000002, 2024-09-17T21:31:21.840 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.840 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.840 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.840 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50851 2024-09-17T21:31:21.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.840 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:31:21.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:31:21.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495286+0000", 2024-09-17T21:31:21.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:31:21.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495287+0000", 2024-09-17T21:31:21.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6640000000000001e-06 2024-09-17T21:31:21.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495293+0000", 2024-09-17T21:31:21.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6710000000000004e-06 2024-09-17T21:31:21.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495298+0000", 2024-09-17T21:31:21.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6700000000000002e-06 2024-09-17T21:31:21.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535017+0000", 2024-09-17T21:31:21.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039719059000000001 2024-09-17T21:31:21.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535067+0000", 2024-09-17T21:31:21.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0046000000000002e-05 2024-09-17T21:31:21.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535497+0000", 2024-09-17T21:31:21.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043050299999999999 2024-09-17T21:31:21.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.844 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.844 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50852 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 7602176~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:21.844 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:21.844 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.300434085999996, 2024-09-17T21:31:21.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041166337999999997, 2024-09-17T21:31:21.845 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.845 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.845 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.845 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50852 2024-09-17T21:31:21.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.845 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:21.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:21.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495305+0000", 2024-09-17T21:31:21.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1600000000000002e-07 2024-09-17T21:31:21.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495307+0000", 2024-09-17T21:31:21.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0980000000000001e-06 2024-09-17T21:31:21.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495312+0000", 2024-09-17T21:31:21.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6210000000000001e-06 2024-09-17T21:31:21.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495317+0000", 2024-09-17T21:31:21.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T21:31:21.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535531+0000", 2024-09-17T21:31:21.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040214580999999999 2024-09-17T21:31:21.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535618+0000", 2024-09-17T21:31:21.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6648000000000001e-05 2024-09-17T21:31:21.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536471+0000", 2024-09-17T21:31:21.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00085251599999999999 2024-09-17T21:31:21.849 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.849 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.849 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.849 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50853 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 9175040~204800] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:21.849 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:21.849 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.300415755000003, 2024-09-17T21:31:21.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041673496999999997, 2024-09-17T21:31:21.849 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.849 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.849 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.850 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.850 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50853 2024-09-17T21:31:21.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.850 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:21.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:21.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:21.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:31:21.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495334+0000", 2024-09-17T21:31:21.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.076e-05 2024-09-17T21:31:21.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495340+0000", 2024-09-17T21:31:21.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:31:21.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495345+0000", 2024-09-17T21:31:21.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:31:21.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536514+0000", 2024-09-17T21:31:21.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041168534999999999 2024-09-17T21:31:21.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536572+0000", 2024-09-17T21:31:21.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8742000000000001e-05 2024-09-17T21:31:21.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536996+0000", 2024-09-17T21:31:21.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042368899999999999 2024-09-17T21:31:21.853 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.853 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.853 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.853 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50854 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 12845056~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:21.853 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:21.854 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.300386696000004, 2024-09-17T21:31:21.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042777397000000002, 2024-09-17T21:31:21.854 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.854 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.854 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.854 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50854 2024-09-17T21:31:21.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.854 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:21.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:21.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:21.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.82e-07 2024-09-17T21:31:21.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495354+0000", 2024-09-17T21:31:21.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.091e-06 2024-09-17T21:31:21.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495360+0000", 2024-09-17T21:31:21.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.716e-06 2024-09-17T21:31:21.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495365+0000", 2024-09-17T21:31:21.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2939999999999996e-06 2024-09-17T21:31:21.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537020+0000", 2024-09-17T21:31:21.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041654419999999998 2024-09-17T21:31:21.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537105+0000", 2024-09-17T21:31:21.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5857000000000001e-05 2024-09-17T21:31:21.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538129+0000", 2024-09-17T21:31:21.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010235369999999999 2024-09-17T21:31:21.857 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.858 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.858 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.858 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50855 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 14417920~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:21.858 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:21.858 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.300366730999997, 2024-09-17T21:31:21.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043254325000000003, 2024-09-17T21:31:21.858 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.858 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.858 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.859 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50855 2024-09-17T21:31:21.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.859 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:21.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:21.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:21.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:31:21.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495374+0000", 2024-09-17T21:31:21.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.922e-06 2024-09-17T21:31:21.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495379+0000", 2024-09-17T21:31:21.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-17T21:31:21.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495384+0000", 2024-09-17T21:31:21.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-17T21:31:21.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538167+0000", 2024-09-17T21:31:21.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042782845 2024-09-17T21:31:21.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538218+0000", 2024-09-17T21:31:21.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1335000000000002e-05 2024-09-17T21:31:21.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538626+0000", 2024-09-17T21:31:21.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000407656 2024-09-17T21:31:21.862 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.862 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.862 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.862 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50856 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 18087936~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:21.862 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:21.862 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.300348275999994, 2024-09-17T21:31:21.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043911444000000001, 2024-09-17T21:31:21.863 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.863 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.863 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.863 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50856 2024-09-17T21:31:21.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.863 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:21.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:21.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:21.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0499999999999999e-07 2024-09-17T21:31:21.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495392+0000", 2024-09-17T21:31:21.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7430000000000001e-06 2024-09-17T21:31:21.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495398+0000", 2024-09-17T21:31:21.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3040000000000001e-06 2024-09-17T21:31:21.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495403+0000", 2024-09-17T21:31:21.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5600000000000004e-06 2024-09-17T21:31:21.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538660+0000", 2024-09-17T21:31:21.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043256832000000002 2024-09-17T21:31:21.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538746+0000", 2024-09-17T21:31:21.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6448999999999999e-05 2024-09-17T21:31:21.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539301+0000", 2024-09-17T21:31:21.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055525099999999999 2024-09-17T21:31:21.866 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.867 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.867 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.867 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50857 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19660800~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:21.867 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:21.867 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.300328871000005, 2024-09-17T21:31:21.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044408156999999997, 2024-09-17T21:31:21.867 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.867 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.867 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.867 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.867 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.867 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.868 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50857 2024-09-17T21:31:21.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.868 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:21.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:21.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495410+0000", 2024-09-17T21:31:21.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:31:21.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495412+0000", 2024-09-17T21:31:21.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9580000000000001e-06 2024-09-17T21:31:21.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495418+0000", 2024-09-17T21:31:21.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1909999999999997e-06 2024-09-17T21:31:21.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495423+0000", 2024-09-17T21:31:21.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2429999999999996e-06 2024-09-17T21:31:21.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539338+0000", 2024-09-17T21:31:21.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043914478 2024-09-17T21:31:21.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539393+0000", 2024-09-17T21:31:21.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5226999999999998e-05 2024-09-17T21:31:21.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539818+0000", 2024-09-17T21:31:21.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042469300000000002 2024-09-17T21:31:21.871 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.871 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.871 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.871 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50858 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23330816~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:21.871 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:21.871 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.300307774000004, 2024-09-17T21:31:21.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045010798999999997, 2024-09-17T21:31:21.872 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.872 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.872 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.872 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.872 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.872 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.872 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50858 2024-09-17T21:31:21.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.872 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:21.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:21.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:21.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:31:21.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495433+0000", 2024-09-17T21:31:21.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.787e-06 2024-09-17T21:31:21.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495438+0000", 2024-09-17T21:31:21.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:31:21.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495442+0000", 2024-09-17T21:31:21.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2640000000000002e-06 2024-09-17T21:31:21.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539852+0000", 2024-09-17T21:31:21.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044409592999999997 2024-09-17T21:31:21.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539940+0000", 2024-09-17T21:31:21.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7696999999999997e-05 2024-09-17T21:31:21.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540441+0000", 2024-09-17T21:31:21.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050164299999999997 2024-09-17T21:31:21.875 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.876 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.876 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.876 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50859 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24903680~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:21.876 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:21.876 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.300290095999998, 2024-09-17T21:31:21.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045571569999999999, 2024-09-17T21:31:21.876 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.876 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.876 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.877 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50859 2024-09-17T21:31:21.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.877 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:21.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:21.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495449+0000", 2024-09-17T21:31:21.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:21.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495450+0000", 2024-09-17T21:31:21.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7889999999999999e-06 2024-09-17T21:31:21.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495456+0000", 2024-09-17T21:31:21.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5679999999999999e-06 2024-09-17T21:31:21.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495460+0000", 2024-09-17T21:31:21.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4989999999999996e-06 2024-09-17T21:31:21.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540472+0000", 2024-09-17T21:31:21.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045011585999999999 2024-09-17T21:31:21.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540554+0000", 2024-09-17T21:31:21.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2443999999999999e-05 2024-09-17T21:31:21.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541020+0000", 2024-09-17T21:31:21.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00046534300000000001 2024-09-17T21:31:21.880 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.880 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.880 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.880 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50860 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28573696~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:21.880 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:21.881 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.300271742999996, 2024-09-17T21:31:21.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046167899999999998, 2024-09-17T21:31:21.881 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.881 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.881 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.881 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50860 2024-09-17T21:31:21.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.881 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:21.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:21.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:21.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:31:21.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495469+0000", 2024-09-17T21:31:21.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.99e-06 2024-09-17T21:31:21.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495475+0000", 2024-09-17T21:31:21.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:31:21.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495480+0000", 2024-09-17T21:31:21.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:31:21.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541050+0000", 2024-09-17T21:31:21.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045569853 2024-09-17T21:31:21.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541138+0000", 2024-09-17T21:31:21.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7997000000000005e-05 2024-09-17T21:31:21.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541634+0000", 2024-09-17T21:31:21.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00049685500000000004 2024-09-17T21:31:21.885 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.885 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.885 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.885 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50861 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 30146560~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:21.885 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:21.885 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.300252565999998, 2024-09-17T21:31:21.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046690553000000003, 2024-09-17T21:31:21.885 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.885 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.885 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.885 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.886 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.886 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.886 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50861 2024-09-17T21:31:21.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.886 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:21.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:21.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:21.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:31:21.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495488+0000", 2024-09-17T21:31:21.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:31:21.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495495+0000", 2024-09-17T21:31:21.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:31:21.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495500+0000", 2024-09-17T21:31:21.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8620000000000002e-06 2024-09-17T21:31:21.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541687+0000", 2024-09-17T21:31:21.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046186434999999998 2024-09-17T21:31:21.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541741+0000", 2024-09-17T21:31:21.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4228999999999999e-05 2024-09-17T21:31:21.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.542176+0000", 2024-09-17T21:31:21.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043531100000000002 2024-09-17T21:31:21.889 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.889 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.889 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.889 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97804 3.7 3:e725aa05:::rbd_data.1120c318df8c.0000000000000003:24 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:21.889 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:21.890 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.834306464999997, 2024-09-17T21:31:21.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035514416, 2024-09-17T21:31:21.890 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:21.890 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:21.890 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:21.890 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:21.890 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:21.890 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:21.890 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97804 2024-09-17T21:31:21.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.890 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:21.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:21.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:21.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:21.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:21.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:21.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:21.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:21.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:31:21.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:21.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961434+0000", 2024-09-17T21:31:21.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8959999999999999e-06 2024-09-17T21:31:21.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:21.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961439+0000", 2024-09-17T21:31:21.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T21:31:21.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:21.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961444+0000", 2024-09-17T21:31:21.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4970000000000003e-06 2024-09-17T21:31:21.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:21.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996466+0000", 2024-09-17T21:31:21.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035022438000000003 2024-09-17T21:31:21.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:21.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996556+0000", 2024-09-17T21:31:21.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9561000000000005e-05 2024-09-17T21:31:21.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:21.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:21.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:21.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996946+0000", 2024-09-17T21:31:21.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039028700000000001 2024-09-17T21:31:21.894 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.894 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.894 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.894 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:21.894 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:21.894 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:21.896 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:31:21.896 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:21.893+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:21.997 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:21.994+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:22.004 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:22.004 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:22.004 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:22.004 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:22.012 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:31:22.097 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:22.095+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:22.115 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:22.116 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:22.116 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:22.116 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:22.116 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:22.124 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:31:22.199 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:22.197+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:22.268 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21342 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2082816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:31:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.588983560000003, 2024-09-17T21:31:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022083523000000001, 2024-09-17T21:31:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21342 2024-09-17T21:31:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:31:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676659+0000", 2024-09-17T21:31:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0950000000000001e-06 2024-09-17T21:31:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676662+0000", 2024-09-17T21:31:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.971e-06 2024-09-17T21:31:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676665+0000", 2024-09-17T21:31:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6189999999999998e-06 2024-09-17T21:31:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698568+0000", 2024-09-17T21:31:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021903549000000001 2024-09-17T21:31:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698668+0000", 2024-09-17T21:31:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9778999999999995e-05 2024-09-17T21:31:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698741+0000", 2024-09-17T21:31:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000006e-05 2024-09-17T21:31:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21343 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2095104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:31:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.588973621999997, 2024-09-17T21:31:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022504024000000001, 2024-09-17T21:31:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21343 2024-09-17T21:31:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.275 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:22.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:22.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:22.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:31:22.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676669+0000", 2024-09-17T21:31:22.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.003e-06 2024-09-17T21:31:22.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676672+0000", 2024-09-17T21:31:22.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9280000000000002e-06 2024-09-17T21:31:22.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676674+0000", 2024-09-17T21:31:22.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5469999999999999e-06 2024-09-17T21:31:22.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699015+0000", 2024-09-17T21:31:22.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022340563000000001 2024-09-17T21:31:22.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699102+0000", 2024-09-17T21:31:22.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7425999999999999e-05 2024-09-17T21:31:22.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699172+0000", 2024-09-17T21:31:22.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9346999999999999e-05 2024-09-17T21:31:22.278 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.278 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.278 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.278 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45576 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:22.278 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:22.278 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.560739822000002, 2024-09-17T21:31:22.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022024660000000001, 2024-09-17T21:31:22.279 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.279 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.279 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.279 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.279 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.279 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.279 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45576 2024-09-17T21:31:22.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.279 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:22.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:22.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704902+0000", 2024-09-17T21:31:22.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9100000000000004e-07 2024-09-17T21:31:22.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704905+0000", 2024-09-17T21:31:22.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6139999999999999e-06 2024-09-17T21:31:22.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704911+0000", 2024-09-17T21:31:22.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8619999999999997e-06 2024-09-17T21:31:22.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704917+0000", 2024-09-17T21:31:22.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:31:22.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726681+0000", 2024-09-17T21:31:22.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021764569000000001 2024-09-17T21:31:22.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726726+0000", 2024-09-17T21:31:22.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4218999999999998e-05 2024-09-17T21:31:22.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726926+0000", 2024-09-17T21:31:22.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000200466 2024-09-17T21:31:22.283 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.283 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.283 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.283 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45577 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:22.283 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:22.283 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.560716166000006, 2024-09-17T21:31:22.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022269694999999999, 2024-09-17T21:31:22.284 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.284 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.284 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.284 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.284 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.284 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.284 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45577 2024-09-17T21:31:22.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.284 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:22.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:22.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704926+0000", 2024-09-17T21:31:22.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:31:22.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704928+0000", 2024-09-17T21:31:22.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9989999999999998e-06 2024-09-17T21:31:22.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704934+0000", 2024-09-17T21:31:22.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:31:22.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704939+0000", 2024-09-17T21:31:22.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5909999999999998e-06 2024-09-17T21:31:22.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726954+0000", 2024-09-17T21:31:22.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022014885000000001 2024-09-17T21:31:22.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726995+0000", 2024-09-17T21:31:22.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1474000000000003e-05 2024-09-17T21:31:22.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727195+0000", 2024-09-17T21:31:22.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019968300000000001 2024-09-17T21:31:22.287 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.288 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.288 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.288 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45578 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:22.288 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:22.288 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.560694921999996, 2024-09-17T21:31:22.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022484006000000001, 2024-09-17T21:31:22.288 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.288 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.288 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.288 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.288 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.288 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.289 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45578 2024-09-17T21:31:22.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.289 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:22.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:22.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704947+0000", 2024-09-17T21:31:22.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1399999999999997e-07 2024-09-17T21:31:22.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704949+0000", 2024-09-17T21:31:22.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0959999999999999e-06 2024-09-17T21:31:22.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704956+0000", 2024-09-17T21:31:22.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9330000000000004e-06 2024-09-17T21:31:22.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704961+0000", 2024-09-17T21:31:22.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6759999999999998e-06 2024-09-17T21:31:22.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727213+0000", 2024-09-17T21:31:22.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022252568 2024-09-17T21:31:22.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727259+0000", 2024-09-17T21:31:22.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5238000000000003e-05 2024-09-17T21:31:22.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727430+0000", 2024-09-17T21:31:22.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000171881 2024-09-17T21:31:22.292 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.292 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.292 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.292 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45579 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:22.292 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:22.292 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.560672994000001, 2024-09-17T21:31:22.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022749080000000001, 2024-09-17T21:31:22.293 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.293 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.293 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.293 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.293 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.293 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.293 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45579 2024-09-17T21:31:22.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.293 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:22.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:22.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704969+0000", 2024-09-17T21:31:22.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:31:22.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704971+0000", 2024-09-17T21:31:22.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.097e-06 2024-09-17T21:31:22.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704977+0000", 2024-09-17T21:31:22.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6329999999999999e-06 2024-09-17T21:31:22.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704983+0000", 2024-09-17T21:31:22.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:31:22.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727457+0000", 2024-09-17T21:31:22.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022474521000000001 2024-09-17T21:31:22.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727498+0000", 2024-09-17T21:31:22.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0558e-05 2024-09-17T21:31:22.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727717+0000", 2024-09-17T21:31:22.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219805 2024-09-17T21:31:22.297 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.297 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.297 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.297 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45652 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:22.297 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:22.297 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.559526856000005, 2024-09-17T21:31:22.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022100465999999999, 2024-09-17T21:31:22.297 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.297 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.297 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.297 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.298 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45652 2024-09-17T21:31:22.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.298 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:22.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:22.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706115+0000", 2024-09-17T21:31:22.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:31:22.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706117+0000", 2024-09-17T21:31:22.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-06 2024-09-17T21:31:22.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706122+0000", 2024-09-17T21:31:22.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.288e-06 2024-09-17T21:31:22.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706136+0000", 2024-09-17T21:31:22.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3409e-05 2024-09-17T21:31:22.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728112+0000", 2024-09-17T21:31:22.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021976724999999999 2024-09-17T21:31:22.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728151+0000", 2024-09-17T21:31:22.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8318999999999997e-05 2024-09-17T21:31:22.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728215+0000", 2024-09-17T21:31:22.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4172999999999997e-05 2024-09-17T21:31:22.301 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.301 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.301 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.302 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45653 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:22.302 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:22.302 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.559497645999997, 2024-09-17T21:31:22.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022249517999999999, 2024-09-17T21:31:22.302 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.302 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.302 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.302 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45653 2024-09-17T21:31:22.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.302 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:22.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:22.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:22.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T21:31:22.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706147+0000", 2024-09-17T21:31:22.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4430000000000002e-06 2024-09-17T21:31:22.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706152+0000", 2024-09-17T21:31:22.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5350000000000004e-06 2024-09-17T21:31:22.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706157+0000", 2024-09-17T21:31:22.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8420000000000004e-06 2024-09-17T21:31:22.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728246+0000", 2024-09-17T21:31:22.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022089237000000001 2024-09-17T21:31:22.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728309+0000", 2024-09-17T21:31:22.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2356999999999999e-05 2024-09-17T21:31:22.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:31:22.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4596000000000001e-05 2024-09-17T21:31:22.306 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.306 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.306 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.306 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45657 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 36864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:22.306 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:22.306 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.559466381999997, 2024-09-17T21:31:22.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022218252000000001, 2024-09-17T21:31:22.306 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.307 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.307 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.307 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.307 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.307 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.307 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45657 2024-09-17T21:31:22.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.307 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:22.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:22.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:22.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8299999999999997e-07 2024-09-17T21:31:22.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706178+0000", 2024-09-17T21:31:22.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3700000000000002e-06 2024-09-17T21:31:22.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706184+0000", 2024-09-17T21:31:22.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.144e-06 2024-09-17T21:31:22.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706190+0000", 2024-09-17T21:31:22.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8590000000000001e-06 2024-09-17T21:31:22.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728240+0000", 2024-09-17T21:31:22.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022049804999999999 2024-09-17T21:31:22.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728307+0000", 2024-09-17T21:31:22.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7477999999999993e-05 2024-09-17T21:31:22.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:31:22.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6112999999999994e-05 2024-09-17T21:31:22.310 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.310 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.310 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.311 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45658 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:22.311 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:22.311 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.559443262000002, 2024-09-17T21:31:22.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022376455, 2024-09-17T21:31:22.311 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.311 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.311 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.311 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.311 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.311 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.311 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45658 2024-09-17T21:31:22.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.311 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:22.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:22.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:22.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:31:22.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706200+0000", 2024-09-17T21:31:22.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:31:22.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706206+0000", 2024-09-17T21:31:22.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7039999999999999e-06 2024-09-17T21:31:22.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706211+0000", 2024-09-17T21:31:22.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9429999999999999e-06 2024-09-17T21:31:22.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728418+0000", 2024-09-17T21:31:22.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022206760999999998 2024-09-17T21:31:22.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728497+0000", 2024-09-17T21:31:22.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9493999999999995e-05 2024-09-17T21:31:22.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728574+0000", 2024-09-17T21:31:22.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7180000000000003e-05 2024-09-17T21:31:22.315 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.315 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.315 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.315 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45707 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 212992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:22.315 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:22.315 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.557739196, 2024-09-17T21:31:22.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022235550999999999, 2024-09-17T21:31:22.315 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.316 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.316 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.316 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.316 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.316 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.316 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45707 2024-09-17T21:31:22.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.316 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:22.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:22.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:22.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:31:22.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707905+0000", 2024-09-17T21:31:22.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2120000000000002e-06 2024-09-17T21:31:22.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707911+0000", 2024-09-17T21:31:22.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1929999999999998e-06 2024-09-17T21:31:22.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707915+0000", 2024-09-17T21:31:22.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:31:22.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729897+0000", 2024-09-17T21:31:22.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021981186 2024-09-17T21:31:22.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729944+0000", 2024-09-17T21:31:22.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571000000000002e-05 2024-09-17T21:31:22.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730138+0000", 2024-09-17T21:31:22.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000193451 2024-09-17T21:31:22.319 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.319 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.320 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.320 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45708 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:22.320 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:22.320 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.557719063999997, 2024-09-17T21:31:22.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022533742999999998, 2024-09-17T21:31:22.320 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.320 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.320 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.320 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.320 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.320 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.320 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45708 2024-09-17T21:31:22.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.321 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:22.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:22.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707923+0000", 2024-09-17T21:31:22.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:31:22.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707932+0000", 2024-09-17T21:31:22.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6430000000000007e-06 2024-09-17T21:31:22.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707939+0000", 2024-09-17T21:31:22.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3539999999999996e-06 2024-09-17T21:31:22.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707944+0000", 2024-09-17T21:31:22.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3249999999999998e-06 2024-09-17T21:31:22.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730176+0000", 2024-09-17T21:31:22.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022232393 2024-09-17T21:31:22.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730263+0000", 2024-09-17T21:31:22.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6638999999999995e-05 2024-09-17T21:31:22.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730456+0000", 2024-09-17T21:31:22.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019304899999999999 2024-09-17T21:31:22.324 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.324 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.324 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.324 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45752 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:22.324 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:22.324 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.557463843999997, 2024-09-17T21:31:22.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022426878000000001, 2024-09-17T21:31:22.325 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.325 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.325 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.325 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.325 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.325 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.325 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45752 2024-09-17T21:31:22.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.325 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:22.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:22.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708178+0000", 2024-09-17T21:31:22.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:31:22.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708181+0000", 2024-09-17T21:31:22.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.909e-06 2024-09-17T21:31:22.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708187+0000", 2024-09-17T21:31:22.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4999999999999996e-06 2024-09-17T21:31:22.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708193+0000", 2024-09-17T21:31:22.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:31:22.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730484+0000", 2024-09-17T21:31:22.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022291765000000002 2024-09-17T21:31:22.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730541+0000", 2024-09-17T21:31:22.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6483000000000001e-05 2024-09-17T21:31:22.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730604+0000", 2024-09-17T21:31:22.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3429000000000005e-05 2024-09-17T21:31:22.329 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.329 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.329 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.329 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45761 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:22.329 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:22.329 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.557435933999997, 2024-09-17T21:31:22.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022614408999999999, 2024-09-17T21:31:22.329 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.329 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.329 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.329 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.329 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.330 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.330 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45761 2024-09-17T21:31:22.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.330 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:22.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:22.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708206+0000", 2024-09-17T21:31:22.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4700000000000001e-07 2024-09-17T21:31:22.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708208+0000", 2024-09-17T21:31:22.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1509999999999998e-06 2024-09-17T21:31:22.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708214+0000", 2024-09-17T21:31:22.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.49e-06 2024-09-17T21:31:22.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708219+0000", 2024-09-17T21:31:22.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8119999999999998e-06 2024-09-17T21:31:22.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730641+0000", 2024-09-17T21:31:22.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022422765000000001 2024-09-17T21:31:22.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730731+0000", 2024-09-17T21:31:22.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9598999999999994e-05 2024-09-17T21:31:22.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730820+0000", 2024-09-17T21:31:22.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8844999999999994e-05 2024-09-17T21:31:22.333 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.333 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.333 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.333 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45785 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:22.334 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:22.334 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.556214147000006, 2024-09-17T21:31:22.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022044113000000001, 2024-09-17T21:31:22.334 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.334 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.334 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.334 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45785 2024-09-17T21:31:22.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.334 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:22.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:22.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709428+0000", 2024-09-17T21:31:22.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:31:22.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709430+0000", 2024-09-17T21:31:22.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:31:22.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709451+0000", 2024-09-17T21:31:22.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1372000000000001e-05 2024-09-17T21:31:22.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709472+0000", 2024-09-17T21:31:22.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1007999999999999e-05 2024-09-17T21:31:22.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731293+0000", 2024-09-17T21:31:22.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021821320000000002 2024-09-17T21:31:22.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731378+0000", 2024-09-17T21:31:22.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5059e-05 2024-09-17T21:31:22.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731471+0000", 2024-09-17T21:31:22.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2867000000000004e-05 2024-09-17T21:31:22.338 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.338 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.338 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.338 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45786 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:22.338 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:22.338 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.556155349999997, 2024-09-17T21:31:22.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022181821000000001, 2024-09-17T21:31:22.338 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.338 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.338 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.339 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.339 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.339 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.339 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45786 2024-09-17T21:31:22.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.339 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:22.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:22.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709487+0000", 2024-09-17T21:31:22.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1299999999999999e-07 2024-09-17T21:31:22.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709491+0000", 2024-09-17T21:31:22.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2089999999999999e-06 2024-09-17T21:31:22.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709498+0000", 2024-09-17T21:31:22.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3980000000000002e-06 2024-09-17T21:31:22.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709504+0000", 2024-09-17T21:31:22.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7219999999999996e-06 2024-09-17T21:31:22.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731499+0000", 2024-09-17T21:31:22.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021995297 2024-09-17T21:31:22.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731591+0000", 2024-09-17T21:31:22.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1373999999999996e-05 2024-09-17T21:31:22.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731668+0000", 2024-09-17T21:31:22.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7107999999999994e-05 2024-09-17T21:31:22.342 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.342 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.342 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.343 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50202 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'139, mlcod=45'137)", 2024-09-17T21:31:22.343 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:22.343 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 71.141419481, 2024-09-17T21:31:22.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059659441000000001, 2024-09-17T21:31:22.343 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.343 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.343 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:22.344 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:22.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:22.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124224+0000", 2024-09-17T21:31:22.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5000000000000002e-06 2024-09-17T21:31:22.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147385+0000", 2024-09-17T21:31:22.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023160544000000002 2024-09-17T21:31:22.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147411+0000", 2024-09-17T21:31:22.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6095999999999999e-05 2024-09-17T21:31:22.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147438+0000", 2024-09-17T21:31:22.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6982999999999999e-05 2024-09-17T21:31:22.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147504+0000", 2024-09-17T21:31:22.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6156999999999997e-05 2024-09-17T21:31:22.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147531+0000", 2024-09-17T21:31:22.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6741999999999999e-05 2024-09-17T21:31:22.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183820+0000", 2024-09-17T21:31:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036289090000000003 2024-09-17T21:31:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183881+0000", 2024-09-17T21:31:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1328999999999995e-05 2024-09-17T21:31:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50203 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'141, mlcod=45'137)", 2024-09-17T21:31:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 71.104156738, 2024-09-17T21:31:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023138823999999999, 2024-09-17T21:31:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161486+0000", 2024-09-17T21:31:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5290000000000001e-06 2024-09-17T21:31:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161496+0000", 2024-09-17T21:31:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0063e-05 2024-09-17T21:31:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161514+0000", 2024-09-17T21:31:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7728e-05 2024-09-17T21:31:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161524+0000", 2024-09-17T21:31:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0339e-05 2024-09-17T21:31:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170872+0000", 2024-09-17T21:31:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093473640000000004 2024-09-17T21:31:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170899+0000", 2024-09-17T21:31:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7321000000000002e-05 2024-09-17T21:31:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184580+0000", 2024-09-17T21:31:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013680988999999999 2024-09-17T21:31:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184623+0000", 2024-09-17T21:31:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3491000000000001e-05 2024-09-17T21:31:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:88193 3.9 3:97498da5:::rbd_data.1120ea662e44.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-17T21:31:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.941975827999997, 2024-09-17T21:31:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026278261000000001, 2024-09-17T21:31:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 88193 2024-09-17T21:31:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323666+0000", 2024-09-17T21:31:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:31:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323670+0000", 2024-09-17T21:31:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.771e-06 2024-09-17T21:31:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323680+0000", 2024-09-17T21:31:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9469999999999998e-06 2024-09-17T21:31:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323685+0000", 2024-09-17T21:31:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9919999999999998e-06 2024-09-17T21:31:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323728+0000", 2024-09-17T21:31:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2811000000000001e-05 2024-09-17T21:31:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323776+0000", 2024-09-17T21:31:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8310000000000003e-05 2024-09-17T21:31:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.328583+0000", 2024-09-17T21:31:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048072899999999997 2024-09-17T21:31:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349861+0000", 2024-09-17T21:31:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021277695999999999 2024-09-17T21:31:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349877+0000", 2024-09-17T21:31:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:31:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349911+0000", 2024-09-17T21:31:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3844999999999999e-05 2024-09-17T21:31:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349944+0000", 2024-09-17T21:31:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2663000000000002e-05 2024-09-17T21:31:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:101384 3.9 3:90f2a32a:::rbd_data.11206b8c0882.0000000000000178:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T21:31:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.939106228999997, 2024-09-17T21:31:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025286194000000001, 2024-09-17T21:31:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 101384 2024-09-17T21:31:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326536+0000", 2024-09-17T21:31:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4799999999999997e-07 2024-09-17T21:31:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326541+0000", 2024-09-17T21:31:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8489999999999997e-06 2024-09-17T21:31:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326552+0000", 2024-09-17T21:31:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0912e-05 2024-09-17T21:31:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326557+0000", 2024-09-17T21:31:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0359999999999997e-06 2024-09-17T21:31:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326603+0000", 2024-09-17T21:31:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6428999999999999e-05 2024-09-17T21:31:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326686+0000", 2024-09-17T21:31:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2521999999999994e-05 2024-09-17T21:31:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.332698+0000", 2024-09-17T21:31:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060124599999999999 2024-09-17T21:31:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351744+0000", 2024-09-17T21:31:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019045838999999998 2024-09-17T21:31:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351754+0000", 2024-09-17T21:31:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0380999999999999e-05 2024-09-17T21:31:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351791+0000", 2024-09-17T21:31:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6678999999999999e-05 2024-09-17T21:31:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351821+0000", 2024-09-17T21:31:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0139000000000001e-05 2024-09-17T21:31:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.364 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:22.366 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:31:22.366 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:22.364+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:22.412 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is unset 2024-09-17T21:31:22.466 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:22.464+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:22.526 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:22.526 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:22.535 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:31:22.568 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:22.565+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:22.669 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:22.666+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:22.678 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:22.678 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:22.678 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:22.678 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:22.679 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:22.687 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:31:22.769 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:22.766+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:22.792 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:22.793 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:22.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:22.793 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:22.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.793 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10799 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.793 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:22.793 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.724199349000003, 2024-09-17T21:31:22.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039122745, 2024-09-17T21:31:22.793 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.793 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.793 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:22.793 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.794 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.794 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.794 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10799 2024-09-17T21:31:22.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.794 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:22.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:22.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065469+0000", 2024-09-17T21:31:22.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:31:22.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065477+0000", 2024-09-17T21:31:22.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1319999999999994e-06 2024-09-17T21:31:22.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065483+0000", 2024-09-17T21:31:22.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2040000000000002e-06 2024-09-17T21:31:22.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065487+0000", 2024-09-17T21:31:22.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.42e-06 2024-09-17T21:31:22.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099984+0000", 2024-09-17T21:31:22.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034496484000000001 2024-09-17T21:31:22.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100065+0000", 2024-09-17T21:31:22.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1200999999999997e-05 2024-09-17T21:31:22.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:22.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100422+0000", 2024-09-17T21:31:22.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035657700000000001 2024-09-17T21:31:22.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:22.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104214+0000", 2024-09-17T21:31:22.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003792045 2024-09-17T21:31:22.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:22.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104505+0000", 2024-09-17T21:31:22.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029141399999999999 2024-09-17T21:31:22.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:22.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104556+0000", 2024-09-17T21:31:22.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1388999999999999e-05 2024-09-17T21:31:22.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104591+0000", 2024-09-17T21:31:22.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4455999999999997e-05 2024-09-17T21:31:22.799 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.799 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.799 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.799 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12187 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.799 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:22.799 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.650106617999995, 2024-09-17T21:31:22.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038825750999999999, 2024-09-17T21:31:22.799 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.799 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.799 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.799 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.800 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.800 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.800 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12187 2024-09-17T21:31:22.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.800 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:22.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:22.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:22.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:31:22.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139563+0000", 2024-09-17T21:31:22.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.624e-06 2024-09-17T21:31:22.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139569+0000", 2024-09-17T21:31:22.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1090000000000003e-06 2024-09-17T21:31:22.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139573+0000", 2024-09-17T21:31:22.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4429999999999996e-06 2024-09-17T21:31:22.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178206+0000", 2024-09-17T21:31:22.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038632376000000003 2024-09-17T21:31:22.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178282+0000", 2024-09-17T21:31:22.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6076999999999997e-05 2024-09-17T21:31:22.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178387+0000", 2024-09-17T21:31:22.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104682 2024-09-17T21:31:22.803 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.803 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.803 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.804 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12188 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.804 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:22.804 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.650088436999994, 2024-09-17T21:31:22.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039053826999999999, 2024-09-17T21:31:22.804 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.804 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.804 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.804 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.804 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.804 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.804 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12188 2024-09-17T21:31:22.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.804 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:22.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:22.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139746+0000", 2024-09-17T21:31:22.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000167281 2024-09-17T21:31:22.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139749+0000", 2024-09-17T21:31:22.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9660000000000001e-06 2024-09-17T21:31:22.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139757+0000", 2024-09-17T21:31:22.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8029999999999997e-06 2024-09-17T21:31:22.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139771+0000", 2024-09-17T21:31:22.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3541e-05 2024-09-17T21:31:22.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178415+0000", 2024-09-17T21:31:22.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038644803999999998 2024-09-17T21:31:22.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178517+0000", 2024-09-17T21:31:22.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101298 2024-09-17T21:31:22.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178633+0000", 2024-09-17T21:31:22.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116134 2024-09-17T21:31:22.808 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.808 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.808 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.808 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12189 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.808 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:22.808 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.649887960000001, 2024-09-17T21:31:22.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039135991000000002, 2024-09-17T21:31:22.809 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.809 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.809 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.809 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.809 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.809 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.809 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12189 2024-09-17T21:31:22.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.809 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:22.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:22.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139780+0000", 2024-09-17T21:31:22.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:31:22.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139782+0000", 2024-09-17T21:31:22.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1950000000000002e-06 2024-09-17T21:31:22.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139788+0000", 2024-09-17T21:31:22.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0639999999999999e-06 2024-09-17T21:31:22.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139792+0000", 2024-09-17T21:31:22.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1960000000000001e-06 2024-09-17T21:31:22.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178671+0000", 2024-09-17T21:31:22.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038878176 2024-09-17T21:31:22.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178778+0000", 2024-09-17T21:31:22.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107278 2024-09-17T21:31:22.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178915+0000", 2024-09-17T21:31:22.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013757900000000001 2024-09-17T21:31:22.813 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.813 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.813 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.813 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12190 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.813 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:22.813 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.649868776999995, 2024-09-17T21:31:22.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039343781000000001, 2024-09-17T21:31:22.813 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.813 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.813 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.813 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.813 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.814 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.814 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12190 2024-09-17T21:31:22.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.814 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:22.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:22.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:22.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:22.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139801+0000", 2024-09-17T21:31:22.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.916e-06 2024-09-17T21:31:22.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139807+0000", 2024-09-17T21:31:22.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4540000000000004e-06 2024-09-17T21:31:22.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139812+0000", 2024-09-17T21:31:22.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9729999999999997e-06 2024-09-17T21:31:22.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178960+0000", 2024-09-17T21:31:22.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039147567000000001 2024-09-17T21:31:22.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179051+0000", 2024-09-17T21:31:22.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0754000000000006e-05 2024-09-17T21:31:22.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179142+0000", 2024-09-17T21:31:22.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1712999999999994e-05 2024-09-17T21:31:22.817 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.817 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.817 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.817 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12191 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.817 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:22.818 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.649848918999993, 2024-09-17T21:31:22.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039464336000000003, 2024-09-17T21:31:22.818 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.818 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.818 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.818 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.818 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.818 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.818 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12191 2024-09-17T21:31:22.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.818 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:22.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:22.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:22.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:31:22.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139821+0000", 2024-09-17T21:31:22.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1260000000000001e-06 2024-09-17T21:31:22.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139827+0000", 2024-09-17T21:31:22.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T21:31:22.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139840+0000", 2024-09-17T21:31:22.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2607e-05 2024-09-17T21:31:22.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179162+0000", 2024-09-17T21:31:22.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039322863999999999 2024-09-17T21:31:22.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179204+0000", 2024-09-17T21:31:22.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1938999999999999e-05 2024-09-17T21:31:22.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179283+0000", 2024-09-17T21:31:22.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8547000000000005e-05 2024-09-17T21:31:22.822 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.822 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.822 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.822 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12192 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.822 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:22.822 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.649821207000002, 2024-09-17T21:31:22.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039652318999999998, 2024-09-17T21:31:22.822 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.822 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.822 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.822 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.823 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.823 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.823 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12192 2024-09-17T21:31:22.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.823 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:22.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:22.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141458+0000", 2024-09-17T21:31:22.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001612098 2024-09-17T21:31:22.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141469+0000", 2024-09-17T21:31:22.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-17T21:31:22.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141478+0000", 2024-09-17T21:31:22.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T21:31:22.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141485+0000", 2024-09-17T21:31:22.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4760000000000003e-06 2024-09-17T21:31:22.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179317+0000", 2024-09-17T21:31:22.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037831832000000003 2024-09-17T21:31:22.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179395+0000", 2024-09-17T21:31:22.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8349000000000005e-05 2024-09-17T21:31:22.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179499+0000", 2024-09-17T21:31:22.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103628 2024-09-17T21:31:22.826 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.826 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.826 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.827 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12257 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.827 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:22.827 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.647632399000003, 2024-09-17T21:31:22.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038862751000000001, 2024-09-17T21:31:22.827 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.827 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.827 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.827 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.827 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.827 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.827 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12257 2024-09-17T21:31:22.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.827 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:22.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:22.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:22.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T21:31:22.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142037+0000", 2024-09-17T21:31:22.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.03e-06 2024-09-17T21:31:22.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142043+0000", 2024-09-17T21:31:22.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6319999999999996e-06 2024-09-17T21:31:22.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142052+0000", 2024-09-17T21:31:22.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2709999999999998e-06 2024-09-17T21:31:22.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180794+0000", 2024-09-17T21:31:22.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038741758000000001 2024-09-17T21:31:22.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180843+0000", 2024-09-17T21:31:22.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9021999999999999e-05 2024-09-17T21:31:22.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180898+0000", 2024-09-17T21:31:22.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673000000000003e-05 2024-09-17T21:31:22.831 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.831 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.831 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.831 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12258 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.831 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:22.831 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.647608882, 2024-09-17T21:31:22.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038950717000000003, 2024-09-17T21:31:22.831 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.831 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.831 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.832 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.832 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.832 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.832 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12258 2024-09-17T21:31:22.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.832 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:22.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:22.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:22.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199999999999999e-07 2024-09-17T21:31:22.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142061+0000", 2024-09-17T21:31:22.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8279999999999999e-06 2024-09-17T21:31:22.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142067+0000", 2024-09-17T21:31:22.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5600000000000001e-06 2024-09-17T21:31:22.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142071+0000", 2024-09-17T21:31:22.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6240000000000001e-06 2024-09-17T21:31:22.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180921+0000", 2024-09-17T21:31:22.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038849366000000003 2024-09-17T21:31:22.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180961+0000", 2024-09-17T21:31:22.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0219000000000002e-05 2024-09-17T21:31:22.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181009+0000", 2024-09-17T21:31:22.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8437999999999999e-05 2024-09-17T21:31:22.835 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.835 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.835 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.836 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12259 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.836 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:22.836 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.647581634999995, 2024-09-17T21:31:22.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039116832999999997, 2024-09-17T21:31:22.836 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.836 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.836 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.836 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12259 2024-09-17T21:31:22.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.836 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:22.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:22.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:22.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2200000000000004e-07 2024-09-17T21:31:22.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142088+0000", 2024-09-17T21:31:22.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0329999999999998e-06 2024-09-17T21:31:22.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142095+0000", 2024-09-17T21:31:22.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4320000000000004e-06 2024-09-17T21:31:22.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142100+0000", 2024-09-17T21:31:22.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:31:22.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181037+0000", 2024-09-17T21:31:22.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038937601000000002 2024-09-17T21:31:22.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181114+0000", 2024-09-17T21:31:22.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6857e-05 2024-09-17T21:31:22.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181203+0000", 2024-09-17T21:31:22.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8437999999999996e-05 2024-09-17T21:31:22.840 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.840 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.840 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.840 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12260 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.840 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:22.840 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.647561347999996, 2024-09-17T21:31:22.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039204184000000003, 2024-09-17T21:31:22.840 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.840 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.840 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.841 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12260 2024-09-17T21:31:22.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.841 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:22.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:22.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142107+0000", 2024-09-17T21:31:22.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:31:22.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142108+0000", 2024-09-17T21:31:22.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.753e-06 2024-09-17T21:31:22.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142114+0000", 2024-09-17T21:31:22.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7660000000000004e-06 2024-09-17T21:31:22.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142119+0000", 2024-09-17T21:31:22.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:31:22.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181216+0000", 2024-09-17T21:31:22.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039097191000000003 2024-09-17T21:31:22.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181260+0000", 2024-09-17T21:31:22.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4731000000000003e-05 2024-09-17T21:31:22.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181310+0000", 2024-09-17T21:31:22.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9818000000000003e-05 2024-09-17T21:31:22.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.844 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.844 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12261 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.845 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:22.845 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.647543024000001, 2024-09-17T21:31:22.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039292200999999999, 2024-09-17T21:31:22.845 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.845 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.845 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.845 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12261 2024-09-17T21:31:22.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.846 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:22.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:22.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142125+0000", 2024-09-17T21:31:22.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:31:22.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142127+0000", 2024-09-17T21:31:22.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.792e-06 2024-09-17T21:31:22.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142132+0000", 2024-09-17T21:31:22.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3979999999999998e-06 2024-09-17T21:31:22.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142136+0000", 2024-09-17T21:31:22.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.211e-06 2024-09-17T21:31:22.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181324+0000", 2024-09-17T21:31:22.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039187560000000003 2024-09-17T21:31:22.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181363+0000", 2024-09-17T21:31:22.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9375000000000002e-05 2024-09-17T21:31:22.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181417+0000", 2024-09-17T21:31:22.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3372000000000003e-05 2024-09-17T21:31:22.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.849 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.849 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12262 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.849 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:22.849 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.647525231000003, 2024-09-17T21:31:22.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039384875, 2024-09-17T21:31:22.850 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.850 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.850 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.850 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.850 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.850 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.850 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12262 2024-09-17T21:31:22.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.850 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:22.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:22.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142143+0000", 2024-09-17T21:31:22.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:31:22.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142148+0000", 2024-09-17T21:31:22.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4360000000000001e-06 2024-09-17T21:31:22.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142154+0000", 2024-09-17T21:31:22.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8719999999999999e-06 2024-09-17T21:31:22.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142158+0000", 2024-09-17T21:31:22.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.476e-06 2024-09-17T21:31:22.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181436+0000", 2024-09-17T21:31:22.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039277919000000001 2024-09-17T21:31:22.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181480+0000", 2024-09-17T21:31:22.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3649999999999997e-05 2024-09-17T21:31:22.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181527+0000", 2024-09-17T21:31:22.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7131e-05 2024-09-17T21:31:22.853 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.853 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.854 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.854 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12263 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1079296~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.854 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:22.854 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.647502795999998, 2024-09-17T21:31:22.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039473339000000003, 2024-09-17T21:31:22.854 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.854 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.854 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.854 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12263 2024-09-17T21:31:22.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.855 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:22.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:22.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:22.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:31:22.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142167+0000", 2024-09-17T21:31:22.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.81e-06 2024-09-17T21:31:22.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142172+0000", 2024-09-17T21:31:22.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:31:22.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142177+0000", 2024-09-17T21:31:22.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T21:31:22.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181553+0000", 2024-09-17T21:31:22.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039376130000000002 2024-09-17T21:31:22.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181592+0000", 2024-09-17T21:31:22.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9153999999999998e-05 2024-09-17T21:31:22.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181638+0000", 2024-09-17T21:31:22.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5864e-05 2024-09-17T21:31:22.858 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.858 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.858 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.858 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12278 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1095680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.858 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:22.858 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.647484821000006, 2024-09-17T21:31:22.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039454222999999997, 2024-09-17T21:31:22.859 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.859 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.859 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.859 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12278 2024-09-17T21:31:22.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.859 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:22.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:22.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:22.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:31:22.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142185+0000", 2024-09-17T21:31:22.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.697e-06 2024-09-17T21:31:22.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142191+0000", 2024-09-17T21:31:22.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:31:22.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142196+0000", 2024-09-17T21:31:22.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4829999999999999e-06 2024-09-17T21:31:22.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181548+0000", 2024-09-17T21:31:22.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039351200000000003 2024-09-17T21:31:22.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181591+0000", 2024-09-17T21:31:22.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3492000000000003e-05 2024-09-17T21:31:22.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181637+0000", 2024-09-17T21:31:22.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5847000000000003e-05 2024-09-17T21:31:22.862 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.862 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.863 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.863 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12279 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.863 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:22.863 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.647461531999994, 2024-09-17T21:31:22.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039539801999999999, 2024-09-17T21:31:22.863 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.863 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.863 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.863 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.863 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.863 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.863 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12279 2024-09-17T21:31:22.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.864 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:22.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:22.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:22.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.89e-07 2024-09-17T21:31:22.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142208+0000", 2024-09-17T21:31:22.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7579999999999999e-06 2024-09-17T21:31:22.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142214+0000", 2024-09-17T21:31:22.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:31:22.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142218+0000", 2024-09-17T21:31:22.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3429999999999998e-06 2024-09-17T21:31:22.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181653+0000", 2024-09-17T21:31:22.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039435322000000002 2024-09-17T21:31:22.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181698+0000", 2024-09-17T21:31:22.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4978999999999997e-05 2024-09-17T21:31:22.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181746+0000", 2024-09-17T21:31:22.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571999999999998e-05 2024-09-17T21:31:22.867 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.867 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.867 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.867 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12280 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1124352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.867 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:22.867 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.647444012999998, 2024-09-17T21:31:22.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039704358000000002, 2024-09-17T21:31:22.868 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.868 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.868 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.868 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.868 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.868 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.868 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12280 2024-09-17T21:31:22.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.868 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:22.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:22.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142224+0000", 2024-09-17T21:31:22.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:31:22.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142226+0000", 2024-09-17T21:31:22.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:31:22.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142231+0000", 2024-09-17T21:31:22.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:31:22.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142240+0000", 2024-09-17T21:31:22.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4590000000000004e-06 2024-09-17T21:31:22.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181774+0000", 2024-09-17T21:31:22.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039533833999999997 2024-09-17T21:31:22.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181859+0000", 2024-09-17T21:31:22.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4328999999999998e-05 2024-09-17T21:31:22.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181928+0000", 2024-09-17T21:31:22.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9139000000000003e-05 2024-09-17T21:31:22.871 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.872 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.872 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.872 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12281 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1136640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.872 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:22.872 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.647421107, 2024-09-17T21:31:22.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039787799999999998, 2024-09-17T21:31:22.872 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.872 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.872 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.872 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.872 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.872 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.873 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12281 2024-09-17T21:31:22.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.873 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:22.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:22.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142309+0000", 2024-09-17T21:31:22.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2323999999999995e-05 2024-09-17T21:31:22.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142317+0000", 2024-09-17T21:31:22.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1859999999999992e-06 2024-09-17T21:31:22.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142324+0000", 2024-09-17T21:31:22.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4699999999999996e-06 2024-09-17T21:31:22.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142330+0000", 2024-09-17T21:31:22.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4650000000000002e-06 2024-09-17T21:31:22.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181948+0000", 2024-09-17T21:31:22.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039618056999999998 2024-09-17T21:31:22.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181986+0000", 2024-09-17T21:31:22.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8442999999999998e-05 2024-09-17T21:31:22.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182034+0000", 2024-09-17T21:31:22.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7855000000000002e-05 2024-09-17T21:31:22.876 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.876 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.876 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.876 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12282 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1150976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.876 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:22.876 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.647328877999996, 2024-09-17T21:31:22.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039871919999999998, 2024-09-17T21:31:22.877 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.877 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.877 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:22.877 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.877 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.877 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.877 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12282 2024-09-17T21:31:22.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.877 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:22.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:22.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:22.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:31:22.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142341+0000", 2024-09-17T21:31:22.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.232e-06 2024-09-17T21:31:22.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142348+0000", 2024-09-17T21:31:22.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7320000000000001e-06 2024-09-17T21:31:22.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142353+0000", 2024-09-17T21:31:22.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:31:22.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182062+0000", 2024-09-17T21:31:22.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039709739000000001 2024-09-17T21:31:22.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:22.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182128+0000", 2024-09-17T21:31:22.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5739000000000002e-05 2024-09-17T21:31:22.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182210+0000", 2024-09-17T21:31:22.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:31:22.880 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.880 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.880 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.881 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12283 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:22.881 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:22.881 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.647308467000002, 2024-09-17T21:31:22.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039896434000000001, 2024-09-17T21:31:22.881 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:22.881 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:22.881 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:31:22.881 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:22.881 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:22.881 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:22.881 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12283 2024-09-17T21:31:22.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.882 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:22.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:22.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:22.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:22.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:22.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:22.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:22.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144852+0000", 2024-09-17T21:31:22.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024934079999999999 2024-09-17T21:31:22.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:22.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144856+0000", 2024-09-17T21:31:22.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:31:22.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:22.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144864+0000", 2024-09-17T21:31:22.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0160000000000008e-06 2024-09-17T21:31:22.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:22.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144869+0000", 2024-09-17T21:31:22.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:31:22.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:22.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182246+0000", 2024-09-17T21:31:22.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037377207000000003 2024-09-17T21:31:22.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:22.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:22.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:22.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182255+0000", 2024-09-17T21:31:22.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1840000000000005e-06 2024-09-17T21:31:22.884 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.885 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.885 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.885 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:22.885 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:22.885 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:22.886 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:31:22.887 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:22.883+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:22.890 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:31:22.987 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:22.984+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:22.992 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:22.992 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:22.992 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:22.992 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:23.001 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:31:23.013 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:23.013 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:23.029 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:23.029 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:23.045 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:23.045 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:23.061 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:23.061 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:23.078 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:23.079 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:23.088 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:23.085+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:23.095 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:23.095 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:23.111 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:23.111 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:23.128 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:23.128 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:23.147 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:23.147 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:23.155 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:31:23.201 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:23.196+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:23.259 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:23.259 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:23.268 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:31:23.289 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:23.287+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:23.390 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:23.388+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:23.412 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:23.412 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:23.420 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:31:23.491 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:23.488+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:23.530 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:23.530 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:23.540 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:31:23.591 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:23.588+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:23.652 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:23.652 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:23.661 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:31:23.704 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:23.702+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:23.764 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:23.764 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:23.772 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:31:23.793 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:23.790+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:23.894 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:23.892+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:23.916 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:23.916 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:23.925 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:31:23.995 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:23.992+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:24.068 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:24.068 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:24.077 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:31:24.120 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:24.118+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:24.179 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:24.179 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:24.188 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:31:24.197 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:24.195+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:24.298 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:24.296+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:24.332 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:24.332 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:24.341 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:31:24.425 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:24.422+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:24.428 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:31:24.486 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:24.486 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:24.495 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:31:24.500 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:24.498+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:24.601 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:24.599+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:24.611 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:24.611 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:24.622 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:31:24.702 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:24.700+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:24.766 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:24.766 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:24.775 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:31:24.805 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:24.802+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:24.878 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:24.878 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:24.886 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:31:24.906 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:24.903+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:24.989 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:24.989 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:24.998 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:31:25.042 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:25.040+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:25.100 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:25.100 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:25.109 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:31:25.139 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:31:25.151 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:25.148+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:25.212 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:25.209+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:25.257 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:25.257 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:25.266 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:31:25.272 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:25.272 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:25.289 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:25.290 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:25.305 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:25.305 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:25.312 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:25.309+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:25.321 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:25.321 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:25.337 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:25.337 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:25.354 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:25.354 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:25.370 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:25.370 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:25.382 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:25.382 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:25.382 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:25.382 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:25.386 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:25.386 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:25.392 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:31:25.438 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:25.436+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:25.509 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is unset 2024-09-17T21:31:25.515 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:25.512+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:25.545 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:25.545 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:25.545 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:25.545 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:25.545 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:25.554 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:31:25.615 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:25.613+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:25.663 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21342 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2082816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:31:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.984020771000004, 2024-09-17T21:31:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022083523000000001, 2024-09-17T21:31:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21342 2024-09-17T21:31:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:31:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676659+0000", 2024-09-17T21:31:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0950000000000001e-06 2024-09-17T21:31:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676662+0000", 2024-09-17T21:31:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.971e-06 2024-09-17T21:31:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676665+0000", 2024-09-17T21:31:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6189999999999998e-06 2024-09-17T21:31:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698568+0000", 2024-09-17T21:31:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021903549000000001 2024-09-17T21:31:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698668+0000", 2024-09-17T21:31:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9778999999999995e-05 2024-09-17T21:31:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698741+0000", 2024-09-17T21:31:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000006e-05 2024-09-17T21:31:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21343 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2095104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:31:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.984010832999999, 2024-09-17T21:31:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022504024000000001, 2024-09-17T21:31:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21343 2024-09-17T21:31:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:31:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676669+0000", 2024-09-17T21:31:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.003e-06 2024-09-17T21:31:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676672+0000", 2024-09-17T21:31:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9280000000000002e-06 2024-09-17T21:31:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676674+0000", 2024-09-17T21:31:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5469999999999999e-06 2024-09-17T21:31:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699015+0000", 2024-09-17T21:31:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022340563000000001 2024-09-17T21:31:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699102+0000", 2024-09-17T21:31:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7425999999999999e-05 2024-09-17T21:31:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699172+0000", 2024-09-17T21:31:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9346999999999999e-05 2024-09-17T21:31:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45576 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.955777033000004, 2024-09-17T21:31:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022024660000000001, 2024-09-17T21:31:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45576 2024-09-17T21:31:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704902+0000", 2024-09-17T21:31:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9100000000000004e-07 2024-09-17T21:31:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704905+0000", 2024-09-17T21:31:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6139999999999999e-06 2024-09-17T21:31:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704911+0000", 2024-09-17T21:31:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8619999999999997e-06 2024-09-17T21:31:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704917+0000", 2024-09-17T21:31:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:31:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726681+0000", 2024-09-17T21:31:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021764569000000001 2024-09-17T21:31:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726726+0000", 2024-09-17T21:31:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4218999999999998e-05 2024-09-17T21:31:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726926+0000", 2024-09-17T21:31:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000200466 2024-09-17T21:31:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45577 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.955753376999994, 2024-09-17T21:31:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022269694999999999, 2024-09-17T21:31:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45577 2024-09-17T21:31:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704926+0000", 2024-09-17T21:31:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:31:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704928+0000", 2024-09-17T21:31:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9989999999999998e-06 2024-09-17T21:31:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704934+0000", 2024-09-17T21:31:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:31:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704939+0000", 2024-09-17T21:31:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5909999999999998e-06 2024-09-17T21:31:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726954+0000", 2024-09-17T21:31:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022014885000000001 2024-09-17T21:31:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726995+0000", 2024-09-17T21:31:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1474000000000003e-05 2024-09-17T21:31:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727195+0000", 2024-09-17T21:31:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019968300000000001 2024-09-17T21:31:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45578 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.955732132999998, 2024-09-17T21:31:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022484006000000001, 2024-09-17T21:31:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45578 2024-09-17T21:31:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704947+0000", 2024-09-17T21:31:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1399999999999997e-07 2024-09-17T21:31:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704949+0000", 2024-09-17T21:31:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0959999999999999e-06 2024-09-17T21:31:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704956+0000", 2024-09-17T21:31:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9330000000000004e-06 2024-09-17T21:31:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704961+0000", 2024-09-17T21:31:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6759999999999998e-06 2024-09-17T21:31:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727213+0000", 2024-09-17T21:31:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022252568 2024-09-17T21:31:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727259+0000", 2024-09-17T21:31:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5238000000000003e-05 2024-09-17T21:31:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727430+0000", 2024-09-17T21:31:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000171881 2024-09-17T21:31:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45579 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.955710205000003, 2024-09-17T21:31:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022749080000000001, 2024-09-17T21:31:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45579 2024-09-17T21:31:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704969+0000", 2024-09-17T21:31:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:31:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704971+0000", 2024-09-17T21:31:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.097e-06 2024-09-17T21:31:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704977+0000", 2024-09-17T21:31:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6329999999999999e-06 2024-09-17T21:31:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704983+0000", 2024-09-17T21:31:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:31:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727457+0000", 2024-09-17T21:31:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022474521000000001 2024-09-17T21:31:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727498+0000", 2024-09-17T21:31:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0558e-05 2024-09-17T21:31:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727717+0000", 2024-09-17T21:31:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219805 2024-09-17T21:31:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45652 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.954564067000007, 2024-09-17T21:31:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022100465999999999, 2024-09-17T21:31:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45652 2024-09-17T21:31:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706115+0000", 2024-09-17T21:31:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:31:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706117+0000", 2024-09-17T21:31:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-06 2024-09-17T21:31:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706122+0000", 2024-09-17T21:31:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.288e-06 2024-09-17T21:31:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706136+0000", 2024-09-17T21:31:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3409e-05 2024-09-17T21:31:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728112+0000", 2024-09-17T21:31:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021976724999999999 2024-09-17T21:31:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728151+0000", 2024-09-17T21:31:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8318999999999997e-05 2024-09-17T21:31:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728215+0000", 2024-09-17T21:31:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4172999999999997e-05 2024-09-17T21:31:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45653 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.954534856999999, 2024-09-17T21:31:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022249517999999999, 2024-09-17T21:31:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45653 2024-09-17T21:31:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T21:31:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706147+0000", 2024-09-17T21:31:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4430000000000002e-06 2024-09-17T21:31:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706152+0000", 2024-09-17T21:31:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5350000000000004e-06 2024-09-17T21:31:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706157+0000", 2024-09-17T21:31:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8420000000000004e-06 2024-09-17T21:31:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728246+0000", 2024-09-17T21:31:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022089237000000001 2024-09-17T21:31:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728309+0000", 2024-09-17T21:31:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2356999999999999e-05 2024-09-17T21:31:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:31:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4596000000000001e-05 2024-09-17T21:31:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45657 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 36864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.954503592999998, 2024-09-17T21:31:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022218252000000001, 2024-09-17T21:31:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45657 2024-09-17T21:31:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8299999999999997e-07 2024-09-17T21:31:25.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706178+0000", 2024-09-17T21:31:25.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3700000000000002e-06 2024-09-17T21:31:25.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706184+0000", 2024-09-17T21:31:25.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.144e-06 2024-09-17T21:31:25.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706190+0000", 2024-09-17T21:31:25.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8590000000000001e-06 2024-09-17T21:31:25.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728240+0000", 2024-09-17T21:31:25.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022049804999999999 2024-09-17T21:31:25.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728307+0000", 2024-09-17T21:31:25.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7477999999999993e-05 2024-09-17T21:31:25.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:31:25.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6112999999999994e-05 2024-09-17T21:31:25.704 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.704 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.704 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.704 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45658 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:25.704 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:25.704 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.954480473000004, 2024-09-17T21:31:25.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022376455, 2024-09-17T21:31:25.704 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.704 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.705 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:25.705 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:25.705 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:25.705 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:25.705 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45658 2024-09-17T21:31:25.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.705 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:25.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:25.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:25.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:31:25.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706200+0000", 2024-09-17T21:31:25.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:31:25.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706206+0000", 2024-09-17T21:31:25.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7039999999999999e-06 2024-09-17T21:31:25.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706211+0000", 2024-09-17T21:31:25.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9429999999999999e-06 2024-09-17T21:31:25.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728418+0000", 2024-09-17T21:31:25.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022206760999999998 2024-09-17T21:31:25.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728497+0000", 2024-09-17T21:31:25.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9493999999999995e-05 2024-09-17T21:31:25.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728574+0000", 2024-09-17T21:31:25.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7180000000000003e-05 2024-09-17T21:31:25.708 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.708 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.708 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.708 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45707 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 212992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:25.709 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:25.709 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.952776407000002, 2024-09-17T21:31:25.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022235550999999999, 2024-09-17T21:31:25.709 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.709 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.709 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:25.709 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:25.709 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:25.709 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:25.709 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45707 2024-09-17T21:31:25.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.709 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:25.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:25.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:25.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:31:25.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707905+0000", 2024-09-17T21:31:25.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2120000000000002e-06 2024-09-17T21:31:25.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707911+0000", 2024-09-17T21:31:25.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1929999999999998e-06 2024-09-17T21:31:25.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707915+0000", 2024-09-17T21:31:25.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:31:25.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729897+0000", 2024-09-17T21:31:25.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021981186 2024-09-17T21:31:25.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729944+0000", 2024-09-17T21:31:25.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571000000000002e-05 2024-09-17T21:31:25.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730138+0000", 2024-09-17T21:31:25.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000193451 2024-09-17T21:31:25.713 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.713 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.713 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.713 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45708 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:25.713 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:25.713 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.952756274999999, 2024-09-17T21:31:25.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022533742999999998, 2024-09-17T21:31:25.713 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.713 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.713 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:25.714 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:25.714 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:25.714 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:25.714 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45708 2024-09-17T21:31:25.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.714 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:25.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:25.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707923+0000", 2024-09-17T21:31:25.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:31:25.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707932+0000", 2024-09-17T21:31:25.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6430000000000007e-06 2024-09-17T21:31:25.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707939+0000", 2024-09-17T21:31:25.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3539999999999996e-06 2024-09-17T21:31:25.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707944+0000", 2024-09-17T21:31:25.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3249999999999998e-06 2024-09-17T21:31:25.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730176+0000", 2024-09-17T21:31:25.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022232393 2024-09-17T21:31:25.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730263+0000", 2024-09-17T21:31:25.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6638999999999995e-05 2024-09-17T21:31:25.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730456+0000", 2024-09-17T21:31:25.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019304899999999999 2024-09-17T21:31:25.717 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.717 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.718 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.718 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45752 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:25.718 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:25.718 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.952501054999999, 2024-09-17T21:31:25.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022426878000000001, 2024-09-17T21:31:25.718 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.718 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.718 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:25.718 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:25.718 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:25.718 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:25.718 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45752 2024-09-17T21:31:25.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.719 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:25.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:25.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708178+0000", 2024-09-17T21:31:25.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:31:25.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708181+0000", 2024-09-17T21:31:25.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.909e-06 2024-09-17T21:31:25.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708187+0000", 2024-09-17T21:31:25.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4999999999999996e-06 2024-09-17T21:31:25.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708193+0000", 2024-09-17T21:31:25.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:31:25.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730484+0000", 2024-09-17T21:31:25.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022291765000000002 2024-09-17T21:31:25.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730541+0000", 2024-09-17T21:31:25.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6483000000000001e-05 2024-09-17T21:31:25.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730604+0000", 2024-09-17T21:31:25.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3429000000000005e-05 2024-09-17T21:31:25.722 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.722 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.722 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.722 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45761 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:25.722 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:25.722 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.952473144999999, 2024-09-17T21:31:25.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022614408999999999, 2024-09-17T21:31:25.722 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.722 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.722 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:25.722 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:25.722 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:25.723 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:25.723 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45761 2024-09-17T21:31:25.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.723 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:25.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:25.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708206+0000", 2024-09-17T21:31:25.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4700000000000001e-07 2024-09-17T21:31:25.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708208+0000", 2024-09-17T21:31:25.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1509999999999998e-06 2024-09-17T21:31:25.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708214+0000", 2024-09-17T21:31:25.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.49e-06 2024-09-17T21:31:25.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708219+0000", 2024-09-17T21:31:25.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8119999999999998e-06 2024-09-17T21:31:25.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730641+0000", 2024-09-17T21:31:25.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022422765000000001 2024-09-17T21:31:25.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730731+0000", 2024-09-17T21:31:25.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9598999999999994e-05 2024-09-17T21:31:25.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730820+0000", 2024-09-17T21:31:25.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8844999999999994e-05 2024-09-17T21:31:25.726 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.726 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.726 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.726 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45785 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:25.726 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:25.726 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.951251357999993, 2024-09-17T21:31:25.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022044113000000001, 2024-09-17T21:31:25.727 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.727 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.727 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:25.727 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:25.727 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:25.727 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:25.727 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45785 2024-09-17T21:31:25.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.727 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:25.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:25.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709428+0000", 2024-09-17T21:31:25.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:31:25.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709430+0000", 2024-09-17T21:31:25.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:31:25.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709451+0000", 2024-09-17T21:31:25.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1372000000000001e-05 2024-09-17T21:31:25.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709472+0000", 2024-09-17T21:31:25.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1007999999999999e-05 2024-09-17T21:31:25.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731293+0000", 2024-09-17T21:31:25.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021821320000000002 2024-09-17T21:31:25.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731378+0000", 2024-09-17T21:31:25.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5059e-05 2024-09-17T21:31:25.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731471+0000", 2024-09-17T21:31:25.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2867000000000004e-05 2024-09-17T21:31:25.730 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.730 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.730 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.730 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45786 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:25.730 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:25.730 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.951192560999999, 2024-09-17T21:31:25.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022181821000000001, 2024-09-17T21:31:25.731 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.731 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.731 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:25.731 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:25.731 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:25.731 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:25.731 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45786 2024-09-17T21:31:25.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.731 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:25.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:25.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709487+0000", 2024-09-17T21:31:25.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1299999999999999e-07 2024-09-17T21:31:25.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709491+0000", 2024-09-17T21:31:25.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2089999999999999e-06 2024-09-17T21:31:25.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709498+0000", 2024-09-17T21:31:25.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3980000000000002e-06 2024-09-17T21:31:25.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709504+0000", 2024-09-17T21:31:25.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7219999999999996e-06 2024-09-17T21:31:25.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731499+0000", 2024-09-17T21:31:25.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021995297 2024-09-17T21:31:25.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731591+0000", 2024-09-17T21:31:25.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1373999999999996e-05 2024-09-17T21:31:25.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731668+0000", 2024-09-17T21:31:25.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7107999999999994e-05 2024-09-17T21:31:25.734 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.734 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.735 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.735 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50202 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'139, mlcod=45'137)", 2024-09-17T21:31:25.735 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:25.735 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 74.536456692000002, 2024-09-17T21:31:25.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059659441000000001, 2024-09-17T21:31:25.735 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.735 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.735 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:25.735 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:25.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:25.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124224+0000", 2024-09-17T21:31:25.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5000000000000002e-06 2024-09-17T21:31:25.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147385+0000", 2024-09-17T21:31:25.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023160544000000002 2024-09-17T21:31:25.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147411+0000", 2024-09-17T21:31:25.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6095999999999999e-05 2024-09-17T21:31:25.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147438+0000", 2024-09-17T21:31:25.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6982999999999999e-05 2024-09-17T21:31:25.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147504+0000", 2024-09-17T21:31:25.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6156999999999997e-05 2024-09-17T21:31:25.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147531+0000", 2024-09-17T21:31:25.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6741999999999999e-05 2024-09-17T21:31:25.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:25.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183820+0000", 2024-09-17T21:31:25.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036289090000000003 2024-09-17T21:31:25.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183881+0000", 2024-09-17T21:31:25.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1328999999999995e-05 2024-09-17T21:31:25.739 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.739 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.739 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.739 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50203 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'141, mlcod=45'137)", 2024-09-17T21:31:25.739 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:25.740 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 74.499193949000002, 2024-09-17T21:31:25.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023138823999999999, 2024-09-17T21:31:25.740 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.740 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.740 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:25.740 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:25.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:25.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161486+0000", 2024-09-17T21:31:25.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5290000000000001e-06 2024-09-17T21:31:25.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161496+0000", 2024-09-17T21:31:25.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0063e-05 2024-09-17T21:31:25.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161514+0000", 2024-09-17T21:31:25.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7728e-05 2024-09-17T21:31:25.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161524+0000", 2024-09-17T21:31:25.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0339e-05 2024-09-17T21:31:25.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170872+0000", 2024-09-17T21:31:25.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093473640000000004 2024-09-17T21:31:25.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170899+0000", 2024-09-17T21:31:25.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7321000000000002e-05 2024-09-17T21:31:25.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:25.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184580+0000", 2024-09-17T21:31:25.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013680988999999999 2024-09-17T21:31:25.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184623+0000", 2024-09-17T21:31:25.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3491000000000001e-05 2024-09-17T21:31:25.744 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.744 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.744 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.744 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:88193 3.9 3:97498da5:::rbd_data.1120ea662e44.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-17T21:31:25.744 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:25.745 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.337013038999999, 2024-09-17T21:31:25.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026278261000000001, 2024-09-17T21:31:25.745 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.745 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.745 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:25.745 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:25.745 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:25.745 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:25.745 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 88193 2024-09-17T21:31:25.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.745 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:25.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:25.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323666+0000", 2024-09-17T21:31:25.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:31:25.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323670+0000", 2024-09-17T21:31:25.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.771e-06 2024-09-17T21:31:25.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323680+0000", 2024-09-17T21:31:25.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9469999999999998e-06 2024-09-17T21:31:25.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323685+0000", 2024-09-17T21:31:25.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9919999999999998e-06 2024-09-17T21:31:25.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323728+0000", 2024-09-17T21:31:25.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2811000000000001e-05 2024-09-17T21:31:25.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323776+0000", 2024-09-17T21:31:25.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8310000000000003e-05 2024-09-17T21:31:25.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:25.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.328583+0000", 2024-09-17T21:31:25.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048072899999999997 2024-09-17T21:31:25.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:25.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349861+0000", 2024-09-17T21:31:25.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021277695999999999 2024-09-17T21:31:25.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:25.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349877+0000", 2024-09-17T21:31:25.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:31:25.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:25.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349911+0000", 2024-09-17T21:31:25.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3844999999999999e-05 2024-09-17T21:31:25.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349944+0000", 2024-09-17T21:31:25.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2663000000000002e-05 2024-09-17T21:31:25.750 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.750 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.750 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.750 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:101384 3.9 3:90f2a32a:::rbd_data.11206b8c0882.0000000000000178:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T21:31:25.750 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:25.750 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.334143439999998, 2024-09-17T21:31:25.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025286194000000001, 2024-09-17T21:31:25.751 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:25.751 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:25.751 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:25.751 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:25.751 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:25.751 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:25.751 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 101384 2024-09-17T21:31:25.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.751 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:25.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:25.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:25.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:25.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:25.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:25.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:25.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326536+0000", 2024-09-17T21:31:25.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4799999999999997e-07 2024-09-17T21:31:25.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:25.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326541+0000", 2024-09-17T21:31:25.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8489999999999997e-06 2024-09-17T21:31:25.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:25.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326552+0000", 2024-09-17T21:31:25.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0912e-05 2024-09-17T21:31:25.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:25.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326557+0000", 2024-09-17T21:31:25.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0359999999999997e-06 2024-09-17T21:31:25.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:25.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326603+0000", 2024-09-17T21:31:25.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6428999999999999e-05 2024-09-17T21:31:25.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:25.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326686+0000", 2024-09-17T21:31:25.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2521999999999994e-05 2024-09-17T21:31:25.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:25.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.332698+0000", 2024-09-17T21:31:25.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060124599999999999 2024-09-17T21:31:25.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:25.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351744+0000", 2024-09-17T21:31:25.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019045838999999998 2024-09-17T21:31:25.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:25.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351754+0000", 2024-09-17T21:31:25.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0380999999999999e-05 2024-09-17T21:31:25.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:25.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351791+0000", 2024-09-17T21:31:25.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6678999999999999e-05 2024-09-17T21:31:25.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:25.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:25.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:25.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351821+0000", 2024-09-17T21:31:25.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0139000000000001e-05 2024-09-17T21:31:25.756 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.756 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.756 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.756 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:25.756 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:25.756 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:25.758 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:31:25.758 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:25.755+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:25.859 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:25.856+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:25.861 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:25.861 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:25.861 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:25.862 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:25.870 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:31:25.960 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:25.957+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:25.972 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:25.972 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:25.973 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:25.973 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:25.973 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:25.982 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:31:26.060 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:26.058+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:26.087 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:26.087 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:26.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10799 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.01923008, 2024-09-17T21:31:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039122745, 2024-09-17T21:31:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10799 2024-09-17T21:31:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065469+0000", 2024-09-17T21:31:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:31:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065477+0000", 2024-09-17T21:31:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1319999999999994e-06 2024-09-17T21:31:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065483+0000", 2024-09-17T21:31:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2040000000000002e-06 2024-09-17T21:31:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065487+0000", 2024-09-17T21:31:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.42e-06 2024-09-17T21:31:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099984+0000", 2024-09-17T21:31:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034496484000000001 2024-09-17T21:31:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100065+0000", 2024-09-17T21:31:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1200999999999997e-05 2024-09-17T21:31:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100422+0000", 2024-09-17T21:31:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035657700000000001 2024-09-17T21:31:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104214+0000", 2024-09-17T21:31:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003792045 2024-09-17T21:31:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104505+0000", 2024-09-17T21:31:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029141399999999999 2024-09-17T21:31:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104556+0000", 2024-09-17T21:31:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1388999999999999e-05 2024-09-17T21:31:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104591+0000", 2024-09-17T21:31:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4455999999999997e-05 2024-09-17T21:31:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12187 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.945137349000007, 2024-09-17T21:31:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038825750999999999, 2024-09-17T21:31:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12187 2024-09-17T21:31:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:31:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139563+0000", 2024-09-17T21:31:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.624e-06 2024-09-17T21:31:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139569+0000", 2024-09-17T21:31:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1090000000000003e-06 2024-09-17T21:31:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139573+0000", 2024-09-17T21:31:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4429999999999996e-06 2024-09-17T21:31:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178206+0000", 2024-09-17T21:31:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038632376000000003 2024-09-17T21:31:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178282+0000", 2024-09-17T21:31:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6076999999999997e-05 2024-09-17T21:31:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178387+0000", 2024-09-17T21:31:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104682 2024-09-17T21:31:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12188 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.945119168000005, 2024-09-17T21:31:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039053826999999999, 2024-09-17T21:31:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12188 2024-09-17T21:31:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139746+0000", 2024-09-17T21:31:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000167281 2024-09-17T21:31:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139749+0000", 2024-09-17T21:31:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9660000000000001e-06 2024-09-17T21:31:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139757+0000", 2024-09-17T21:31:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8029999999999997e-06 2024-09-17T21:31:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139771+0000", 2024-09-17T21:31:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3541e-05 2024-09-17T21:31:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178415+0000", 2024-09-17T21:31:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038644803999999998 2024-09-17T21:31:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178517+0000", 2024-09-17T21:31:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101298 2024-09-17T21:31:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178633+0000", 2024-09-17T21:31:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116134 2024-09-17T21:31:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12189 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.944918690999998, 2024-09-17T21:31:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039135991000000002, 2024-09-17T21:31:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12189 2024-09-17T21:31:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139780+0000", 2024-09-17T21:31:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:31:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139782+0000", 2024-09-17T21:31:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1950000000000002e-06 2024-09-17T21:31:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139788+0000", 2024-09-17T21:31:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0639999999999999e-06 2024-09-17T21:31:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139792+0000", 2024-09-17T21:31:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1960000000000001e-06 2024-09-17T21:31:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178671+0000", 2024-09-17T21:31:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038878176 2024-09-17T21:31:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178778+0000", 2024-09-17T21:31:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107278 2024-09-17T21:31:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178915+0000", 2024-09-17T21:31:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013757900000000001 2024-09-17T21:31:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12190 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.944899508000006, 2024-09-17T21:31:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039343781000000001, 2024-09-17T21:31:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12190 2024-09-17T21:31:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139801+0000", 2024-09-17T21:31:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.916e-06 2024-09-17T21:31:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139807+0000", 2024-09-17T21:31:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4540000000000004e-06 2024-09-17T21:31:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139812+0000", 2024-09-17T21:31:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9729999999999997e-06 2024-09-17T21:31:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178960+0000", 2024-09-17T21:31:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039147567000000001 2024-09-17T21:31:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179051+0000", 2024-09-17T21:31:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0754000000000006e-05 2024-09-17T21:31:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179142+0000", 2024-09-17T21:31:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1712999999999994e-05 2024-09-17T21:31:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12191 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.944879650000004, 2024-09-17T21:31:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039464336000000003, 2024-09-17T21:31:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12191 2024-09-17T21:31:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:31:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139821+0000", 2024-09-17T21:31:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1260000000000001e-06 2024-09-17T21:31:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139827+0000", 2024-09-17T21:31:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T21:31:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139840+0000", 2024-09-17T21:31:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2607e-05 2024-09-17T21:31:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179162+0000", 2024-09-17T21:31:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039322863999999999 2024-09-17T21:31:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179204+0000", 2024-09-17T21:31:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1938999999999999e-05 2024-09-17T21:31:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179283+0000", 2024-09-17T21:31:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8547000000000005e-05 2024-09-17T21:31:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12192 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.944851937999999, 2024-09-17T21:31:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039652318999999998, 2024-09-17T21:31:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12192 2024-09-17T21:31:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141458+0000", 2024-09-17T21:31:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001612098 2024-09-17T21:31:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141469+0000", 2024-09-17T21:31:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-17T21:31:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141478+0000", 2024-09-17T21:31:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T21:31:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141485+0000", 2024-09-17T21:31:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4760000000000003e-06 2024-09-17T21:31:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179317+0000", 2024-09-17T21:31:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037831832000000003 2024-09-17T21:31:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179395+0000", 2024-09-17T21:31:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8349000000000005e-05 2024-09-17T21:31:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179499+0000", 2024-09-17T21:31:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103628 2024-09-17T21:31:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12257 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.94266313, 2024-09-17T21:31:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038862751000000001, 2024-09-17T21:31:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12257 2024-09-17T21:31:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T21:31:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142037+0000", 2024-09-17T21:31:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.03e-06 2024-09-17T21:31:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142043+0000", 2024-09-17T21:31:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6319999999999996e-06 2024-09-17T21:31:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142052+0000", 2024-09-17T21:31:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2709999999999998e-06 2024-09-17T21:31:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180794+0000", 2024-09-17T21:31:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038741758000000001 2024-09-17T21:31:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180843+0000", 2024-09-17T21:31:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9021999999999999e-05 2024-09-17T21:31:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180898+0000", 2024-09-17T21:31:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673000000000003e-05 2024-09-17T21:31:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12258 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.942639612999997, 2024-09-17T21:31:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038950717000000003, 2024-09-17T21:31:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12258 2024-09-17T21:31:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199999999999999e-07 2024-09-17T21:31:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142061+0000", 2024-09-17T21:31:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8279999999999999e-06 2024-09-17T21:31:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142067+0000", 2024-09-17T21:31:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5600000000000001e-06 2024-09-17T21:31:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142071+0000", 2024-09-17T21:31:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6240000000000001e-06 2024-09-17T21:31:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180921+0000", 2024-09-17T21:31:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038849366000000003 2024-09-17T21:31:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180961+0000", 2024-09-17T21:31:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0219000000000002e-05 2024-09-17T21:31:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181009+0000", 2024-09-17T21:31:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8437999999999999e-05 2024-09-17T21:31:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12259 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.942612366000006, 2024-09-17T21:31:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039116832999999997, 2024-09-17T21:31:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12259 2024-09-17T21:31:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2200000000000004e-07 2024-09-17T21:31:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142088+0000", 2024-09-17T21:31:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0329999999999998e-06 2024-09-17T21:31:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142095+0000", 2024-09-17T21:31:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4320000000000004e-06 2024-09-17T21:31:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142100+0000", 2024-09-17T21:31:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:31:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181037+0000", 2024-09-17T21:31:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038937601000000002 2024-09-17T21:31:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181114+0000", 2024-09-17T21:31:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6857e-05 2024-09-17T21:31:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181203+0000", 2024-09-17T21:31:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8437999999999996e-05 2024-09-17T21:31:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12260 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.942592078999994, 2024-09-17T21:31:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039204184000000003, 2024-09-17T21:31:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12260 2024-09-17T21:31:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142107+0000", 2024-09-17T21:31:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:31:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142108+0000", 2024-09-17T21:31:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.753e-06 2024-09-17T21:31:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142114+0000", 2024-09-17T21:31:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7660000000000004e-06 2024-09-17T21:31:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142119+0000", 2024-09-17T21:31:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:31:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181216+0000", 2024-09-17T21:31:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039097191000000003 2024-09-17T21:31:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181260+0000", 2024-09-17T21:31:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4731000000000003e-05 2024-09-17T21:31:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181310+0000", 2024-09-17T21:31:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9818000000000003e-05 2024-09-17T21:31:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12261 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.942573754999998, 2024-09-17T21:31:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039292200999999999, 2024-09-17T21:31:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12261 2024-09-17T21:31:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142125+0000", 2024-09-17T21:31:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:31:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142127+0000", 2024-09-17T21:31:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.792e-06 2024-09-17T21:31:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142132+0000", 2024-09-17T21:31:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3979999999999998e-06 2024-09-17T21:31:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142136+0000", 2024-09-17T21:31:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.211e-06 2024-09-17T21:31:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181324+0000", 2024-09-17T21:31:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039187560000000003 2024-09-17T21:31:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181363+0000", 2024-09-17T21:31:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9375000000000002e-05 2024-09-17T21:31:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181417+0000", 2024-09-17T21:31:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3372000000000003e-05 2024-09-17T21:31:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12262 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.942555962, 2024-09-17T21:31:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039384875, 2024-09-17T21:31:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12262 2024-09-17T21:31:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142143+0000", 2024-09-17T21:31:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:31:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142148+0000", 2024-09-17T21:31:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4360000000000001e-06 2024-09-17T21:31:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142154+0000", 2024-09-17T21:31:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8719999999999999e-06 2024-09-17T21:31:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142158+0000", 2024-09-17T21:31:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.476e-06 2024-09-17T21:31:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181436+0000", 2024-09-17T21:31:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039277919000000001 2024-09-17T21:31:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181480+0000", 2024-09-17T21:31:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3649999999999997e-05 2024-09-17T21:31:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181527+0000", 2024-09-17T21:31:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7131e-05 2024-09-17T21:31:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12263 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1079296~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.942533526999995, 2024-09-17T21:31:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039473339000000003, 2024-09-17T21:31:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12263 2024-09-17T21:31:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:31:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142167+0000", 2024-09-17T21:31:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.81e-06 2024-09-17T21:31:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142172+0000", 2024-09-17T21:31:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:31:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142177+0000", 2024-09-17T21:31:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T21:31:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181553+0000", 2024-09-17T21:31:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039376130000000002 2024-09-17T21:31:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181592+0000", 2024-09-17T21:31:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9153999999999998e-05 2024-09-17T21:31:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181638+0000", 2024-09-17T21:31:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5864e-05 2024-09-17T21:31:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12278 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1095680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.942515552000003, 2024-09-17T21:31:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039454222999999997, 2024-09-17T21:31:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12278 2024-09-17T21:31:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:31:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142185+0000", 2024-09-17T21:31:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.697e-06 2024-09-17T21:31:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142191+0000", 2024-09-17T21:31:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:31:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142196+0000", 2024-09-17T21:31:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4829999999999999e-06 2024-09-17T21:31:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181548+0000", 2024-09-17T21:31:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039351200000000003 2024-09-17T21:31:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181591+0000", 2024-09-17T21:31:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3492000000000003e-05 2024-09-17T21:31:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181637+0000", 2024-09-17T21:31:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5847000000000003e-05 2024-09-17T21:31:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12279 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.942492263000005, 2024-09-17T21:31:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039539801999999999, 2024-09-17T21:31:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12279 2024-09-17T21:31:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.89e-07 2024-09-17T21:31:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142208+0000", 2024-09-17T21:31:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7579999999999999e-06 2024-09-17T21:31:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142214+0000", 2024-09-17T21:31:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:31:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142218+0000", 2024-09-17T21:31:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3429999999999998e-06 2024-09-17T21:31:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181653+0000", 2024-09-17T21:31:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039435322000000002 2024-09-17T21:31:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181698+0000", 2024-09-17T21:31:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4978999999999997e-05 2024-09-17T21:31:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181746+0000", 2024-09-17T21:31:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571999999999998e-05 2024-09-17T21:31:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12280 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1124352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.942474743999995, 2024-09-17T21:31:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039704358000000002, 2024-09-17T21:31:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12280 2024-09-17T21:31:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142224+0000", 2024-09-17T21:31:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:31:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142226+0000", 2024-09-17T21:31:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:31:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142231+0000", 2024-09-17T21:31:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:31:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142240+0000", 2024-09-17T21:31:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4590000000000004e-06 2024-09-17T21:31:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181774+0000", 2024-09-17T21:31:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039533833999999997 2024-09-17T21:31:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181859+0000", 2024-09-17T21:31:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4328999999999998e-05 2024-09-17T21:31:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181928+0000", 2024-09-17T21:31:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9139000000000003e-05 2024-09-17T21:31:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12281 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1136640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.942451837999997, 2024-09-17T21:31:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039787799999999998, 2024-09-17T21:31:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12281 2024-09-17T21:31:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142309+0000", 2024-09-17T21:31:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2323999999999995e-05 2024-09-17T21:31:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142317+0000", 2024-09-17T21:31:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1859999999999992e-06 2024-09-17T21:31:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142324+0000", 2024-09-17T21:31:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4699999999999996e-06 2024-09-17T21:31:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142330+0000", 2024-09-17T21:31:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4650000000000002e-06 2024-09-17T21:31:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181948+0000", 2024-09-17T21:31:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039618056999999998 2024-09-17T21:31:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181986+0000", 2024-09-17T21:31:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8442999999999998e-05 2024-09-17T21:31:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182034+0000", 2024-09-17T21:31:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7855000000000002e-05 2024-09-17T21:31:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12282 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1150976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.942359608999993, 2024-09-17T21:31:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039871919999999998, 2024-09-17T21:31:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12282 2024-09-17T21:31:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:31:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142341+0000", 2024-09-17T21:31:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.232e-06 2024-09-17T21:31:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142348+0000", 2024-09-17T21:31:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7320000000000001e-06 2024-09-17T21:31:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142353+0000", 2024-09-17T21:31:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:31:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182062+0000", 2024-09-17T21:31:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039709739000000001 2024-09-17T21:31:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182128+0000", 2024-09-17T21:31:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5739000000000002e-05 2024-09-17T21:31:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182210+0000", 2024-09-17T21:31:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:31:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12283 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.942339197999999, 2024-09-17T21:31:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039896434000000001, 2024-09-17T21:31:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:31:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12283 2024-09-17T21:31:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144852+0000", 2024-09-17T21:31:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024934079999999999 2024-09-17T21:31:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144856+0000", 2024-09-17T21:31:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:31:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144864+0000", 2024-09-17T21:31:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0160000000000008e-06 2024-09-17T21:31:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144869+0000", 2024-09-17T21:31:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:31:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182246+0000", 2024-09-17T21:31:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037377207000000003 2024-09-17T21:31:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182255+0000", 2024-09-17T21:31:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1840000000000005e-06 2024-09-17T21:31:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.177 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:26.178 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:31:26.179 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:26.176+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:26.279 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:26.277+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:26.282 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:26.283 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:26.283 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:26.283 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:26.291 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:31:26.381 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:26.378+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:26.395 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:26.395 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:26.395 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:26.395 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:26.395 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:26.404 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:31:26.484 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:26.481+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:26.512 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-17T21:31:26.513 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:31:26.513 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T21:31:26.513 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.216740 2024-09-17T21:31:26.513 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0.2167396127609329 2024-09-17T21:31:26.559 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:26.559 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:26.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:26.559 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:26.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.559 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:36935 3.a 3:54c8759b:::rbd_data.1120e62eea89.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-17T21:31:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:31:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 89.596464423, 2024-09-17T21:31:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020540151999999999, 2024-09-17T21:31:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 36935 2024-09-17T21:31:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:31:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:31:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958910+0000", 2024-09-17T21:31:26.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:31:26.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958918+0000", 2024-09-17T21:31:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6469999999999998e-06 2024-09-17T21:31:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958933+0000", 2024-09-17T21:31:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4791e-05 2024-09-17T21:31:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958940+0000", 2024-09-17T21:31:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-17T21:31:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958990+0000", 2024-09-17T21:31:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9759000000000003e-05 2024-09-17T21:31:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.959039+0000", 2024-09-17T21:31:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9487000000000002e-05 2024-09-17T21:31:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.962098+0000", 2024-09-17T21:31:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003059258 2024-09-17T21:31:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979325+0000", 2024-09-17T21:31:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017226841 2024-09-17T21:31:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979343+0000", 2024-09-17T21:31:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7486000000000002e-05 2024-09-17T21:31:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979401+0000", 2024-09-17T21:31:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7980999999999999e-05 2024-09-17T21:31:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979449+0000", 2024-09-17T21:31:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8639e-05 2024-09-17T21:31:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39447 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 909312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587526522999994, 2024-09-17T21:31:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017419153999999999, 2024-09-17T21:31:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39447 2024-09-17T21:31:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:26.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:26.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:31:26.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967849+0000", 2024-09-17T21:31:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4339999999999999e-06 2024-09-17T21:31:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967853+0000", 2024-09-17T21:31:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:31:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967857+0000", 2024-09-17T21:31:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:31:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985193+0000", 2024-09-17T21:31:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017335973000000001 2024-09-17T21:31:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985228+0000", 2024-09-17T21:31:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5154999999999998e-05 2024-09-17T21:31:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985266+0000", 2024-09-17T21:31:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8448000000000002e-05 2024-09-17T21:31:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39448 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 974848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587513706999999, 2024-09-17T21:31:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017517066000000001, 2024-09-17T21:31:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39448 2024-09-17T21:31:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.72e-07 2024-09-17T21:31:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967861+0000", 2024-09-17T21:31:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.096e-06 2024-09-17T21:31:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967865+0000", 2024-09-17T21:31:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9339999999999999e-06 2024-09-17T21:31:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967868+0000", 2024-09-17T21:31:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.508e-06 2024-09-17T21:31:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985285+0000", 2024-09-17T21:31:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017417208 2024-09-17T21:31:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985339+0000", 2024-09-17T21:31:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3931000000000003e-05 2024-09-17T21:31:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985377+0000", 2024-09-17T21:31:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8016999999999999e-05 2024-09-17T21:31:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39449 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587501596999999, 2024-09-17T21:31:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017595137, 2024-09-17T21:31:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39449 2024-09-17T21:31:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2700000000000001e-07 2024-09-17T21:31:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967873+0000", 2024-09-17T21:31:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.164e-06 2024-09-17T21:31:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967877+0000", 2024-09-17T21:31:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.23e-06 2024-09-17T21:31:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967880+0000", 2024-09-17T21:31:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.286e-06 2024-09-17T21:31:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985397+0000", 2024-09-17T21:31:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017516982 2024-09-17T21:31:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985431+0000", 2024-09-17T21:31:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3899999999999997e-05 2024-09-17T21:31:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985467+0000", 2024-09-17T21:31:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6347999999999998e-05 2024-09-17T21:31:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39450 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1105920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:31:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587481339999997, 2024-09-17T21:31:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017677824000000002, 2024-09-17T21:31:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39450 2024-09-17T21:31:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:31:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:31:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967893+0000", 2024-09-17T21:31:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9100000000000004e-07 2024-09-17T21:31:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967895+0000", 2024-09-17T21:31:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:31:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967901+0000", 2024-09-17T21:31:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.083e-06 2024-09-17T21:31:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967906+0000", 2024-09-17T21:31:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:31:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985493+0000", 2024-09-17T21:31:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017587565999999999 2024-09-17T21:31:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985531+0000", 2024-09-17T21:31:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7490000000000002e-05 2024-09-17T21:31:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985570+0000", 2024-09-17T21:31:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9205000000000002e-05 2024-09-17T21:31:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39451 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1171456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587460960000001, 2024-09-17T21:31:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017752337999999999, 2024-09-17T21:31:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39451 2024-09-17T21:31:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:31:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967915+0000", 2024-09-17T21:31:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780000000000001e-06 2024-09-17T21:31:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967922+0000", 2024-09-17T21:31:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7020000000000004e-06 2024-09-17T21:31:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967926+0000", 2024-09-17T21:31:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.809e-06 2024-09-17T21:31:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985590+0000", 2024-09-17T21:31:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017663913999999999 2024-09-17T21:31:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985627+0000", 2024-09-17T21:31:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6896e-05 2024-09-17T21:31:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985665+0000", 2024-09-17T21:31:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7793999999999998e-05 2024-09-17T21:31:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39452 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1236992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587441018000007, 2024-09-17T21:31:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017829602999999999, 2024-09-17T21:31:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39452 2024-09-17T21:31:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:31:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967947+0000", 2024-09-17T21:31:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4280999999999999e-05 2024-09-17T21:31:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967954+0000", 2024-09-17T21:31:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3249999999999996e-06 2024-09-17T21:31:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967959+0000", 2024-09-17T21:31:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2299999999999999e-06 2024-09-17T21:31:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985687+0000", 2024-09-17T21:31:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017727844999999999 2024-09-17T21:31:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985725+0000", 2024-09-17T21:31:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8243e-05 2024-09-17T21:31:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985762+0000", 2024-09-17T21:31:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7023000000000001e-05 2024-09-17T21:31:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39453 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.601 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:31:26.601 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587404530000001, 2024-09-17T21:31:26.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017893793000000002, 2024-09-17T21:31:26.601 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.601 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.601 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.602 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39453 2024-09-17T21:31:26.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.602 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:31:26.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:31:26.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967970+0000", 2024-09-17T21:31:26.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:31:26.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967972+0000", 2024-09-17T21:31:26.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0229999999999999e-06 2024-09-17T21:31:26.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967978+0000", 2024-09-17T21:31:26.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0430000000000004e-06 2024-09-17T21:31:26.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967982+0000", 2024-09-17T21:31:26.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2570000000000001e-06 2024-09-17T21:31:26.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985782+0000", 2024-09-17T21:31:26.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017799683 2024-09-17T21:31:26.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985818+0000", 2024-09-17T21:31:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6304999999999999e-05 2024-09-17T21:31:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985863+0000", 2024-09-17T21:31:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4953999999999997e-05 2024-09-17T21:31:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39454 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1368064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:31:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587385201999993, 2024-09-17T21:31:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017968886, 2024-09-17T21:31:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39454 2024-09-17T21:31:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.608 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:31:26.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:31:26.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967989+0000", 2024-09-17T21:31:26.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.15e-07 2024-09-17T21:31:26.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967991+0000", 2024-09-17T21:31:26.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.945e-06 2024-09-17T21:31:26.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967997+0000", 2024-09-17T21:31:26.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0190000000000003e-06 2024-09-17T21:31:26.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968001+0000", 2024-09-17T21:31:26.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6650000000000002e-06 2024-09-17T21:31:26.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985883+0000", 2024-09-17T21:31:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017881971999999999 2024-09-17T21:31:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985920+0000", 2024-09-17T21:31:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-05 2024-09-17T21:31:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985957+0000", 2024-09-17T21:31:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7370000000000003e-05 2024-09-17T21:31:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.612 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.612 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39455 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.612 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:26.612 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587366028000005, 2024-09-17T21:31:26.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018044437999999999, 2024-09-17T21:31:26.612 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.612 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.612 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.612 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.612 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39455 2024-09-17T21:31:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:31:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968010+0000", 2024-09-17T21:31:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.886e-06 2024-09-17T21:31:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968017+0000", 2024-09-17T21:31:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:31:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968021+0000", 2024-09-17T21:31:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850000000000004e-06 2024-09-17T21:31:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985977+0000", 2024-09-17T21:31:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017956368 2024-09-17T21:31:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986015+0000", 2024-09-17T21:31:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7610000000000001e-05 2024-09-17T21:31:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986052+0000", 2024-09-17T21:31:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7109e-05 2024-09-17T21:31:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39456 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1499136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587345728000003, 2024-09-17T21:31:26.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018114514000000002, 2024-09-17T21:31:26.617 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.617 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.617 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.617 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.617 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.617 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.617 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39456 2024-09-17T21:31:26.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.617 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-17T21:31:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968030+0000", 2024-09-17T21:31:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0049999999999999e-06 2024-09-17T21:31:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968036+0000", 2024-09-17T21:31:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0129999999999999e-06 2024-09-17T21:31:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968041+0000", 2024-09-17T21:31:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8300000000000003e-06 2024-09-17T21:31:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986071+0000", 2024-09-17T21:31:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018029620999999999 2024-09-17T21:31:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986106+0000", 2024-09-17T21:31:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4867999999999999e-05 2024-09-17T21:31:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986142+0000", 2024-09-17T21:31:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6566000000000002e-05 2024-09-17T21:31:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39457 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1564672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587325911999997, 2024-09-17T21:31:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018188644, 2024-09-17T21:31:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39457 2024-09-17T21:31:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7199999999999999e-07 2024-09-17T21:31:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968050+0000", 2024-09-17T21:31:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1110000000000002e-06 2024-09-17T21:31:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968057+0000", 2024-09-17T21:31:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6170000000000003e-06 2024-09-17T21:31:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968062+0000", 2024-09-17T21:31:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:31:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986163+0000", 2024-09-17T21:31:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018101435999999999 2024-09-17T21:31:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986199+0000", 2024-09-17T21:31:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.591e-05 2024-09-17T21:31:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986236+0000", 2024-09-17T21:31:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7416999999999998e-05 2024-09-17T21:31:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39458 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1630208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587298971999999, 2024-09-17T21:31:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018255856000000001, 2024-09-17T21:31:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39458 2024-09-17T21:31:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:31:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968077+0000", 2024-09-17T21:31:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.779e-06 2024-09-17T21:31:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968081+0000", 2024-09-17T21:31:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.532e-06 2024-09-17T21:31:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968085+0000", 2024-09-17T21:31:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999999e-06 2024-09-17T21:31:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986256+0000", 2024-09-17T21:31:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018171273000000002 2024-09-17T21:31:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986293+0000", 2024-09-17T21:31:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6534e-05 2024-09-17T21:31:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986330+0000", 2024-09-17T21:31:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7536000000000001e-05 2024-09-17T21:31:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39459 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1695744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:31:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587284393999994, 2024-09-17T21:31:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334493, 2024-09-17T21:31:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39459 2024-09-17T21:31:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:31:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:31:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968090+0000", 2024-09-17T21:31:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:31:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968091+0000", 2024-09-17T21:31:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3319999999999999e-06 2024-09-17T21:31:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968094+0000", 2024-09-17T21:31:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2590000000000001e-06 2024-09-17T21:31:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968097+0000", 2024-09-17T21:31:26.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:31:26.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986350+0000", 2024-09-17T21:31:26.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018253364000000001 2024-09-17T21:31:26.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986387+0000", 2024-09-17T21:31:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6350999999999998e-05 2024-09-17T21:31:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986424+0000", 2024-09-17T21:31:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6897999999999998e-05 2024-09-17T21:31:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.638 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.640 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39460 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1761280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:31:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587273257999996, 2024-09-17T21:31:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018415918999999999, 2024-09-17T21:31:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39460 2024-09-17T21:31:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:31:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:31:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968101+0000", 2024-09-17T21:31:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T21:31:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968102+0000", 2024-09-17T21:31:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1689999999999999e-06 2024-09-17T21:31:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968105+0000", 2024-09-17T21:31:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:31:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968108+0000", 2024-09-17T21:31:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6630000000000002e-06 2024-09-17T21:31:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986442+0000", 2024-09-17T21:31:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334487 2024-09-17T21:31:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986479+0000", 2024-09-17T21:31:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6557000000000003e-05 2024-09-17T21:31:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986516+0000", 2024-09-17T21:31:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7648999999999998e-05 2024-09-17T21:31:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39461 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1826816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587262498000001, 2024-09-17T21:31:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018499768, 2024-09-17T21:31:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39461 2024-09-17T21:31:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4699999999999998e-07 2024-09-17T21:31:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968112+0000", 2024-09-17T21:31:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-06 2024-09-17T21:31:26.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968116+0000", 2024-09-17T21:31:26.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1080000000000001e-06 2024-09-17T21:31:26.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968122+0000", 2024-09-17T21:31:26.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1070000000000001e-06 2024-09-17T21:31:26.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986537+0000", 2024-09-17T21:31:26.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018414834000000001 2024-09-17T21:31:26.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986572+0000", 2024-09-17T21:31:26.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5726000000000003e-05 2024-09-17T21:31:26.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986611+0000", 2024-09-17T21:31:26.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8584000000000002e-05 2024-09-17T21:31:26.650 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.650 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.651 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.651 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39462 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1892352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.651 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:31:26.651 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587248134000006, 2024-09-17T21:31:26.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018583252000000001, 2024-09-17T21:31:26.651 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.651 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.652 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.652 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.652 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.652 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.652 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39462 2024-09-17T21:31:26.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.652 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:31:26.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:31:26.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968126+0000", 2024-09-17T21:31:26.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:26.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968127+0000", 2024-09-17T21:31:26.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.229e-06 2024-09-17T21:31:26.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968131+0000", 2024-09-17T21:31:26.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8010000000000001e-06 2024-09-17T21:31:26.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968134+0000", 2024-09-17T21:31:26.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8880000000000001e-06 2024-09-17T21:31:26.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986634+0000", 2024-09-17T21:31:26.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018500501999999999 2024-09-17T21:31:26.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986671+0000", 2024-09-17T21:31:26.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6881000000000001e-05 2024-09-17T21:31:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986709+0000", 2024-09-17T21:31:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7546999999999998e-05 2024-09-17T21:31:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39463 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1957888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587236519000001, 2024-09-17T21:31:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018667186999999998, 2024-09-17T21:31:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39463 2024-09-17T21:31:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:31:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968138+0000", 2024-09-17T21:31:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:31:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968142+0000", 2024-09-17T21:31:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.145e-06 2024-09-17T21:31:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968144+0000", 2024-09-17T21:31:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6869999999999999e-06 2024-09-17T21:31:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986728+0000", 2024-09-17T21:31:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018584163000000001 2024-09-17T21:31:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986766+0000", 2024-09-17T21:31:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7987000000000001e-05 2024-09-17T21:31:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986804+0000", 2024-09-17T21:31:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7892000000000002e-05 2024-09-17T21:31:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39464 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:31:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587226209999997, 2024-09-17T21:31:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018753723999999999, 2024-09-17T21:31:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39464 2024-09-17T21:31:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:31:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:31:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968148+0000", 2024-09-17T21:31:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:31:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968152+0000", 2024-09-17T21:31:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3749999999999996e-06 2024-09-17T21:31:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968155+0000", 2024-09-17T21:31:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.371e-06 2024-09-17T21:31:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968158+0000", 2024-09-17T21:31:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T21:31:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986824+0000", 2024-09-17T21:31:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018665779 2024-09-17T21:31:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986863+0000", 2024-09-17T21:31:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9273000000000002e-05 2024-09-17T21:31:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986901+0000", 2024-09-17T21:31:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7978000000000001e-05 2024-09-17T21:31:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39465 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.587210811000006, 2024-09-17T21:31:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018825341999999998, 2024-09-17T21:31:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39465 2024-09-17T21:31:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-17T21:31:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968164+0000", 2024-09-17T21:31:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1289999999999999e-06 2024-09-17T21:31:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968167+0000", 2024-09-17T21:31:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1379999999999999e-06 2024-09-17T21:31:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968170+0000", 2024-09-17T21:31:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:31:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986914+0000", 2024-09-17T21:31:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018743829 2024-09-17T21:31:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986950+0000", 2024-09-17T21:31:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5819999999999999e-05 2024-09-17T21:31:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986988+0000", 2024-09-17T21:31:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8362000000000003e-05 2024-09-17T21:31:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.670 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.670 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:26.670 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:26.670 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:26.671 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:31:26.672 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:26.668+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:26.772 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:26.770+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:26.776 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:26.776 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:26.776 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:26.776 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:26.785 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:31:26.874 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:26.871+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:26.888 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:26.888 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:26.889 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:26.889 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:26.889 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:26.897 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:31:26.974 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:26.971+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:27.002 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:31:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:31:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 119.05949656, 2024-09-17T21:31:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61572000699999996, 2024-09-17T21:31:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:31:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:31:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T21:31:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:31:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:31:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939745+0000", 2024-09-17T21:31:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6899999999999999e-06 2024-09-17T21:31:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939753+0000", 2024-09-17T21:31:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1950000000000003e-06 2024-09-17T21:31:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939767+0000", 2024-09-17T21:31:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3429e-05 2024-09-17T21:31:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939776+0000", 2024-09-17T21:31:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:31:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939858+0000", 2024-09-17T21:31:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2496000000000004e-05 2024-09-17T21:31:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T21:31:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939868+0000", 2024-09-17T21:31:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1910000000000006e-06 2024-09-17T21:31:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940612+0000", 2024-09-17T21:31:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074429599999999998 2024-09-17T21:31:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:31:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940621+0000", 2024-09-17T21:31:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9139999999999997e-06 2024-09-17T21:31:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554006+0000", 2024-09-17T21:31:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61338505099999996 2024-09-17T21:31:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554166+0000", 2024-09-17T21:31:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015982600000000001 2024-09-17T21:31:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:31:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554296+0000", 2024-09-17T21:31:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013002399999999999 2024-09-17T21:31:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554936+0000", 2024-09-17T21:31:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064009400000000001 2024-09-17T21:31:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555389+0000", 2024-09-17T21:31:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045355200000000002 2024-09-17T21:31:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555432+0000", 2024-09-17T21:31:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2477e-05 2024-09-17T21:31:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555463+0000", 2024-09-17T21:31:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1612000000000002e-05 2024-09-17T21:31:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17527 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1005568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:31:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.594268729000007, 2024-09-17T21:31:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046452090000000001, 2024-09-17T21:31:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17527 2024-09-17T21:31:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:31:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:31:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405496+0000", 2024-09-17T21:31:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052498400000000004 2024-09-17T21:31:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405500+0000", 2024-09-17T21:31:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8410000000000002e-06 2024-09-17T21:31:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405511+0000", 2024-09-17T21:31:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0749e-05 2024-09-17T21:31:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405517+0000", 2024-09-17T21:31:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:31:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451275+0000", 2024-09-17T21:31:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045758134999999998 2024-09-17T21:31:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451355+0000", 2024-09-17T21:31:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9453e-05 2024-09-17T21:31:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451423+0000", 2024-09-17T21:31:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8782999999999995e-05 2024-09-17T21:31:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17615 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.589028240999994, 2024-09-17T21:31:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046741309000000002, 2024-09-17T21:31:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17615 2024-09-17T21:31:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:31:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410213+0000", 2024-09-17T21:31:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:31:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410217+0000", 2024-09-17T21:31:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8199999999999998e-06 2024-09-17T21:31:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410220+0000", 2024-09-17T21:31:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6900000000000001e-06 2024-09-17T21:31:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455264+0000", 2024-09-17T21:31:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045044002999999999 2024-09-17T21:31:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455338+0000", 2024-09-17T21:31:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4182000000000002e-05 2024-09-17T21:31:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.456953+0000", 2024-09-17T21:31:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016150089999999999 2024-09-17T21:31:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17616 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 872448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.589016189000006, 2024-09-17T21:31:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946039000000002, 2024-09-17T21:31:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17616 2024-09-17T21:31:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T21:31:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410225+0000", 2024-09-17T21:31:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3459999999999999e-06 2024-09-17T21:31:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410228+0000", 2024-09-17T21:31:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.083e-06 2024-09-17T21:31:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410232+0000", 2024-09-17T21:31:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4419999999999998e-06 2024-09-17T21:31:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457004+0000", 2024-09-17T21:31:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046771873999999998 2024-09-17T21:31:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457084+0000", 2024-09-17T21:31:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0506000000000005e-05 2024-09-17T21:31:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457170+0000", 2024-09-17T21:31:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5494000000000005e-05 2024-09-17T21:31:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17617 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.589004786999993, 2024-09-17T21:31:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047084168000000003, 2024-09-17T21:31:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17617 2024-09-17T21:31:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-17T21:31:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410237+0000", 2024-09-17T21:31:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1960000000000001e-06 2024-09-17T21:31:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410246+0000", 2024-09-17T21:31:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7880000000000001e-06 2024-09-17T21:31:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410250+0000", 2024-09-17T21:31:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.518e-06 2024-09-17T21:31:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457209+0000", 2024-09-17T21:31:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046959283999999997 2024-09-17T21:31:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457264+0000", 2024-09-17T21:31:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673999999999999e-05 2024-09-17T21:31:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457319+0000", 2024-09-17T21:31:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5497000000000001e-05 2024-09-17T21:31:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17618 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 899072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.588985343000004, 2024-09-17T21:31:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047204208999999997, 2024-09-17T21:31:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17618 2024-09-17T21:31:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8799999999999998e-07 2024-09-17T21:31:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410256+0000", 2024-09-17T21:31:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T21:31:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410260+0000", 2024-09-17T21:31:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6500000000000002e-06 2024-09-17T21:31:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410263+0000", 2024-09-17T21:31:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7089999999999998e-06 2024-09-17T21:31:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457355+0000", 2024-09-17T21:31:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047092756999999999 2024-09-17T21:31:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457407+0000", 2024-09-17T21:31:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1808999999999998e-05 2024-09-17T21:31:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457459+0000", 2024-09-17T21:31:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1684000000000002e-05 2024-09-17T21:31:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17619 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 913408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:27.037 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.588974105000005, 2024-09-17T21:31:27.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047293443999999997, 2024-09-17T21:31:27.037 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.037 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.037 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:27.037 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.037 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.037 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.037 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17619 2024-09-17T21:31:27.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.037 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:27.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:27.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:27.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:31:27.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410267+0000", 2024-09-17T21:31:27.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:31:27.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410270+0000", 2024-09-17T21:31:27.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.01e-06 2024-09-17T21:31:27.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410273+0000", 2024-09-17T21:31:27.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6410000000000002e-06 2024-09-17T21:31:27.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457473+0000", 2024-09-17T21:31:27.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047199829999999998 2024-09-17T21:31:27.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457517+0000", 2024-09-17T21:31:27.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4058999999999999e-05 2024-09-17T21:31:27.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457559+0000", 2024-09-17T21:31:27.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2404999999999998e-05 2024-09-17T21:31:27.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.041 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.041 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17620 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 927744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:27.041 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:27.041 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.588962933000005, 2024-09-17T21:31:27.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047402409, 2024-09-17T21:31:27.042 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.042 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.042 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:27.042 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.042 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.042 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.042 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17620 2024-09-17T21:31:27.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.042 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:27.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:27.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:27.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:31:27.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410278+0000", 2024-09-17T21:31:27.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1790000000000001e-06 2024-09-17T21:31:27.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410281+0000", 2024-09-17T21:31:27.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.061e-06 2024-09-17T21:31:27.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410284+0000", 2024-09-17T21:31:27.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.65e-06 2024-09-17T21:31:27.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457587+0000", 2024-09-17T21:31:27.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047303206 2024-09-17T21:31:27.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457633+0000", 2024-09-17T21:31:27.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6063999999999998e-05 2024-09-17T21:31:27.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457679+0000", 2024-09-17T21:31:27.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6007000000000002e-05 2024-09-17T21:31:27.050 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.050 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.050 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.050 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17621 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 940032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:27.050 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:31:27.050 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.588952516999996, 2024-09-17T21:31:27.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047486550000000002, 2024-09-17T21:31:27.051 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.051 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.051 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:27.051 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.051 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.051 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.051 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17621 2024-09-17T21:31:27.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.051 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:31:27.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:31:27.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410288+0000", 2024-09-17T21:31:27.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:31:27.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410289+0000", 2024-09-17T21:31:27.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1179999999999999e-06 2024-09-17T21:31:27.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410292+0000", 2024-09-17T21:31:27.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:31:27.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410294+0000", 2024-09-17T21:31:27.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T21:31:27.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457694+0000", 2024-09-17T21:31:27.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047399416999999999 2024-09-17T21:31:27.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457734+0000", 2024-09-17T21:31:27.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0528999999999998e-05 2024-09-17T21:31:27.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457774+0000", 2024-09-17T21:31:27.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9529e-05 2024-09-17T21:31:27.055 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.055 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.055 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.055 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17622 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:27.055 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:27.055 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.588942334999999, 2024-09-17T21:31:27.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047596619, 2024-09-17T21:31:27.056 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.056 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.056 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:27.056 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.056 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.056 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.056 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17622 2024-09-17T21:31:27.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.056 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:27.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:27.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:27.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:31:27.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410303+0000", 2024-09-17T21:31:27.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2979999999999999e-06 2024-09-17T21:31:27.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410307+0000", 2024-09-17T21:31:27.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:31:27.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410310+0000", 2024-09-17T21:31:27.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.146e-06 2024-09-17T21:31:27.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457801+0000", 2024-09-17T21:31:27.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047491322000000002 2024-09-17T21:31:27.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457850+0000", 2024-09-17T21:31:27.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8585000000000003e-05 2024-09-17T21:31:27.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457894+0000", 2024-09-17T21:31:27.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4455e-05 2024-09-17T21:31:27.060 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.060 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.060 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.060 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17623 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 968704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:27.060 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:27.060 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.588925950999993, 2024-09-17T21:31:27.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047689377999999998, 2024-09-17T21:31:27.060 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.060 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.060 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:27.061 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.061 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.061 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.061 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17623 2024-09-17T21:31:27.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.061 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:27.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:27.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:27.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:31:27.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410315+0000", 2024-09-17T21:31:27.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0499999999999999e-06 2024-09-17T21:31:27.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410319+0000", 2024-09-17T21:31:27.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.517e-06 2024-09-17T21:31:27.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410321+0000", 2024-09-17T21:31:27.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:31:27.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457922+0000", 2024-09-17T21:31:27.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047600638000000001 2024-09-17T21:31:27.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457964+0000", 2024-09-17T21:31:27.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2126000000000003e-05 2024-09-17T21:31:27.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458003+0000", 2024-09-17T21:31:27.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9138999999999999e-05 2024-09-17T21:31:27.065 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.065 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.065 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.065 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17624 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:27.065 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:27.065 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.588915366999998, 2024-09-17T21:31:27.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047798292999999999, 2024-09-17T21:31:27.065 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.065 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.065 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:27.066 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.066 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.066 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.066 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17624 2024-09-17T21:31:27.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.066 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:27.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:27.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:27.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:31:27.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410326+0000", 2024-09-17T21:31:27.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0610000000000001e-06 2024-09-17T21:31:27.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410329+0000", 2024-09-17T21:31:27.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0319999999999999e-06 2024-09-17T21:31:27.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410331+0000", 2024-09-17T21:31:27.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5749999999999999e-06 2024-09-17T21:31:27.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458033+0000", 2024-09-17T21:31:27.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047701371999999999 2024-09-17T21:31:27.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458079+0000", 2024-09-17T21:31:27.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:31:27.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458123+0000", 2024-09-17T21:31:27.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850999999999997e-05 2024-09-17T21:31:27.070 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.070 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.070 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.070 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17625 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:27.070 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:27.070 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.588905206999996, 2024-09-17T21:31:27.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047883051000000003, 2024-09-17T21:31:27.070 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.070 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.070 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:27.070 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.070 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.071 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.071 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17625 2024-09-17T21:31:27.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.071 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:27.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:27.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:31:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410336+0000", 2024-09-17T21:31:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1310000000000001e-06 2024-09-17T21:31:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410339+0000", 2024-09-17T21:31:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000001e-06 2024-09-17T21:31:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410342+0000", 2024-09-17T21:31:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7460000000000001e-06 2024-09-17T21:31:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458137+0000", 2024-09-17T21:31:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047794931999999998 2024-09-17T21:31:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458178+0000", 2024-09-17T21:31:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1013999999999997e-05 2024-09-17T21:31:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458218+0000", 2024-09-17T21:31:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9675999999999998e-05 2024-09-17T21:31:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17626 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1009664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:31:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.588894585999995, 2024-09-17T21:31:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047991601000000002, 2024-09-17T21:31:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17626 2024-09-17T21:31:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:31:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:31:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410346+0000", 2024-09-17T21:31:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:31:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410347+0000", 2024-09-17T21:31:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.114e-06 2024-09-17T21:31:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410350+0000", 2024-09-17T21:31:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.101e-06 2024-09-17T21:31:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410353+0000", 2024-09-17T21:31:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T21:31:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458245+0000", 2024-09-17T21:31:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0478921 2024-09-17T21:31:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458292+0000", 2024-09-17T21:31:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397999999999999e-05 2024-09-17T21:31:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458337+0000", 2024-09-17T21:31:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5221e-05 2024-09-17T21:31:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17627 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:31:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.588857485000005, 2024-09-17T21:31:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048047383999999999, 2024-09-17T21:31:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17627 2024-09-17T21:31:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:31:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:31:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410383+0000", 2024-09-17T21:31:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:31:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410385+0000", 2024-09-17T21:31:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:31:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410391+0000", 2024-09-17T21:31:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:31:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410395+0000", 2024-09-17T21:31:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6539999999999998e-06 2024-09-17T21:31:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458350+0000", 2024-09-17T21:31:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047954855999999997 2024-09-17T21:31:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458392+0000", 2024-09-17T21:31:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2094000000000001e-05 2024-09-17T21:31:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458430+0000", 2024-09-17T21:31:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7474000000000001e-05 2024-09-17T21:31:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17628 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.588837924000003, 2024-09-17T21:31:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048142676000000002, 2024-09-17T21:31:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17628 2024-09-17T21:31:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:31:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410404+0000", 2024-09-17T21:31:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-06 2024-09-17T21:31:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410411+0000", 2024-09-17T21:31:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6850000000000004e-06 2024-09-17T21:31:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410415+0000", 2024-09-17T21:31:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6750000000000001e-06 2024-09-17T21:31:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458457+0000", 2024-09-17T21:31:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048042045999999998 2024-09-17T21:31:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458501+0000", 2024-09-17T21:31:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3661000000000001e-05 2024-09-17T21:31:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458545+0000", 2024-09-17T21:31:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3529000000000003e-05 2024-09-17T21:31:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17687 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:31:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.586392970999995, 2024-09-17T21:31:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050670448, 2024-09-17T21:31:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17687 2024-09-17T21:31:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:31:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:31:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412848+0000", 2024-09-17T21:31:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9100000000000003e-07 2024-09-17T21:31:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413102+0000", 2024-09-17T21:31:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025415600000000001 2024-09-17T21:31:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413111+0000", 2024-09-17T21:31:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2010000000000005e-06 2024-09-17T21:31:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413116+0000", 2024-09-17T21:31:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:31:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458559+0000", 2024-09-17T21:31:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045442914000000001 2024-09-17T21:31:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458632+0000", 2024-09-17T21:31:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3207000000000006e-05 2024-09-17T21:31:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:31:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458785+0000", 2024-09-17T21:31:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015226700000000001 2024-09-17T21:31:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.461648+0000", 2024-09-17T21:31:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028633069999999998 2024-09-17T21:31:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463416+0000", 2024-09-17T21:31:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017680230000000001 2024-09-17T21:31:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463475+0000", 2024-09-17T21:31:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9333000000000003e-05 2024-09-17T21:31:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463517+0000", 2024-09-17T21:31:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2162000000000001e-05 2024-09-17T21:31:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17688 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [write 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.586114103, 2024-09-17T21:31:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050625098, 2024-09-17T21:31:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17688 2024-09-17T21:31:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:31:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413128+0000", 2024-09-17T21:31:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.813e-06 2024-09-17T21:31:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413132+0000", 2024-09-17T21:31:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:31:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413135+0000", 2024-09-17T21:31:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:31:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459756+0000", 2024-09-17T21:31:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046621293000000001 2024-09-17T21:31:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459816+0000", 2024-09-17T21:31:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9156000000000001e-05 2024-09-17T21:31:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:31:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459912+0000", 2024-09-17T21:31:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6302000000000001e-05 2024-09-17T21:31:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463231+0000", 2024-09-17T21:31:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033188290000000001 2024-09-17T21:31:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463701+0000", 2024-09-17T21:31:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047068199999999999 2024-09-17T21:31:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463728+0000", 2024-09-17T21:31:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6726000000000001e-05 2024-09-17T21:31:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463751+0000", 2024-09-17T21:31:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2824e-05 2024-09-17T21:31:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17691 3.d 3:b3a055f3:::rbd_data.11207744cc2c.0000000000000083:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:31:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.582569199999995, 2024-09-17T21:31:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049309370999999998, 2024-09-17T21:31:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17691 2024-09-17T21:31:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:31:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:31:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416672+0000", 2024-09-17T21:31:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.47e-07 2024-09-17T21:31:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416915+0000", 2024-09-17T21:31:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024344500000000001 2024-09-17T21:31:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416924+0000", 2024-09-17T21:31:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7739999999999993e-06 2024-09-17T21:31:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416929+0000", 2024-09-17T21:31:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6990000000000002e-06 2024-09-17T21:31:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460414+0000", 2024-09-17T21:31:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043485466 2024-09-17T21:31:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460540+0000", 2024-09-17T21:31:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012608899999999999 2024-09-17T21:31:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:31:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460746+0000", 2024-09-17T21:31:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000205577 2024-09-17T21:31:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463800+0000", 2024-09-17T21:31:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030542030000000001 2024-09-17T21:31:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465867+0000", 2024-09-17T21:31:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020665929999999998 2024-09-17T21:31:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465914+0000", 2024-09-17T21:31:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497000000000003e-05 2024-09-17T21:31:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465980+0000", 2024-09-17T21:31:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6081000000000006e-05 2024-09-17T21:31:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:70218 3.16 3:68692d8f:::rbd_data.1120b9193b9f.000000000000001b:head [stat,delete] snapc 1d=[1d] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-17T21:31:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.193602110999997, 2024-09-17T21:31:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.951400786, 2024-09-17T21:31:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 70218 2024-09-17T21:31:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4799999999999998e-07 2024-09-17T21:31:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805642+0000", 2024-09-17T21:31:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2009999999999999e-06 2024-09-17T21:31:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805650+0000", 2024-09-17T21:31:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1000000000000004e-06 2024-09-17T21:31:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805655+0000", 2024-09-17T21:31:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:31:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805722+0000", 2024-09-17T21:31:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6878000000000006e-05 2024-09-17T21:31:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:31:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805733+0000", 2024-09-17T21:31:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0895e-05 2024-09-17T21:31:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755252+0000", 2024-09-17T21:31:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.94951843800000002 2024-09-17T21:31:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755398+0000", 2024-09-17T21:31:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014622600000000001 2024-09-17T21:31:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.757039+0000", 2024-09-17T21:31:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001640819 2024-09-17T21:31:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:27.113 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:27.115 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:31:27.115 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:27.111+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:27.216 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:27.213+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:27.218 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:27.218 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:27.219 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:27.219 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:27.227 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:31:27.318 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:27.314+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:27.330 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:27.330 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:27.330 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:27.330 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:27.330 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:27.340 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:31:27.399 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:31:27.408 INFO:teuthology.orchestra.run.smithi129.stderr:set osd.2 primary-affinity to 0.21674 (8142042) 2024-09-17T21:31:27.418 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:27.414+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:27.457 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:27.457 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:27.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:27.457 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:27.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.458 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97778 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 245760~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.458 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:31:27.458 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.485610731000001, 2024-09-17T21:31:27.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050933728999999997, 2024-09-17T21:31:27.458 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.458 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.458 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.458 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.458 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.458 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.458 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97778 2024-09-17T21:31:27.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.459 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:31:27.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:31:27.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966899+0000", 2024-09-17T21:31:27.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:31:27.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966900+0000", 2024-09-17T21:31:27.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.384e-06 2024-09-17T21:31:27.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966905+0000", 2024-09-17T21:31:27.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6530000000000002e-06 2024-09-17T21:31:27.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966909+0000", 2024-09-17T21:31:27.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2889999999999999e-06 2024-09-17T21:31:27.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017585+0000", 2024-09-17T21:31:27.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050675691000000002 2024-09-17T21:31:27.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017642+0000", 2024-09-17T21:31:27.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7167999999999999e-05 2024-09-17T21:31:27.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017832+0000", 2024-09-17T21:31:27.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019022 2024-09-17T21:31:27.462 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.462 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.462 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.462 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97779 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.462 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:31:27.462 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.485593663000003, 2024-09-17T21:31:27.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051245027999999998, 2024-09-17T21:31:27.463 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.463 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.463 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.463 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.463 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.463 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.463 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97779 2024-09-17T21:31:27.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.463 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:31:27.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:31:27.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966916+0000", 2024-09-17T21:31:27.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:31:27.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966918+0000", 2024-09-17T21:31:27.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8369999999999999e-06 2024-09-17T21:31:27.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966924+0000", 2024-09-17T21:31:27.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2260000000000002e-06 2024-09-17T21:31:27.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966928+0000", 2024-09-17T21:31:27.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3819999999999997e-06 2024-09-17T21:31:27.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017866+0000", 2024-09-17T21:31:27.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050938079999999997 2024-09-17T21:31:27.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017945+0000", 2024-09-17T21:31:27.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8275000000000005e-05 2024-09-17T21:31:27.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018161+0000", 2024-09-17T21:31:27.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021579499999999999 2024-09-17T21:31:27.467 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.467 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.467 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.467 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97780 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 344064~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.467 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:27.467 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.485569384999998, 2024-09-17T21:31:27.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051535301999999998, 2024-09-17T21:31:27.467 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.467 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.467 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.467 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.467 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.468 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.468 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97780 2024-09-17T21:31:27.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.468 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:27.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:27.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:27.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:31:27.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966942+0000", 2024-09-17T21:31:27.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7439999999999999e-06 2024-09-17T21:31:27.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966946+0000", 2024-09-17T21:31:27.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5510000000000001e-06 2024-09-17T21:31:27.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966951+0000", 2024-09-17T21:31:27.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:31:27.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018191+0000", 2024-09-17T21:31:27.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051240290000000001 2024-09-17T21:31:27.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018269+0000", 2024-09-17T21:31:27.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7416999999999994e-05 2024-09-17T21:31:27.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018475+0000", 2024-09-17T21:31:27.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206486 2024-09-17T21:31:27.471 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.471 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.471 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.471 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97789 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 40960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.471 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:27.471 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.485549216000003, 2024-09-17T21:31:27.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051862143999999999, 2024-09-17T21:31:27.472 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.472 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.472 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.472 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.472 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.472 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.472 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97789 2024-09-17T21:31:27.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.472 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:27.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:27.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:27.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5899999999999996e-07 2024-09-17T21:31:27.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966962+0000", 2024-09-17T21:31:27.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9700000000000002e-06 2024-09-17T21:31:27.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966969+0000", 2024-09-17T21:31:27.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6939999999999998e-06 2024-09-17T21:31:27.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966980+0000", 2024-09-17T21:31:27.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0679999999999999e-05 2024-09-17T21:31:27.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018513+0000", 2024-09-17T21:31:27.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051533420000000003 2024-09-17T21:31:27.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018595+0000", 2024-09-17T21:31:27.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1260999999999993e-05 2024-09-17T21:31:27.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018822+0000", 2024-09-17T21:31:27.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022756 2024-09-17T21:31:27.476 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.476 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.476 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.476 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97790 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 90112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.476 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:31:27.476 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.485518718000002, 2024-09-17T21:31:27.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052173479000000002, 2024-09-17T21:31:27.476 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.476 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.477 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.477 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.477 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.477 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.477 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97790 2024-09-17T21:31:27.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.477 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:31:27.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:31:27.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966991+0000", 2024-09-17T21:31:27.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9500000000000003e-07 2024-09-17T21:31:27.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966993+0000", 2024-09-17T21:31:27.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.238e-06 2024-09-17T21:31:27.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967000+0000", 2024-09-17T21:31:27.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9500000000000004e-06 2024-09-17T21:31:27.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967005+0000", 2024-09-17T21:31:27.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:31:27.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018852+0000", 2024-09-17T21:31:27.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051847250999999997 2024-09-17T21:31:27.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018927+0000", 2024-09-17T21:31:27.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4944000000000006e-05 2024-09-17T21:31:27.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019164+0000", 2024-09-17T21:31:27.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000236803 2024-09-17T21:31:27.480 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.480 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.480 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.480 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97791 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 139264~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.480 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:31:27.480 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.485496974, 2024-09-17T21:31:27.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052464594000000003, 2024-09-17T21:31:27.481 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.481 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.481 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.481 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97791 2024-09-17T21:31:27.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.481 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:31:27.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:31:27.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967013+0000", 2024-09-17T21:31:27.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:31:27.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967014+0000", 2024-09-17T21:31:27.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8679999999999999e-06 2024-09-17T21:31:27.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967020+0000", 2024-09-17T21:31:27.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.322e-06 2024-09-17T21:31:27.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967024+0000", 2024-09-17T21:31:27.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2660000000000003e-06 2024-09-17T21:31:27.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019193+0000", 2024-09-17T21:31:27.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052169010000000002 2024-09-17T21:31:27.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019269+0000", 2024-09-17T21:31:27.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6149999999999994e-05 2024-09-17T21:31:27.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019477+0000", 2024-09-17T21:31:27.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020753699999999999 2024-09-17T21:31:27.484 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.484 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.484 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.485 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97792 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 188416~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.485 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:27.485 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.485479235, 2024-09-17T21:31:27.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052764958000000001, 2024-09-17T21:31:27.485 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.485 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.485 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.485 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.485 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.485 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.485 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97792 2024-09-17T21:31:27.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.486 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:27.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:27.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:27.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0900000000000002e-07 2024-09-17T21:31:27.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967032+0000", 2024-09-17T21:31:27.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9740000000000001e-06 2024-09-17T21:31:27.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967039+0000", 2024-09-17T21:31:27.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3679999999999998e-06 2024-09-17T21:31:27.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967043+0000", 2024-09-17T21:31:27.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3379999999999998e-06 2024-09-17T21:31:27.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019505+0000", 2024-09-17T21:31:27.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052462264000000002 2024-09-17T21:31:27.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019582+0000", 2024-09-17T21:31:27.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7146999999999998e-05 2024-09-17T21:31:27.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019795+0000", 2024-09-17T21:31:27.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000212458 2024-09-17T21:31:27.489 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.489 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.489 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.489 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97793 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 237568~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.489 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:31:27.489 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.485457734999997, 2024-09-17T21:31:27.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053070436999999998, 2024-09-17T21:31:27.490 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.490 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.490 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.490 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.490 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.490 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.490 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97793 2024-09-17T21:31:27.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.490 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:31:27.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:31:27.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967052+0000", 2024-09-17T21:31:27.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7500000000000001e-07 2024-09-17T21:31:27.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967054+0000", 2024-09-17T21:31:27.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:31:27.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967059+0000", 2024-09-17T21:31:27.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5840000000000002e-06 2024-09-17T21:31:27.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967064+0000", 2024-09-17T21:31:27.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:31:27.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019826+0000", 2024-09-17T21:31:27.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052762396000000003 2024-09-17T21:31:27.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019900+0000", 2024-09-17T21:31:27.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3594e-05 2024-09-17T21:31:27.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020122+0000", 2024-09-17T21:31:27.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022185999999999999 2024-09-17T21:31:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97794 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 286720~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.485437599000001, 2024-09-17T21:31:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053453734000000003, 2024-09-17T21:31:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.495 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97794 2024-09-17T21:31:27.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.495 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:27.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:27.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:27.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2199999999999999e-07 2024-09-17T21:31:27.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967074+0000", 2024-09-17T21:31:27.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8419999999999999e-06 2024-09-17T21:31:27.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967080+0000", 2024-09-17T21:31:27.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5660000000000003e-06 2024-09-17T21:31:27.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967120+0000", 2024-09-17T21:31:27.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9504e-05 2024-09-17T21:31:27.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020189+0000", 2024-09-17T21:31:27.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053069144999999998 2024-09-17T21:31:27.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020287+0000", 2024-09-17T21:31:27.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8259999999999998e-05 2024-09-17T21:31:27.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020525+0000", 2024-09-17T21:31:27.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023799499999999999 2024-09-17T21:31:27.498 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.498 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.498 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.498 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97795 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 335872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.499 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:27.499 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.485379465000001, 2024-09-17T21:31:27.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053707207999999999, 2024-09-17T21:31:27.499 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.499 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.499 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.499 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97795 2024-09-17T21:31:27.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.499 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:27.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:27.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:27.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:31:27.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967139+0000", 2024-09-17T21:31:27.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5790000000000004e-06 2024-09-17T21:31:27.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967145+0000", 2024-09-17T21:31:27.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3550000000000001e-06 2024-09-17T21:31:27.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967155+0000", 2024-09-17T21:31:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8859999999999999e-06 2024-09-17T21:31:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020555+0000", 2024-09-17T21:31:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053400359000000001 2024-09-17T21:31:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020630+0000", 2024-09-17T21:31:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4894999999999994e-05 2024-09-17T21:31:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020837+0000", 2024-09-17T21:31:27.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020670500000000001 2024-09-17T21:31:27.503 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.503 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.503 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.503 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97796 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 385024~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.503 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:27.503 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.485347204999997, 2024-09-17T21:31:27.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053990227000000002, 2024-09-17T21:31:27.503 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.504 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.504 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.504 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.504 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.504 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.504 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97796 2024-09-17T21:31:27.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.504 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:27.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:27.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:27.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:31:27.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967164+0000", 2024-09-17T21:31:27.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0370000000000001e-06 2024-09-17T21:31:27.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967171+0000", 2024-09-17T21:31:27.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2190000000000001e-06 2024-09-17T21:31:27.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967175+0000", 2024-09-17T21:31:27.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2599999999999999e-06 2024-09-17T21:31:27.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020866+0000", 2024-09-17T21:31:27.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053691539000000003 2024-09-17T21:31:27.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020942+0000", 2024-09-17T21:31:27.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5487000000000004e-05 2024-09-17T21:31:27.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021152+0000", 2024-09-17T21:31:27.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021021599999999999 2024-09-17T21:31:27.508 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.508 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.508 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.508 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97805 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 8192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.508 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:27.508 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.485328525, 2024-09-17T21:31:27.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054331771000000001, 2024-09-17T21:31:27.508 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.508 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.509 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.509 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97805 2024-09-17T21:31:27.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.509 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:27.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:27.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:27.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:31:27.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967183+0000", 2024-09-17T21:31:27.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6840000000000001e-06 2024-09-17T21:31:27.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967189+0000", 2024-09-17T21:31:27.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0299999999999999e-06 2024-09-17T21:31:27.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967193+0000", 2024-09-17T21:31:27.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-17T21:31:27.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021182+0000", 2024-09-17T21:31:27.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053989319000000001 2024-09-17T21:31:27.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021261+0000", 2024-09-17T21:31:27.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8460000000000004e-05 2024-09-17T21:31:27.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021512+0000", 2024-09-17T21:31:27.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025158500000000002 2024-09-17T21:31:27.512 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.512 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.513 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.513 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97806 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.513 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:27.513 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.485310521999999, 2024-09-17T21:31:27.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054654343000000001, 2024-09-17T21:31:27.513 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.513 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.513 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.513 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.513 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.513 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.513 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97806 2024-09-17T21:31:27.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.514 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:27.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:27.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:27.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:31:27.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967201+0000", 2024-09-17T21:31:27.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.691e-06 2024-09-17T21:31:27.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967206+0000", 2024-09-17T21:31:27.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7780000000000004e-06 2024-09-17T21:31:27.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967224+0000", 2024-09-17T21:31:27.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7357999999999999e-05 2024-09-17T21:31:27.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021546+0000", 2024-09-17T21:31:27.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054322237000000002 2024-09-17T21:31:27.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021618+0000", 2024-09-17T21:31:27.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2007000000000004e-05 2024-09-17T21:31:27.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021853+0000", 2024-09-17T21:31:27.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023495800000000001 2024-09-17T21:31:27.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.517 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.517 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97807 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 106496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.517 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:31:27.517 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.485278121999997, 2024-09-17T21:31:27.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055014183000000001, 2024-09-17T21:31:27.518 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.518 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.518 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.518 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97807 2024-09-17T21:31:27.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.518 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:31:27.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:31:27.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967232+0000", 2024-09-17T21:31:27.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:31:27.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967234+0000", 2024-09-17T21:31:27.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.283e-06 2024-09-17T21:31:27.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967240+0000", 2024-09-17T21:31:27.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.144e-06 2024-09-17T21:31:27.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967252+0000", 2024-09-17T21:31:27.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2322e-05 2024-09-17T21:31:27.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021882+0000", 2024-09-17T21:31:27.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054629952000000002 2024-09-17T21:31:27.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021952+0000", 2024-09-17T21:31:27.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9919999999999995e-05 2024-09-17T21:31:27.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022245+0000", 2024-09-17T21:31:27.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00029302100000000003 2024-09-17T21:31:27.521 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.521 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.522 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.522 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97808 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 155648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.522 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:27.522 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.485249523, 2024-09-17T21:31:27.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055447485999999997, 2024-09-17T21:31:27.522 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.522 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.522 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.523 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.523 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97808 2024-09-17T21:31:27.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.523 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:27.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:27.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:27.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4299999999999998e-07 2024-09-17T21:31:27.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967267+0000", 2024-09-17T21:31:27.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9820000000000002e-06 2024-09-17T21:31:27.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967273+0000", 2024-09-17T21:31:27.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:31:27.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967277+0000", 2024-09-17T21:31:27.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5050000000000001e-06 2024-09-17T21:31:27.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022297+0000", 2024-09-17T21:31:27.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055020131999999999 2024-09-17T21:31:27.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022379+0000", 2024-09-17T21:31:27.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1810000000000004e-05 2024-09-17T21:31:27.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022707+0000", 2024-09-17T21:31:27.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00032784300000000003 2024-09-17T21:31:27.526 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.526 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.526 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.526 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97809 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.526 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:31:27.527 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.485226007999998, 2024-09-17T21:31:27.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056012109999999997, 2024-09-17T21:31:27.527 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.527 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.527 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.527 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97809 2024-09-17T21:31:27.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.527 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:31:27.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:31:27.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967284+0000", 2024-09-17T21:31:27.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:31:27.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967285+0000", 2024-09-17T21:31:27.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.826e-06 2024-09-17T21:31:27.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967291+0000", 2024-09-17T21:31:27.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:31:27.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967302+0000", 2024-09-17T21:31:27.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1114999999999999e-05 2024-09-17T21:31:27.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022901+0000", 2024-09-17T21:31:27.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055599335 2024-09-17T21:31:27.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022990+0000", 2024-09-17T21:31:27.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8671999999999994e-05 2024-09-17T21:31:27.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023295+0000", 2024-09-17T21:31:27.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030554900000000001 2024-09-17T21:31:27.531 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.531 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.531 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.531 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97810 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 253952~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.531 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:27.531 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.48519958, 2024-09-17T21:31:27.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056355668999999997, 2024-09-17T21:31:27.531 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.532 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.532 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.532 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.532 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.532 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.532 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97810 2024-09-17T21:31:27.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.532 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:27.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:27.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:27.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0200000000000002e-07 2024-09-17T21:31:27.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967313+0000", 2024-09-17T21:31:27.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4549999999999998e-06 2024-09-17T21:31:27.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967319+0000", 2024-09-17T21:31:27.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3080000000000004e-06 2024-09-17T21:31:27.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967324+0000", 2024-09-17T21:31:27.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0159999999999999e-06 2024-09-17T21:31:27.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023342+0000", 2024-09-17T21:31:27.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056018635999999997 2024-09-17T21:31:27.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023423+0000", 2024-09-17T21:31:27.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0473e-05 2024-09-17T21:31:27.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023665+0000", 2024-09-17T21:31:27.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242279 2024-09-17T21:31:27.535 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.535 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.535 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.536 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97811 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 303104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.536 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:27.536 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.485173322000001, 2024-09-17T21:31:27.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056743978, 2024-09-17T21:31:27.536 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.536 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.536 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.536 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.536 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.536 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.536 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97811 2024-09-17T21:31:27.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.537 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:27.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:27.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:27.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1300000000000001e-07 2024-09-17T21:31:27.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967338+0000", 2024-09-17T21:31:27.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9740000000000001e-06 2024-09-17T21:31:27.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967346+0000", 2024-09-17T21:31:27.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8129999999999996e-06 2024-09-17T21:31:27.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967351+0000", 2024-09-17T21:31:27.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6129999999999997e-06 2024-09-17T21:31:27.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023750+0000", 2024-09-17T21:31:27.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056399796000000002 2024-09-17T21:31:27.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023839+0000", 2024-09-17T21:31:27.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.852e-05 2024-09-17T21:31:27.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024080+0000", 2024-09-17T21:31:27.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024084900000000001 2024-09-17T21:31:27.540 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.540 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.540 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.540 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97812 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 352256~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:27.540 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:31:27.540 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.485152706000001, 2024-09-17T21:31:27.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.057024455000000002, 2024-09-17T21:31:27.541 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.541 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.541 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.541 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.541 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.541 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.541 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97812 2024-09-17T21:31:27.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.541 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:31:27.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:31:27.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967357+0000", 2024-09-17T21:31:27.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:31:27.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967359+0000", 2024-09-17T21:31:27.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:31:27.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967365+0000", 2024-09-17T21:31:27.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:31:27.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967369+0000", 2024-09-17T21:31:27.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5029999999999999e-06 2024-09-17T21:31:27.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024109+0000", 2024-09-17T21:31:27.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056740038 2024-09-17T21:31:27.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024175+0000", 2024-09-17T21:31:27.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5106999999999999e-05 2024-09-17T21:31:27.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024381+0000", 2024-09-17T21:31:27.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020629700000000001 2024-09-17T21:31:27.544 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.545 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.545 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.545 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:112033 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-09-17T21:31:27.545 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:31:11.421210+0000", 2024-09-17T21:31:27.545 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 16.031298939999999, 2024-09-17T21:31:27.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0104142869999999, 2024-09-17T21:31:27.545 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:27.545 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:27.545 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:27.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:27.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:27.546 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:27.546 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 112033 2024-09-17T21:31:27.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.546 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:27.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:27.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421210+0000", 2024-09-17T21:31:27.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:27.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421210+0000", 2024-09-17T21:31:27.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:27.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:27.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421212+0000", 2024-09-17T21:31:27.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.376e-06 2024-09-17T21:31:27.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:27.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421217+0000", 2024-09-17T21:31:27.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.259e-06 2024-09-17T21:31:27.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:27.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421228+0000", 2024-09-17T21:31:27.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0944000000000001e-05 2024-09-17T21:31:27.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:27.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421238+0000", 2024-09-17T21:31:27.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:31:27.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421626+0000", 2024-09-17T21:31:27.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00038804099999999999 2024-09-17T21:31:27.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:31:27.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421637+0000", 2024-09-17T21:31:27.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1783000000000001e-05 2024-09-17T21:31:27.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:27.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:12.431245+0000", 2024-09-17T21:31:27.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.00960765 2024-09-17T21:31:27.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:27.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:12.431413+0000", 2024-09-17T21:31:27.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000167784 2024-09-17T21:31:27.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:27.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:27.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:27.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:12.431624+0000", 2024-09-17T21:31:27.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000211704 2024-09-17T21:31:27.550 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.550 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.550 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.550 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:27.550 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:27.550 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:27.551 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:31:27.552 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:27.548+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:27.553 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:27.553 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:31:27.553 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:27.561 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:27.561 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:27.578 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:27.578 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:27.595 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:27.595 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:27.612 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:27.612 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:27.629 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:27.629 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:27.646 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:27.646 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:27.652 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:27.650+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:27.662 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:27.663 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:27.668 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:27.668 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:27.677 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:31:27.761 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:27.759+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:27.822 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:27.822 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:27.830 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:31:27.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T21:31:27.863+0000 7f7b637fe640 0 --2- 172.21.15.129:0/4114038485 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f7b640a98d0 0x7f7b6413feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-17T21:31:27.866 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:27.863+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:27.954 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:27.952+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:27.971 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:27.971 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:27.980 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:31:28.056 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:28.053+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:28.124 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:28.124 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:28.133 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:31:28.156 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:28.154+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:28.235 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:28.235 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:28.243 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:31:28.286 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:28.284+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:28.347 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:28.347 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:28.356 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:31:28.357 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:28.355+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:28.414 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is set 2024-09-17T21:31:28.474 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:28.472+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:28.516 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:28.516 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:28.526 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:31:28.559 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:28.556+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:28.661 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:28.658+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:28.669 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:28.669 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:28.681 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:31:28.761 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:28.759+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:28.785 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:28.785 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:28.794 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:31:28.862 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:28.859+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:28.910 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:28.910 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:28.919 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:31:29.003 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:29.000+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:29.064 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:29.064 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:29.064 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:29.061+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:29.072 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:31:29.165 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:29.162+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:29.216 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:29.216 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:29.225 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:31:29.266 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:29.263+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:29.367 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:29.365+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:29.368 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:29.368 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:29.376 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:31:29.468 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:29.466+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:29.478 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:29.478 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:29.487 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:31:29.569 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:29.566+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:29.598 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:29.598 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:29.608 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:31:29.670 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:29.668+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:29.675 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:31:29.751 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:29.752 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:29.760 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:31:29.770 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:29.768+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:29.795 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:29.795 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:29.811 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:29.811 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:29.827 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:29.828 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:29.843 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:29.843 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:29.860 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:29.860 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:29.863 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:29.863 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:29.871 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:29.868+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:29.872 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:31:29.877 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:29.877 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:29.893 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:29.893 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:29.910 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:29.910 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:29.972 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:29.970+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:30.017 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:30.017 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:30.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:30.017 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:30.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.017 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:36935 3.a 3:54c8759b:::rbd_data.1120e62eea89.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-17T21:31:30.017 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:31:30.017 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.055245842999994, 2024-09-17T21:31:30.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020540151999999999, 2024-09-17T21:31:30.018 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.018 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.018 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:30.018 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.018 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.018 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.018 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 36935 2024-09-17T21:31:30.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.018 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:31:30.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:31:30.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958910+0000", 2024-09-17T21:31:30.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:31:30.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958918+0000", 2024-09-17T21:31:30.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6469999999999998e-06 2024-09-17T21:31:30.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958933+0000", 2024-09-17T21:31:30.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4791e-05 2024-09-17T21:31:30.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958940+0000", 2024-09-17T21:31:30.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-17T21:31:30.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958990+0000", 2024-09-17T21:31:30.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9759000000000003e-05 2024-09-17T21:31:30.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.959039+0000", 2024-09-17T21:31:30.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9487000000000002e-05 2024-09-17T21:31:30.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:30.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.962098+0000", 2024-09-17T21:31:30.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003059258 2024-09-17T21:31:30.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:30.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979325+0000", 2024-09-17T21:31:30.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017226841 2024-09-17T21:31:30.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:30.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979343+0000", 2024-09-17T21:31:30.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7486000000000002e-05 2024-09-17T21:31:30.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:30.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979401+0000", 2024-09-17T21:31:30.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7980999999999999e-05 2024-09-17T21:31:30.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979449+0000", 2024-09-17T21:31:30.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8639e-05 2024-09-17T21:31:30.024 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.024 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.024 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.024 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39447 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 909312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.024 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:30.024 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.046307943000002, 2024-09-17T21:31:30.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017419153999999999, 2024-09-17T21:31:30.024 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.024 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.024 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.025 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.025 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.025 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.025 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39447 2024-09-17T21:31:30.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.025 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:30.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:30.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:30.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:31:30.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967849+0000", 2024-09-17T21:31:30.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4339999999999999e-06 2024-09-17T21:31:30.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967853+0000", 2024-09-17T21:31:30.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:31:30.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967857+0000", 2024-09-17T21:31:30.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:31:30.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985193+0000", 2024-09-17T21:31:30.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017335973000000001 2024-09-17T21:31:30.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985228+0000", 2024-09-17T21:31:30.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5154999999999998e-05 2024-09-17T21:31:30.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985266+0000", 2024-09-17T21:31:30.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8448000000000002e-05 2024-09-17T21:31:30.028 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.028 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.028 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.028 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39448 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 974848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.028 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:30.029 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.046295126999993, 2024-09-17T21:31:30.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017517066000000001, 2024-09-17T21:31:30.029 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.029 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.029 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.029 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.029 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.029 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.029 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39448 2024-09-17T21:31:30.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.029 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:30.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:30.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:30.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.72e-07 2024-09-17T21:31:30.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967861+0000", 2024-09-17T21:31:30.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.096e-06 2024-09-17T21:31:30.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967865+0000", 2024-09-17T21:31:30.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9339999999999999e-06 2024-09-17T21:31:30.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967868+0000", 2024-09-17T21:31:30.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.508e-06 2024-09-17T21:31:30.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985285+0000", 2024-09-17T21:31:30.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017417208 2024-09-17T21:31:30.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985339+0000", 2024-09-17T21:31:30.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3931000000000003e-05 2024-09-17T21:31:30.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985377+0000", 2024-09-17T21:31:30.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8016999999999999e-05 2024-09-17T21:31:30.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.033 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.033 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.033 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39449 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.033 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:30.033 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.046283016999993, 2024-09-17T21:31:30.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017595137, 2024-09-17T21:31:30.033 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.033 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.033 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.034 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.034 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.034 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.034 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39449 2024-09-17T21:31:30.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.034 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:30.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:30.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:30.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2700000000000001e-07 2024-09-17T21:31:30.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967873+0000", 2024-09-17T21:31:30.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.164e-06 2024-09-17T21:31:30.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967877+0000", 2024-09-17T21:31:30.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.23e-06 2024-09-17T21:31:30.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967880+0000", 2024-09-17T21:31:30.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.286e-06 2024-09-17T21:31:30.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985397+0000", 2024-09-17T21:31:30.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017516982 2024-09-17T21:31:30.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985431+0000", 2024-09-17T21:31:30.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3899999999999997e-05 2024-09-17T21:31:30.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985467+0000", 2024-09-17T21:31:30.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6347999999999998e-05 2024-09-17T21:31:30.037 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.037 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39450 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1105920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:31:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.046262760000005, 2024-09-17T21:31:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017677824000000002, 2024-09-17T21:31:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39450 2024-09-17T21:31:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:31:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:31:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967893+0000", 2024-09-17T21:31:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9100000000000004e-07 2024-09-17T21:31:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967895+0000", 2024-09-17T21:31:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:31:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967901+0000", 2024-09-17T21:31:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.083e-06 2024-09-17T21:31:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967906+0000", 2024-09-17T21:31:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:31:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985493+0000", 2024-09-17T21:31:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017587565999999999 2024-09-17T21:31:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985531+0000", 2024-09-17T21:31:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7490000000000002e-05 2024-09-17T21:31:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985570+0000", 2024-09-17T21:31:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9205000000000002e-05 2024-09-17T21:31:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39451 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1171456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.046242379999995, 2024-09-17T21:31:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017752337999999999, 2024-09-17T21:31:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39451 2024-09-17T21:31:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:31:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967915+0000", 2024-09-17T21:31:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780000000000001e-06 2024-09-17T21:31:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967922+0000", 2024-09-17T21:31:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7020000000000004e-06 2024-09-17T21:31:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967926+0000", 2024-09-17T21:31:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.809e-06 2024-09-17T21:31:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985590+0000", 2024-09-17T21:31:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017663913999999999 2024-09-17T21:31:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985627+0000", 2024-09-17T21:31:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6896e-05 2024-09-17T21:31:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985665+0000", 2024-09-17T21:31:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7793999999999998e-05 2024-09-17T21:31:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39452 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1236992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.046222438000001, 2024-09-17T21:31:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017829602999999999, 2024-09-17T21:31:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39452 2024-09-17T21:31:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:31:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967947+0000", 2024-09-17T21:31:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4280999999999999e-05 2024-09-17T21:31:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967954+0000", 2024-09-17T21:31:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3249999999999996e-06 2024-09-17T21:31:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967959+0000", 2024-09-17T21:31:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2299999999999999e-06 2024-09-17T21:31:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985687+0000", 2024-09-17T21:31:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017727844999999999 2024-09-17T21:31:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985725+0000", 2024-09-17T21:31:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8243e-05 2024-09-17T21:31:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985762+0000", 2024-09-17T21:31:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7023000000000001e-05 2024-09-17T21:31:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39453 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:31:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.046185949999995, 2024-09-17T21:31:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017893793000000002, 2024-09-17T21:31:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39453 2024-09-17T21:31:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:31:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:31:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967970+0000", 2024-09-17T21:31:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:31:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967972+0000", 2024-09-17T21:31:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0229999999999999e-06 2024-09-17T21:31:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967978+0000", 2024-09-17T21:31:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0430000000000004e-06 2024-09-17T21:31:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967982+0000", 2024-09-17T21:31:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2570000000000001e-06 2024-09-17T21:31:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985782+0000", 2024-09-17T21:31:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017799683 2024-09-17T21:31:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985818+0000", 2024-09-17T21:31:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6304999999999999e-05 2024-09-17T21:31:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985863+0000", 2024-09-17T21:31:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4953999999999997e-05 2024-09-17T21:31:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39454 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1368064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:31:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.046166622000001, 2024-09-17T21:31:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017968886, 2024-09-17T21:31:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39454 2024-09-17T21:31:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:31:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:31:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967989+0000", 2024-09-17T21:31:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.15e-07 2024-09-17T21:31:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967991+0000", 2024-09-17T21:31:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.945e-06 2024-09-17T21:31:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967997+0000", 2024-09-17T21:31:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0190000000000003e-06 2024-09-17T21:31:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968001+0000", 2024-09-17T21:31:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6650000000000002e-06 2024-09-17T21:31:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985883+0000", 2024-09-17T21:31:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017881971999999999 2024-09-17T21:31:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985920+0000", 2024-09-17T21:31:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-05 2024-09-17T21:31:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985957+0000", 2024-09-17T21:31:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7370000000000003e-05 2024-09-17T21:31:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39455 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.046147447999999, 2024-09-17T21:31:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018044437999999999, 2024-09-17T21:31:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39455 2024-09-17T21:31:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:31:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968010+0000", 2024-09-17T21:31:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.886e-06 2024-09-17T21:31:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968017+0000", 2024-09-17T21:31:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:31:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968021+0000", 2024-09-17T21:31:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850000000000004e-06 2024-09-17T21:31:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985977+0000", 2024-09-17T21:31:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017956368 2024-09-17T21:31:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986015+0000", 2024-09-17T21:31:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7610000000000001e-05 2024-09-17T21:31:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986052+0000", 2024-09-17T21:31:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7109e-05 2024-09-17T21:31:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39456 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1499136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.046127147999997, 2024-09-17T21:31:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018114514000000002, 2024-09-17T21:31:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39456 2024-09-17T21:31:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-17T21:31:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968030+0000", 2024-09-17T21:31:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0049999999999999e-06 2024-09-17T21:31:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968036+0000", 2024-09-17T21:31:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0129999999999999e-06 2024-09-17T21:31:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968041+0000", 2024-09-17T21:31:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8300000000000003e-06 2024-09-17T21:31:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986071+0000", 2024-09-17T21:31:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018029620999999999 2024-09-17T21:31:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986106+0000", 2024-09-17T21:31:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4867999999999999e-05 2024-09-17T21:31:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986142+0000", 2024-09-17T21:31:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6566000000000002e-05 2024-09-17T21:31:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39457 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1564672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.046107332000005, 2024-09-17T21:31:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018188644, 2024-09-17T21:31:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39457 2024-09-17T21:31:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7199999999999999e-07 2024-09-17T21:31:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968050+0000", 2024-09-17T21:31:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1110000000000002e-06 2024-09-17T21:31:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968057+0000", 2024-09-17T21:31:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6170000000000003e-06 2024-09-17T21:31:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968062+0000", 2024-09-17T21:31:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:31:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986163+0000", 2024-09-17T21:31:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018101435999999999 2024-09-17T21:31:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986199+0000", 2024-09-17T21:31:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.591e-05 2024-09-17T21:31:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986236+0000", 2024-09-17T21:31:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7416999999999998e-05 2024-09-17T21:31:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39458 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1630208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.046080391999993, 2024-09-17T21:31:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018255856000000001, 2024-09-17T21:31:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39458 2024-09-17T21:31:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:31:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968077+0000", 2024-09-17T21:31:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.779e-06 2024-09-17T21:31:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968081+0000", 2024-09-17T21:31:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.532e-06 2024-09-17T21:31:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968085+0000", 2024-09-17T21:31:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999999e-06 2024-09-17T21:31:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986256+0000", 2024-09-17T21:31:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018171273000000002 2024-09-17T21:31:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986293+0000", 2024-09-17T21:31:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6534e-05 2024-09-17T21:31:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986330+0000", 2024-09-17T21:31:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7536000000000001e-05 2024-09-17T21:31:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39459 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1695744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:31:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.046065814000002, 2024-09-17T21:31:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334493, 2024-09-17T21:31:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39459 2024-09-17T21:31:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:31:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:31:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968090+0000", 2024-09-17T21:31:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:31:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968091+0000", 2024-09-17T21:31:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3319999999999999e-06 2024-09-17T21:31:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968094+0000", 2024-09-17T21:31:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2590000000000001e-06 2024-09-17T21:31:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968097+0000", 2024-09-17T21:31:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:31:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986350+0000", 2024-09-17T21:31:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018253364000000001 2024-09-17T21:31:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986387+0000", 2024-09-17T21:31:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6350999999999998e-05 2024-09-17T21:31:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986424+0000", 2024-09-17T21:31:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6897999999999998e-05 2024-09-17T21:31:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39460 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1761280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:31:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.046054678000004, 2024-09-17T21:31:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018415918999999999, 2024-09-17T21:31:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39460 2024-09-17T21:31:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:31:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:31:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968101+0000", 2024-09-17T21:31:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T21:31:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968102+0000", 2024-09-17T21:31:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1689999999999999e-06 2024-09-17T21:31:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968105+0000", 2024-09-17T21:31:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:31:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968108+0000", 2024-09-17T21:31:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6630000000000002e-06 2024-09-17T21:31:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986442+0000", 2024-09-17T21:31:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334487 2024-09-17T21:31:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986479+0000", 2024-09-17T21:31:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6557000000000003e-05 2024-09-17T21:31:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986516+0000", 2024-09-17T21:31:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7648999999999998e-05 2024-09-17T21:31:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39461 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1826816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.046043917999995, 2024-09-17T21:31:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018499768, 2024-09-17T21:31:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39461 2024-09-17T21:31:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4699999999999998e-07 2024-09-17T21:31:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968112+0000", 2024-09-17T21:31:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-06 2024-09-17T21:31:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968116+0000", 2024-09-17T21:31:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1080000000000001e-06 2024-09-17T21:31:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968122+0000", 2024-09-17T21:31:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1070000000000001e-06 2024-09-17T21:31:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986537+0000", 2024-09-17T21:31:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018414834000000001 2024-09-17T21:31:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986572+0000", 2024-09-17T21:31:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5726000000000003e-05 2024-09-17T21:31:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986611+0000", 2024-09-17T21:31:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8584000000000002e-05 2024-09-17T21:31:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39462 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1892352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:31:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.046029554, 2024-09-17T21:31:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018583252000000001, 2024-09-17T21:31:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39462 2024-09-17T21:31:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:31:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:31:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968126+0000", 2024-09-17T21:31:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968127+0000", 2024-09-17T21:31:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.229e-06 2024-09-17T21:31:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968131+0000", 2024-09-17T21:31:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8010000000000001e-06 2024-09-17T21:31:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968134+0000", 2024-09-17T21:31:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8880000000000001e-06 2024-09-17T21:31:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986634+0000", 2024-09-17T21:31:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018500501999999999 2024-09-17T21:31:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986671+0000", 2024-09-17T21:31:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6881000000000001e-05 2024-09-17T21:31:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986709+0000", 2024-09-17T21:31:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7546999999999998e-05 2024-09-17T21:31:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39463 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1957888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.046017938999995, 2024-09-17T21:31:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018667186999999998, 2024-09-17T21:31:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39463 2024-09-17T21:31:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:31:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968138+0000", 2024-09-17T21:31:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:31:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968142+0000", 2024-09-17T21:31:30.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.145e-06 2024-09-17T21:31:30.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968144+0000", 2024-09-17T21:31:30.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6869999999999999e-06 2024-09-17T21:31:30.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986728+0000", 2024-09-17T21:31:30.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018584163000000001 2024-09-17T21:31:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986766+0000", 2024-09-17T21:31:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7987000000000001e-05 2024-09-17T21:31:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986804+0000", 2024-09-17T21:31:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7892000000000002e-05 2024-09-17T21:31:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39464 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:31:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.046007630000005, 2024-09-17T21:31:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018753723999999999, 2024-09-17T21:31:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39464 2024-09-17T21:31:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.103 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:31:30.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:31:30.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968148+0000", 2024-09-17T21:31:30.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:31:30.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968152+0000", 2024-09-17T21:31:30.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3749999999999996e-06 2024-09-17T21:31:30.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968155+0000", 2024-09-17T21:31:30.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.371e-06 2024-09-17T21:31:30.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968158+0000", 2024-09-17T21:31:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T21:31:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986824+0000", 2024-09-17T21:31:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018665779 2024-09-17T21:31:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986863+0000", 2024-09-17T21:31:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9273000000000002e-05 2024-09-17T21:31:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986901+0000", 2024-09-17T21:31:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7978000000000001e-05 2024-09-17T21:31:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39465 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.045992231, 2024-09-17T21:31:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018825341999999998, 2024-09-17T21:31:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39465 2024-09-17T21:31:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:30.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:30.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-17T21:31:30.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968164+0000", 2024-09-17T21:31:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1289999999999999e-06 2024-09-17T21:31:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968167+0000", 2024-09-17T21:31:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1379999999999999e-06 2024-09-17T21:31:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968170+0000", 2024-09-17T21:31:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:31:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986914+0000", 2024-09-17T21:31:30.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018743829 2024-09-17T21:31:30.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986950+0000", 2024-09-17T21:31:30.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5819999999999999e-05 2024-09-17T21:31:30.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986988+0000", 2024-09-17T21:31:30.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8362000000000003e-05 2024-09-17T21:31:30.110 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.110 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.111 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.111 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.111 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.111 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:30.112 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:31:30.113 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:30.110+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:30.214 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:30.210+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:30.217 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:30.217 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:30.217 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:30.217 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:30.226 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:31:30.317 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:30.315+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:30.370 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:30.370 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:30.370 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:30.370 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:30.370 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:30.378 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:31:30.422 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:30.419+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:30.432 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:31:30.484 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:30.484 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:30.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:30.484 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:30.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.484 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:31:30.484 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:31:30.484 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 122.541290988, 2024-09-17T21:31:30.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61572000699999996, 2024-09-17T21:31:30.484 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.484 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.485 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:30.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:31:30.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:31:30.485 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T21:31:30.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.485 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:31:30.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:31:30.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939745+0000", 2024-09-17T21:31:30.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6899999999999999e-06 2024-09-17T21:31:30.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939753+0000", 2024-09-17T21:31:30.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1950000000000003e-06 2024-09-17T21:31:30.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939767+0000", 2024-09-17T21:31:30.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3429e-05 2024-09-17T21:31:30.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939776+0000", 2024-09-17T21:31:30.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:31:30.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939858+0000", 2024-09-17T21:31:30.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2496000000000004e-05 2024-09-17T21:31:30.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T21:31:30.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939868+0000", 2024-09-17T21:31:30.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1910000000000006e-06 2024-09-17T21:31:30.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940612+0000", 2024-09-17T21:31:30.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074429599999999998 2024-09-17T21:31:30.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:31:30.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940621+0000", 2024-09-17T21:31:30.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9139999999999997e-06 2024-09-17T21:31:30.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554006+0000", 2024-09-17T21:31:30.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61338505099999996 2024-09-17T21:31:30.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554166+0000", 2024-09-17T21:31:30.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015982600000000001 2024-09-17T21:31:30.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:31:30.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554296+0000", 2024-09-17T21:31:30.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013002399999999999 2024-09-17T21:31:30.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:30.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554936+0000", 2024-09-17T21:31:30.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064009400000000001 2024-09-17T21:31:30.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:30.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555389+0000", 2024-09-17T21:31:30.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045355200000000002 2024-09-17T21:31:30.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:30.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555432+0000", 2024-09-17T21:31:30.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2477e-05 2024-09-17T21:31:30.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555463+0000", 2024-09-17T21:31:30.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1612000000000002e-05 2024-09-17T21:31:30.492 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.492 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.492 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.492 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17527 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1005568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:30.492 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:31:30.492 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.07606315699999, 2024-09-17T21:31:30.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046452090000000001, 2024-09-17T21:31:30.492 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.492 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.493 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.493 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17527 2024-09-17T21:31:30.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.493 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:31:30.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:31:30.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405496+0000", 2024-09-17T21:31:30.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052498400000000004 2024-09-17T21:31:30.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405500+0000", 2024-09-17T21:31:30.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8410000000000002e-06 2024-09-17T21:31:30.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405511+0000", 2024-09-17T21:31:30.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0749e-05 2024-09-17T21:31:30.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405517+0000", 2024-09-17T21:31:30.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:31:30.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451275+0000", 2024-09-17T21:31:30.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045758134999999998 2024-09-17T21:31:30.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451355+0000", 2024-09-17T21:31:30.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9453e-05 2024-09-17T21:31:30.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451423+0000", 2024-09-17T21:31:30.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8782999999999995e-05 2024-09-17T21:31:30.496 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.497 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.497 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.497 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17615 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:30.497 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:30.497 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.07082266899999, 2024-09-17T21:31:30.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046741309000000002, 2024-09-17T21:31:30.497 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.497 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.497 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.497 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.497 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.497 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.498 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17615 2024-09-17T21:31:30.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.498 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:30.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:30.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:30.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:31:30.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410213+0000", 2024-09-17T21:31:30.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:31:30.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410217+0000", 2024-09-17T21:31:30.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8199999999999998e-06 2024-09-17T21:31:30.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410220+0000", 2024-09-17T21:31:30.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6900000000000001e-06 2024-09-17T21:31:30.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455264+0000", 2024-09-17T21:31:30.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045044002999999999 2024-09-17T21:31:30.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455338+0000", 2024-09-17T21:31:30.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4182000000000002e-05 2024-09-17T21:31:30.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.456953+0000", 2024-09-17T21:31:30.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016150089999999999 2024-09-17T21:31:30.501 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.501 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.501 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.502 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17616 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 872448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:30.502 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:30.502 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.07081061700001, 2024-09-17T21:31:30.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946039000000002, 2024-09-17T21:31:30.502 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.502 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.502 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.502 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17616 2024-09-17T21:31:30.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.503 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:30.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:30.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:30.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T21:31:30.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410225+0000", 2024-09-17T21:31:30.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3459999999999999e-06 2024-09-17T21:31:30.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410228+0000", 2024-09-17T21:31:30.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.083e-06 2024-09-17T21:31:30.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410232+0000", 2024-09-17T21:31:30.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4419999999999998e-06 2024-09-17T21:31:30.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457004+0000", 2024-09-17T21:31:30.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046771873999999998 2024-09-17T21:31:30.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457084+0000", 2024-09-17T21:31:30.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0506000000000005e-05 2024-09-17T21:31:30.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457170+0000", 2024-09-17T21:31:30.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5494000000000005e-05 2024-09-17T21:31:30.506 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.506 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.506 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.507 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17617 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:30.507 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:30.507 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.07079921499999, 2024-09-17T21:31:30.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047084168000000003, 2024-09-17T21:31:30.507 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.507 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.507 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.508 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.508 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17617 2024-09-17T21:31:30.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.508 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:30.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:30.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:30.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-17T21:31:30.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410237+0000", 2024-09-17T21:31:30.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1960000000000001e-06 2024-09-17T21:31:30.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410246+0000", 2024-09-17T21:31:30.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7880000000000001e-06 2024-09-17T21:31:30.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410250+0000", 2024-09-17T21:31:30.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.518e-06 2024-09-17T21:31:30.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457209+0000", 2024-09-17T21:31:30.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046959283999999997 2024-09-17T21:31:30.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457264+0000", 2024-09-17T21:31:30.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673999999999999e-05 2024-09-17T21:31:30.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457319+0000", 2024-09-17T21:31:30.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5497000000000001e-05 2024-09-17T21:31:30.511 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.511 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.511 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.511 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17618 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 899072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:30.511 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:30.512 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.07077977100001, 2024-09-17T21:31:30.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047204208999999997, 2024-09-17T21:31:30.512 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.512 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.512 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.512 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17618 2024-09-17T21:31:30.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.512 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:30.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:30.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:30.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8799999999999998e-07 2024-09-17T21:31:30.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410256+0000", 2024-09-17T21:31:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T21:31:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410260+0000", 2024-09-17T21:31:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6500000000000002e-06 2024-09-17T21:31:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410263+0000", 2024-09-17T21:31:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7089999999999998e-06 2024-09-17T21:31:30.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457355+0000", 2024-09-17T21:31:30.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047092756999999999 2024-09-17T21:31:30.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457407+0000", 2024-09-17T21:31:30.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1808999999999998e-05 2024-09-17T21:31:30.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457459+0000", 2024-09-17T21:31:30.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1684000000000002e-05 2024-09-17T21:31:30.516 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.516 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.516 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.516 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17619 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 913408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:30.516 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:30.516 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.07076853300001, 2024-09-17T21:31:30.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047293443999999997, 2024-09-17T21:31:30.516 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.516 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.517 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.517 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.517 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.517 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.517 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17619 2024-09-17T21:31:30.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:31:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410267+0000", 2024-09-17T21:31:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:31:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410270+0000", 2024-09-17T21:31:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.01e-06 2024-09-17T21:31:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410273+0000", 2024-09-17T21:31:30.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6410000000000002e-06 2024-09-17T21:31:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457473+0000", 2024-09-17T21:31:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047199829999999998 2024-09-17T21:31:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457517+0000", 2024-09-17T21:31:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4058999999999999e-05 2024-09-17T21:31:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457559+0000", 2024-09-17T21:31:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2404999999999998e-05 2024-09-17T21:31:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17620 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 927744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.07075736100001, 2024-09-17T21:31:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047402409, 2024-09-17T21:31:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.524 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17620 2024-09-17T21:31:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:31:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410278+0000", 2024-09-17T21:31:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1790000000000001e-06 2024-09-17T21:31:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410281+0000", 2024-09-17T21:31:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.061e-06 2024-09-17T21:31:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410284+0000", 2024-09-17T21:31:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.65e-06 2024-09-17T21:31:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457587+0000", 2024-09-17T21:31:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047303206 2024-09-17T21:31:30.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457633+0000", 2024-09-17T21:31:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6063999999999998e-05 2024-09-17T21:31:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457679+0000", 2024-09-17T21:31:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6007000000000002e-05 2024-09-17T21:31:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17621 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 940032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:31:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.070746945, 2024-09-17T21:31:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047486550000000002, 2024-09-17T21:31:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17621 2024-09-17T21:31:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.529 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:31:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:31:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410288+0000", 2024-09-17T21:31:30.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:31:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410289+0000", 2024-09-17T21:31:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1179999999999999e-06 2024-09-17T21:31:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410292+0000", 2024-09-17T21:31:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:31:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410294+0000", 2024-09-17T21:31:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T21:31:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457694+0000", 2024-09-17T21:31:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047399416999999999 2024-09-17T21:31:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457734+0000", 2024-09-17T21:31:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0528999999999998e-05 2024-09-17T21:31:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457774+0000", 2024-09-17T21:31:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9529e-05 2024-09-17T21:31:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17622 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.070736763, 2024-09-17T21:31:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047596619, 2024-09-17T21:31:30.533 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17622 2024-09-17T21:31:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:31:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410303+0000", 2024-09-17T21:31:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2979999999999999e-06 2024-09-17T21:31:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410307+0000", 2024-09-17T21:31:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:31:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410310+0000", 2024-09-17T21:31:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.146e-06 2024-09-17T21:31:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457801+0000", 2024-09-17T21:31:30.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047491322000000002 2024-09-17T21:31:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457850+0000", 2024-09-17T21:31:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8585000000000003e-05 2024-09-17T21:31:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457894+0000", 2024-09-17T21:31:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4455e-05 2024-09-17T21:31:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17623 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 968704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.07072037899999, 2024-09-17T21:31:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047689377999999998, 2024-09-17T21:31:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17623 2024-09-17T21:31:30.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:30.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:31:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410315+0000", 2024-09-17T21:31:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0499999999999999e-06 2024-09-17T21:31:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410319+0000", 2024-09-17T21:31:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.517e-06 2024-09-17T21:31:30.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410321+0000", 2024-09-17T21:31:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:31:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457922+0000", 2024-09-17T21:31:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047600638000000001 2024-09-17T21:31:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457964+0000", 2024-09-17T21:31:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2126000000000003e-05 2024-09-17T21:31:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458003+0000", 2024-09-17T21:31:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9138999999999999e-05 2024-09-17T21:31:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17624 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:30.542 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.070709795, 2024-09-17T21:31:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047798292999999999, 2024-09-17T21:31:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17624 2024-09-17T21:31:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:31:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410326+0000", 2024-09-17T21:31:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0610000000000001e-06 2024-09-17T21:31:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410329+0000", 2024-09-17T21:31:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0319999999999999e-06 2024-09-17T21:31:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410331+0000", 2024-09-17T21:31:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5749999999999999e-06 2024-09-17T21:31:30.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458033+0000", 2024-09-17T21:31:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047701371999999999 2024-09-17T21:31:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458079+0000", 2024-09-17T21:31:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:31:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458123+0000", 2024-09-17T21:31:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850999999999997e-05 2024-09-17T21:31:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17625 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.070699635, 2024-09-17T21:31:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047883051000000003, 2024-09-17T21:31:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17625 2024-09-17T21:31:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:31:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410336+0000", 2024-09-17T21:31:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1310000000000001e-06 2024-09-17T21:31:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410339+0000", 2024-09-17T21:31:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000001e-06 2024-09-17T21:31:30.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410342+0000", 2024-09-17T21:31:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7460000000000001e-06 2024-09-17T21:31:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458137+0000", 2024-09-17T21:31:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047794931999999998 2024-09-17T21:31:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458178+0000", 2024-09-17T21:31:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1013999999999997e-05 2024-09-17T21:31:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458218+0000", 2024-09-17T21:31:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9675999999999998e-05 2024-09-17T21:31:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17626 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1009664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:31:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.070689014, 2024-09-17T21:31:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047991601000000002, 2024-09-17T21:31:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17626 2024-09-17T21:31:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:31:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:31:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410346+0000", 2024-09-17T21:31:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:31:30.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410347+0000", 2024-09-17T21:31:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.114e-06 2024-09-17T21:31:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410350+0000", 2024-09-17T21:31:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.101e-06 2024-09-17T21:31:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410353+0000", 2024-09-17T21:31:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T21:31:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458245+0000", 2024-09-17T21:31:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0478921 2024-09-17T21:31:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458292+0000", 2024-09-17T21:31:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397999999999999e-05 2024-09-17T21:31:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458337+0000", 2024-09-17T21:31:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5221e-05 2024-09-17T21:31:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17627 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:31:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.07065191300001, 2024-09-17T21:31:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048047383999999999, 2024-09-17T21:31:30.556 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17627 2024-09-17T21:31:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:31:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:31:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410383+0000", 2024-09-17T21:31:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:31:30.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410385+0000", 2024-09-17T21:31:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:31:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410391+0000", 2024-09-17T21:31:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:31:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410395+0000", 2024-09-17T21:31:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6539999999999998e-06 2024-09-17T21:31:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458350+0000", 2024-09-17T21:31:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047954855999999997 2024-09-17T21:31:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458392+0000", 2024-09-17T21:31:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2094000000000001e-05 2024-09-17T21:31:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458430+0000", 2024-09-17T21:31:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7474000000000001e-05 2024-09-17T21:31:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17628 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.070632352, 2024-09-17T21:31:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048142676000000002, 2024-09-17T21:31:30.561 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17628 2024-09-17T21:31:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:31:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410404+0000", 2024-09-17T21:31:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-06 2024-09-17T21:31:30.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410411+0000", 2024-09-17T21:31:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6850000000000004e-06 2024-09-17T21:31:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410415+0000", 2024-09-17T21:31:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6750000000000001e-06 2024-09-17T21:31:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458457+0000", 2024-09-17T21:31:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048042045999999998 2024-09-17T21:31:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458501+0000", 2024-09-17T21:31:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3661000000000001e-05 2024-09-17T21:31:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458545+0000", 2024-09-17T21:31:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3529000000000003e-05 2024-09-17T21:31:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.565 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17687 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:31:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.068187399, 2024-09-17T21:31:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050670448, 2024-09-17T21:31:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.566 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17687 2024-09-17T21:31:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:31:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:31:30.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412848+0000", 2024-09-17T21:31:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9100000000000003e-07 2024-09-17T21:31:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413102+0000", 2024-09-17T21:31:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025415600000000001 2024-09-17T21:31:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413111+0000", 2024-09-17T21:31:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2010000000000005e-06 2024-09-17T21:31:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413116+0000", 2024-09-17T21:31:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:31:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458559+0000", 2024-09-17T21:31:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045442914000000001 2024-09-17T21:31:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458632+0000", 2024-09-17T21:31:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3207000000000006e-05 2024-09-17T21:31:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:31:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458785+0000", 2024-09-17T21:31:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015226700000000001 2024-09-17T21:31:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.461648+0000", 2024-09-17T21:31:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028633069999999998 2024-09-17T21:31:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463416+0000", 2024-09-17T21:31:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017680230000000001 2024-09-17T21:31:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463475+0000", 2024-09-17T21:31:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9333000000000003e-05 2024-09-17T21:31:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463517+0000", 2024-09-17T21:31:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2162000000000001e-05 2024-09-17T21:31:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.571 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17688 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [write 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.067908531, 2024-09-17T21:31:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050625098, 2024-09-17T21:31:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.572 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17688 2024-09-17T21:31:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:31:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413128+0000", 2024-09-17T21:31:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.813e-06 2024-09-17T21:31:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413132+0000", 2024-09-17T21:31:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:31:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413135+0000", 2024-09-17T21:31:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:31:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459756+0000", 2024-09-17T21:31:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046621293000000001 2024-09-17T21:31:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459816+0000", 2024-09-17T21:31:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9156000000000001e-05 2024-09-17T21:31:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:31:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459912+0000", 2024-09-17T21:31:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6302000000000001e-05 2024-09-17T21:31:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463231+0000", 2024-09-17T21:31:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033188290000000001 2024-09-17T21:31:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463701+0000", 2024-09-17T21:31:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047068199999999999 2024-09-17T21:31:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463728+0000", 2024-09-17T21:31:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6726000000000001e-05 2024-09-17T21:31:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463751+0000", 2024-09-17T21:31:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2824e-05 2024-09-17T21:31:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17691 3.d 3:b3a055f3:::rbd_data.11207744cc2c.0000000000000083:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:31:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.064363628, 2024-09-17T21:31:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049309370999999998, 2024-09-17T21:31:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17691 2024-09-17T21:31:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:31:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:31:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416672+0000", 2024-09-17T21:31:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.47e-07 2024-09-17T21:31:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416915+0000", 2024-09-17T21:31:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024344500000000001 2024-09-17T21:31:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416924+0000", 2024-09-17T21:31:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7739999999999993e-06 2024-09-17T21:31:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416929+0000", 2024-09-17T21:31:30.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6990000000000002e-06 2024-09-17T21:31:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460414+0000", 2024-09-17T21:31:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043485466 2024-09-17T21:31:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460540+0000", 2024-09-17T21:31:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012608899999999999 2024-09-17T21:31:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:31:30.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460746+0000", 2024-09-17T21:31:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000205577 2024-09-17T21:31:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463800+0000", 2024-09-17T21:31:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030542030000000001 2024-09-17T21:31:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465867+0000", 2024-09-17T21:31:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020665929999999998 2024-09-17T21:31:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465914+0000", 2024-09-17T21:31:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497000000000003e-05 2024-09-17T21:31:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465980+0000", 2024-09-17T21:31:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6081000000000006e-05 2024-09-17T21:31:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.583 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:70218 3.16 3:68692d8f:::rbd_data.1120b9193b9f.000000000000001b:head [stat,delete] snapc 1d=[1d] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-17T21:31:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.675396538999998, 2024-09-17T21:31:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.951400786, 2024-09-17T21:31:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 70218 2024-09-17T21:31:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4799999999999998e-07 2024-09-17T21:31:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805642+0000", 2024-09-17T21:31:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2009999999999999e-06 2024-09-17T21:31:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805650+0000", 2024-09-17T21:31:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1000000000000004e-06 2024-09-17T21:31:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805655+0000", 2024-09-17T21:31:30.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:31:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805722+0000", 2024-09-17T21:31:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6878000000000006e-05 2024-09-17T21:31:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:31:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805733+0000", 2024-09-17T21:31:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0895e-05 2024-09-17T21:31:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755252+0000", 2024-09-17T21:31:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.94951843800000002 2024-09-17T21:31:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755398+0000", 2024-09-17T21:31:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014622600000000001 2024-09-17T21:31:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.757039+0000", 2024-09-17T21:31:30.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001640819 2024-09-17T21:31:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:30.589 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:30.589 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:30.591 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:31:30.591 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:30.587+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:30.691 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:30.689+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:30.704 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:30.704 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:30.704 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:30.704 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:30.714 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:31:30.792 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:30.790+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:30.816 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:30.817 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:30.817 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:30.817 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:30.817 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:30.825 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:31:30.893 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:30.891+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:30.930 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97778 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 245760~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:31:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.961014388999999, 2024-09-17T21:31:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050933728999999997, 2024-09-17T21:31:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97778 2024-09-17T21:31:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:31:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:31:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966899+0000", 2024-09-17T21:31:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:31:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966900+0000", 2024-09-17T21:31:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.384e-06 2024-09-17T21:31:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966905+0000", 2024-09-17T21:31:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6530000000000002e-06 2024-09-17T21:31:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966909+0000", 2024-09-17T21:31:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2889999999999999e-06 2024-09-17T21:31:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017585+0000", 2024-09-17T21:31:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050675691000000002 2024-09-17T21:31:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017642+0000", 2024-09-17T21:31:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7167999999999999e-05 2024-09-17T21:31:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017832+0000", 2024-09-17T21:31:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019022 2024-09-17T21:31:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97779 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:31:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.960997321000001, 2024-09-17T21:31:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051245027999999998, 2024-09-17T21:31:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97779 2024-09-17T21:31:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:31:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:31:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966916+0000", 2024-09-17T21:31:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:31:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966918+0000", 2024-09-17T21:31:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8369999999999999e-06 2024-09-17T21:31:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966924+0000", 2024-09-17T21:31:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2260000000000002e-06 2024-09-17T21:31:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966928+0000", 2024-09-17T21:31:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3819999999999997e-06 2024-09-17T21:31:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017866+0000", 2024-09-17T21:31:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050938079999999997 2024-09-17T21:31:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017945+0000", 2024-09-17T21:31:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8275000000000005e-05 2024-09-17T21:31:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018161+0000", 2024-09-17T21:31:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021579499999999999 2024-09-17T21:31:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97780 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 344064~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.960973043000003, 2024-09-17T21:31:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051535301999999998, 2024-09-17T21:31:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97780 2024-09-17T21:31:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:31:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966942+0000", 2024-09-17T21:31:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7439999999999999e-06 2024-09-17T21:31:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966946+0000", 2024-09-17T21:31:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5510000000000001e-06 2024-09-17T21:31:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966951+0000", 2024-09-17T21:31:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:31:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018191+0000", 2024-09-17T21:31:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051240290000000001 2024-09-17T21:31:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018269+0000", 2024-09-17T21:31:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7416999999999994e-05 2024-09-17T21:31:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018475+0000", 2024-09-17T21:31:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206486 2024-09-17T21:31:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97789 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 40960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.960952874, 2024-09-17T21:31:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051862143999999999, 2024-09-17T21:31:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97789 2024-09-17T21:31:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5899999999999996e-07 2024-09-17T21:31:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966962+0000", 2024-09-17T21:31:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9700000000000002e-06 2024-09-17T21:31:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966969+0000", 2024-09-17T21:31:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6939999999999998e-06 2024-09-17T21:31:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966980+0000", 2024-09-17T21:31:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0679999999999999e-05 2024-09-17T21:31:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018513+0000", 2024-09-17T21:31:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051533420000000003 2024-09-17T21:31:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018595+0000", 2024-09-17T21:31:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1260999999999993e-05 2024-09-17T21:31:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018822+0000", 2024-09-17T21:31:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022756 2024-09-17T21:31:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97790 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 90112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:31:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.960922375999999, 2024-09-17T21:31:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052173479000000002, 2024-09-17T21:31:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97790 2024-09-17T21:31:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:31:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:31:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966991+0000", 2024-09-17T21:31:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9500000000000003e-07 2024-09-17T21:31:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966993+0000", 2024-09-17T21:31:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.238e-06 2024-09-17T21:31:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967000+0000", 2024-09-17T21:31:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9500000000000004e-06 2024-09-17T21:31:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967005+0000", 2024-09-17T21:31:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:31:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018852+0000", 2024-09-17T21:31:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051847250999999997 2024-09-17T21:31:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018927+0000", 2024-09-17T21:31:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4944000000000006e-05 2024-09-17T21:31:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019164+0000", 2024-09-17T21:31:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000236803 2024-09-17T21:31:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97791 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 139264~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:31:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.960900631999998, 2024-09-17T21:31:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052464594000000003, 2024-09-17T21:31:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97791 2024-09-17T21:31:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:31:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:31:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967013+0000", 2024-09-17T21:31:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:31:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967014+0000", 2024-09-17T21:31:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8679999999999999e-06 2024-09-17T21:31:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967020+0000", 2024-09-17T21:31:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.322e-06 2024-09-17T21:31:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967024+0000", 2024-09-17T21:31:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2660000000000003e-06 2024-09-17T21:31:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019193+0000", 2024-09-17T21:31:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052169010000000002 2024-09-17T21:31:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019269+0000", 2024-09-17T21:31:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6149999999999994e-05 2024-09-17T21:31:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019477+0000", 2024-09-17T21:31:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020753699999999999 2024-09-17T21:31:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97792 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 188416~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.960882892999997, 2024-09-17T21:31:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052764958000000001, 2024-09-17T21:31:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97792 2024-09-17T21:31:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0900000000000002e-07 2024-09-17T21:31:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967032+0000", 2024-09-17T21:31:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9740000000000001e-06 2024-09-17T21:31:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967039+0000", 2024-09-17T21:31:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3679999999999998e-06 2024-09-17T21:31:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967043+0000", 2024-09-17T21:31:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3379999999999998e-06 2024-09-17T21:31:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019505+0000", 2024-09-17T21:31:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052462264000000002 2024-09-17T21:31:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019582+0000", 2024-09-17T21:31:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7146999999999998e-05 2024-09-17T21:31:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019795+0000", 2024-09-17T21:31:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000212458 2024-09-17T21:31:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97793 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 237568~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:31:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.960861393000002, 2024-09-17T21:31:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053070436999999998, 2024-09-17T21:31:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97793 2024-09-17T21:31:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:31:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:31:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967052+0000", 2024-09-17T21:31:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7500000000000001e-07 2024-09-17T21:31:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967054+0000", 2024-09-17T21:31:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:31:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967059+0000", 2024-09-17T21:31:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5840000000000002e-06 2024-09-17T21:31:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967064+0000", 2024-09-17T21:31:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:31:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019826+0000", 2024-09-17T21:31:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052762396000000003 2024-09-17T21:31:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019900+0000", 2024-09-17T21:31:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3594e-05 2024-09-17T21:31:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020122+0000", 2024-09-17T21:31:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022185999999999999 2024-09-17T21:31:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97794 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 286720~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.960841256999998, 2024-09-17T21:31:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053453734000000003, 2024-09-17T21:31:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97794 2024-09-17T21:31:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2199999999999999e-07 2024-09-17T21:31:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967074+0000", 2024-09-17T21:31:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8419999999999999e-06 2024-09-17T21:31:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967080+0000", 2024-09-17T21:31:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5660000000000003e-06 2024-09-17T21:31:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967120+0000", 2024-09-17T21:31:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9504e-05 2024-09-17T21:31:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020189+0000", 2024-09-17T21:31:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053069144999999998 2024-09-17T21:31:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020287+0000", 2024-09-17T21:31:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8259999999999998e-05 2024-09-17T21:31:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020525+0000", 2024-09-17T21:31:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023799499999999999 2024-09-17T21:31:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97795 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 335872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.960783122999999, 2024-09-17T21:31:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053707207999999999, 2024-09-17T21:31:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97795 2024-09-17T21:31:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:31:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967139+0000", 2024-09-17T21:31:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5790000000000004e-06 2024-09-17T21:31:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967145+0000", 2024-09-17T21:31:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3550000000000001e-06 2024-09-17T21:31:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967155+0000", 2024-09-17T21:31:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8859999999999999e-06 2024-09-17T21:31:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020555+0000", 2024-09-17T21:31:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053400359000000001 2024-09-17T21:31:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020630+0000", 2024-09-17T21:31:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4894999999999994e-05 2024-09-17T21:31:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020837+0000", 2024-09-17T21:31:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020670500000000001 2024-09-17T21:31:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97796 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 385024~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.960750863000001, 2024-09-17T21:31:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053990227000000002, 2024-09-17T21:31:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97796 2024-09-17T21:31:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:31:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967164+0000", 2024-09-17T21:31:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0370000000000001e-06 2024-09-17T21:31:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967171+0000", 2024-09-17T21:31:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2190000000000001e-06 2024-09-17T21:31:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967175+0000", 2024-09-17T21:31:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2599999999999999e-06 2024-09-17T21:31:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020866+0000", 2024-09-17T21:31:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053691539000000003 2024-09-17T21:31:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020942+0000", 2024-09-17T21:31:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5487000000000004e-05 2024-09-17T21:31:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021152+0000", 2024-09-17T21:31:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021021599999999999 2024-09-17T21:31:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97805 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 8192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.960732182999998, 2024-09-17T21:31:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054331771000000001, 2024-09-17T21:31:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:30.982 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:30.982 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:30.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:30.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:30.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.982 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97805 2024-09-17T21:31:30.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.982 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:30.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:30.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:30.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:30.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:30.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:30.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:30.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:31:30.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:30.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967183+0000", 2024-09-17T21:31:30.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6840000000000001e-06 2024-09-17T21:31:30.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:30.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967189+0000", 2024-09-17T21:31:30.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0299999999999999e-06 2024-09-17T21:31:30.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967193+0000", 2024-09-17T21:31:30.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-17T21:31:30.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:30.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021182+0000", 2024-09-17T21:31:30.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053989319000000001 2024-09-17T21:31:30.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:30.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021261+0000", 2024-09-17T21:31:30.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8460000000000004e-05 2024-09-17T21:31:30.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:30.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021512+0000", 2024-09-17T21:31:30.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025158500000000002 2024-09-17T21:31:30.985 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.985 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:30.985 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97806 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.960714179999997, 2024-09-17T21:31:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054654343000000001, 2024-09-17T21:31:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97806 2024-09-17T21:31:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:30.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:30.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:30.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:30.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:30.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:30.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:30.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:31:30.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:30.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967201+0000", 2024-09-17T21:31:30.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.691e-06 2024-09-17T21:31:30.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:30.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967206+0000", 2024-09-17T21:31:30.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7780000000000004e-06 2024-09-17T21:31:30.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967224+0000", 2024-09-17T21:31:30.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7357999999999999e-05 2024-09-17T21:31:30.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:30.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021546+0000", 2024-09-17T21:31:30.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054322237000000002 2024-09-17T21:31:30.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:30.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021618+0000", 2024-09-17T21:31:30.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2007000000000004e-05 2024-09-17T21:31:30.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:30.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021853+0000", 2024-09-17T21:31:30.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023495800000000001 2024-09-17T21:31:30.990 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.990 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:30.990 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.990 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97807 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 106496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:30.990 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:31:30.990 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.960681780000002, 2024-09-17T21:31:30.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055014183000000001, 2024-09-17T21:31:30.990 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:30.990 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:30.990 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:30.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:30.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:30.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.991 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97807 2024-09-17T21:31:30.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.991 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:30.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:30.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:31:30.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:30.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:31:30.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:30.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967232+0000", 2024-09-17T21:31:30.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:31:30.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:30.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967234+0000", 2024-09-17T21:31:30.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.283e-06 2024-09-17T21:31:30.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:30.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967240+0000", 2024-09-17T21:31:30.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.144e-06 2024-09-17T21:31:30.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967252+0000", 2024-09-17T21:31:30.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2322e-05 2024-09-17T21:31:30.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:30.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021882+0000", 2024-09-17T21:31:30.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054629952000000002 2024-09-17T21:31:30.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:30.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021952+0000", 2024-09-17T21:31:30.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9919999999999995e-05 2024-09-17T21:31:30.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:30.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022245+0000", 2024-09-17T21:31:30.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00029302100000000003 2024-09-17T21:31:30.994 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.994 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:30.994 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.995 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97808 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 155648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:30.995 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:30.995 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.960653180999998, 2024-09-17T21:31:30.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055447485999999997, 2024-09-17T21:31:30.995 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:30.995 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:30.995 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:30.995 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:30.995 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:30.995 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:30.995 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97808 2024-09-17T21:31:30.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.995 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:30.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:30.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:30.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:30.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:30.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:30.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:30.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:30.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4299999999999998e-07 2024-09-17T21:31:30.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:30.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967267+0000", 2024-09-17T21:31:30.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9820000000000002e-06 2024-09-17T21:31:30.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:30.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967273+0000", 2024-09-17T21:31:30.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:31:30.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:30.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967277+0000", 2024-09-17T21:31:30.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5050000000000001e-06 2024-09-17T21:31:30.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:30.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022297+0000", 2024-09-17T21:31:30.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055020131999999999 2024-09-17T21:31:30.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:30.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022379+0000", 2024-09-17T21:31:30.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1810000000000004e-05 2024-09-17T21:31:30.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:30.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022707+0000", 2024-09-17T21:31:30.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00032784300000000003 2024-09-17T21:31:30.999 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.999 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:30.999 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:30.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:30.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:30.999 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97809 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:30.999 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:31:30.999 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.960629666000003, 2024-09-17T21:31:30.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056012109999999997, 2024-09-17T21:31:30.999 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:30.999 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.000 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.000 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.000 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.000 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.000 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97809 2024-09-17T21:31:31.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.000 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:31:31.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:31:31.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967284+0000", 2024-09-17T21:31:31.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:31:31.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967285+0000", 2024-09-17T21:31:31.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.826e-06 2024-09-17T21:31:31.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967291+0000", 2024-09-17T21:31:31.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:31:31.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967302+0000", 2024-09-17T21:31:31.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1114999999999999e-05 2024-09-17T21:31:31.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022901+0000", 2024-09-17T21:31:31.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055599335 2024-09-17T21:31:31.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022990+0000", 2024-09-17T21:31:31.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8671999999999994e-05 2024-09-17T21:31:31.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023295+0000", 2024-09-17T21:31:31.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030554900000000001 2024-09-17T21:31:31.003 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.004 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.004 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.004 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97810 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 253952~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:31.004 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:31.004 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.960603237999997, 2024-09-17T21:31:31.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056355668999999997, 2024-09-17T21:31:31.004 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.004 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.004 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.004 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.004 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.005 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.005 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97810 2024-09-17T21:31:31.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.005 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:31.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:31.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:31.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0200000000000002e-07 2024-09-17T21:31:31.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967313+0000", 2024-09-17T21:31:31.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4549999999999998e-06 2024-09-17T21:31:31.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967319+0000", 2024-09-17T21:31:31.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3080000000000004e-06 2024-09-17T21:31:31.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967324+0000", 2024-09-17T21:31:31.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0159999999999999e-06 2024-09-17T21:31:31.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023342+0000", 2024-09-17T21:31:31.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056018635999999997 2024-09-17T21:31:31.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023423+0000", 2024-09-17T21:31:31.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0473e-05 2024-09-17T21:31:31.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023665+0000", 2024-09-17T21:31:31.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242279 2024-09-17T21:31:31.008 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.008 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.008 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.008 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97811 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 303104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:31.009 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:31.009 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.960576979999999, 2024-09-17T21:31:31.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056743978, 2024-09-17T21:31:31.009 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.009 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.009 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.009 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.009 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.009 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.009 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97811 2024-09-17T21:31:31.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.009 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:31.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:31.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:31.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1300000000000001e-07 2024-09-17T21:31:31.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967338+0000", 2024-09-17T21:31:31.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9740000000000001e-06 2024-09-17T21:31:31.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967346+0000", 2024-09-17T21:31:31.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8129999999999996e-06 2024-09-17T21:31:31.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967351+0000", 2024-09-17T21:31:31.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6129999999999997e-06 2024-09-17T21:31:31.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023750+0000", 2024-09-17T21:31:31.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056399796000000002 2024-09-17T21:31:31.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023839+0000", 2024-09-17T21:31:31.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.852e-05 2024-09-17T21:31:31.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024080+0000", 2024-09-17T21:31:31.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024084900000000001 2024-09-17T21:31:31.013 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.013 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.013 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.013 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97812 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 352256~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:31.013 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:31:31.013 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.960556363999999, 2024-09-17T21:31:31.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.057024455000000002, 2024-09-17T21:31:31.013 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.013 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.014 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.014 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97812 2024-09-17T21:31:31.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.014 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:31:31.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:31:31.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967357+0000", 2024-09-17T21:31:31.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:31:31.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967359+0000", 2024-09-17T21:31:31.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:31:31.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967365+0000", 2024-09-17T21:31:31.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:31:31.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967369+0000", 2024-09-17T21:31:31.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5029999999999999e-06 2024-09-17T21:31:31.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024109+0000", 2024-09-17T21:31:31.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056740038 2024-09-17T21:31:31.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024175+0000", 2024-09-17T21:31:31.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5106999999999999e-05 2024-09-17T21:31:31.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024381+0000", 2024-09-17T21:31:31.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020629700000000001 2024-09-17T21:31:31.017 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.017 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.017 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.018 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:112033 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-09-17T21:31:31.018 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:31:11.421210+0000", 2024-09-17T21:31:31.018 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.506702598, 2024-09-17T21:31:31.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0104142869999999, 2024-09-17T21:31:31.018 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.018 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.018 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.018 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.018 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.018 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.018 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 112033 2024-09-17T21:31:31.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.018 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421210+0000", 2024-09-17T21:31:31.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421210+0000", 2024-09-17T21:31:31.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421212+0000", 2024-09-17T21:31:31.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.376e-06 2024-09-17T21:31:31.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421217+0000", 2024-09-17T21:31:31.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.259e-06 2024-09-17T21:31:31.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421228+0000", 2024-09-17T21:31:31.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0944000000000001e-05 2024-09-17T21:31:31.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421238+0000", 2024-09-17T21:31:31.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:31:31.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421626+0000", 2024-09-17T21:31:31.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00038804099999999999 2024-09-17T21:31:31.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:31:31.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421637+0000", 2024-09-17T21:31:31.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1783000000000001e-05 2024-09-17T21:31:31.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:12.431245+0000", 2024-09-17T21:31:31.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.00960765 2024-09-17T21:31:31.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:12.431413+0000", 2024-09-17T21:31:31.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000167784 2024-09-17T21:31:31.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:12.431624+0000", 2024-09-17T21:31:31.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000211704 2024-09-17T21:31:31.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.023 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.023 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.023 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:31.024 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:31.022+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:31.025 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:31:31.125 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:31.123+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:31.169 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:31.169 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:31.169 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:31.169 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:31.177 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:31:31.265 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:31.259+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:31.321 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:31.321 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:31.321 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:31.321 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:31.322 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:31.327 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:31.325+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:31.330 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:31:31.428 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:31.426+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:31.475 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:37817 3.17 3:eab85a8d:::rbd_header.1120e62eea89:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:31:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:31:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 89.776919851000002, 2024-09-17T21:31:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86635970399999995, 2024-09-17T21:31:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37817 2024-09-17T21:31:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:31:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:31:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695457+0000", 2024-09-17T21:31:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5799999999999998e-07 2024-09-17T21:31:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695461+0000", 2024-09-17T21:31:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1110000000000001e-06 2024-09-17T21:31:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695471+0000", 2024-09-17T21:31:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0441e-05 2024-09-17T21:31:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695477+0000", 2024-09-17T21:31:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7599999999999999e-06 2024-09-17T21:31:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695533+0000", 2024-09-17T21:31:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6570999999999998e-05 2024-09-17T21:31:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:31:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695543+0000", 2024-09-17T21:31:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9979999999999998e-06 2024-09-17T21:31:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561472+0000", 2024-09-17T21:31:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86592841899999995 2024-09-17T21:31:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561610+0000", 2024-09-17T21:31:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013854699999999999 2024-09-17T21:31:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561816+0000", 2024-09-17T21:31:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020509900000000001 2024-09-17T21:31:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50202 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:31:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 80.364749060999998, 2024-09-17T21:31:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.076620261999999995, 2024-09-17T21:31:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50202 2024-09-17T21:31:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:31:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:31:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107629+0000", 2024-09-17T21:31:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3709999999999998e-06 2024-09-17T21:31:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119765+0000", 2024-09-17T21:31:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012135683 2024-09-17T21:31:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119787+0000", 2024-09-17T21:31:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2748999999999999e-05 2024-09-17T21:31:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119797+0000", 2024-09-17T21:31:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8169999999999994e-06 2024-09-17T21:31:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119872+0000", 2024-09-17T21:31:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270999999999994e-05 2024-09-17T21:31:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119987+0000", 2024-09-17T21:31:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011557699999999999 2024-09-17T21:31:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.121766+0000", 2024-09-17T21:31:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00177853 2024-09-17T21:31:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178307+0000", 2024-09-17T21:31:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056541559999999998 2024-09-17T21:31:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184097+0000", 2024-09-17T21:31:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057899029999999999 2024-09-17T21:31:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184172+0000", 2024-09-17T21:31:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5226000000000001e-05 2024-09-17T21:31:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184247+0000", 2024-09-17T21:31:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4574999999999997e-05 2024-09-17T21:31:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50203 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [truncate 20971520] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:31:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 80.352549155999995, 2024-09-17T21:31:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065009779000000004, 2024-09-17T21:31:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50203 2024-09-17T21:31:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:31:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:31:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119828+0000", 2024-09-17T21:31:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6390000000000001e-06 2024-09-17T21:31:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119832+0000", 2024-09-17T21:31:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7699999999999999e-06 2024-09-17T21:31:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119845+0000", 2024-09-17T21:31:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3093e-05 2024-09-17T21:31:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119852+0000", 2024-09-17T21:31:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6950000000000003e-06 2024-09-17T21:31:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.160945+0000", 2024-09-17T21:31:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041093076999999999 2024-09-17T21:31:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161071+0000", 2024-09-17T21:31:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012569799999999999 2024-09-17T21:31:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161283+0000", 2024-09-17T21:31:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021201999999999999 2024-09-17T21:31:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.179360+0000", 2024-09-17T21:31:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018077784999999999 2024-09-17T21:31:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184726+0000", 2024-09-17T21:31:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053652960000000003 2024-09-17T21:31:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184787+0000", 2024-09-17T21:31:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1402000000000006e-05 2024-09-17T21:31:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184836+0000", 2024-09-17T21:31:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9304000000000001e-05 2024-09-17T21:31:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50206 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:31:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 80.315467616000007, 2024-09-17T21:31:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.090361968000000001, 2024-09-17T21:31:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50206 2024-09-17T21:31:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:31:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:31:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156911+0000", 2024-09-17T21:31:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-17T21:31:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173670+0000", 2024-09-17T21:31:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016758907 2024-09-17T21:31:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173696+0000", 2024-09-17T21:31:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6329000000000002e-05 2024-09-17T21:31:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173708+0000", 2024-09-17T21:31:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2512e-05 2024-09-17T21:31:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173788+0000", 2024-09-17T21:31:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9332999999999994e-05 2024-09-17T21:31:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173918+0000", 2024-09-17T21:31:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013063300000000001 2024-09-17T21:31:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.176228+0000", 2024-09-17T21:31:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002309757 2024-09-17T21:31:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.245164+0000", 2024-09-17T21:31:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068936301000000005 2024-09-17T21:31:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247176+0000", 2024-09-17T21:31:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020117590000000001 2024-09-17T21:31:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247215+0000", 2024-09-17T21:31:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9094000000000002e-05 2024-09-17T21:31:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247270+0000", 2024-09-17T21:31:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4793999999999997e-05 2024-09-17T21:31:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50207 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [truncate 21556327] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:31:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 80.298618636, 2024-09-17T21:31:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.074055363999999999, 2024-09-17T21:31:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50207 2024-09-17T21:31:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:31:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:31:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173759+0000", 2024-09-17T21:31:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8160000000000001e-06 2024-09-17T21:31:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173763+0000", 2024-09-17T21:31:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5360000000000003e-06 2024-09-17T21:31:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173771+0000", 2024-09-17T21:31:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8849999999999999e-06 2024-09-17T21:31:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173777+0000", 2024-09-17T21:31:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7799999999999997e-06 2024-09-17T21:31:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221363+0000", 2024-09-17T21:31:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047585493 2024-09-17T21:31:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221430+0000", 2024-09-17T21:31:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7129000000000001e-05 2024-09-17T21:31:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221550+0000", 2024-09-17T21:31:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120658 2024-09-17T21:31:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246113+0000", 2024-09-17T21:31:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02456262 2024-09-17T21:31:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247744+0000", 2024-09-17T21:31:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001630692 2024-09-17T21:31:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247777+0000", 2024-09-17T21:31:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3015000000000002e-05 2024-09-17T21:31:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247812+0000", 2024-09-17T21:31:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.574e-05 2024-09-17T21:31:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50828 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 79.990506703999998, 2024-09-17T21:31:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047324185999999997, 2024-09-17T21:31:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50828 2024-09-17T21:31:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.46e-07 2024-09-17T21:31:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481871+0000", 2024-09-17T21:31:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T21:31:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481878+0000", 2024-09-17T21:31:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:31:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481882+0000", 2024-09-17T21:31:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:31:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529038+0000", 2024-09-17T21:31:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047156034999999999 2024-09-17T21:31:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529099+0000", 2024-09-17T21:31:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0316000000000003e-05 2024-09-17T21:31:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529193+0000", 2024-09-17T21:31:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4369000000000004e-05 2024-09-17T21:31:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50829 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 12845056~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 79.990487909999999, 2024-09-17T21:31:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047887596999999997, 2024-09-17T21:31:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50829 2024-09-17T21:31:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:31:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481890+0000", 2024-09-17T21:31:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.877e-06 2024-09-17T21:31:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481895+0000", 2024-09-17T21:31:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:31:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481900+0000", 2024-09-17T21:31:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4440000000000001e-06 2024-09-17T21:31:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529225+0000", 2024-09-17T21:31:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047325117999999999 2024-09-17T21:31:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529292+0000", 2024-09-17T21:31:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7100000000000005e-05 2024-09-17T21:31:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529775+0000", 2024-09-17T21:31:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048322799999999999 2024-09-17T21:31:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50830 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:31:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 79.990469508000004, 2024-09-17T21:31:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048062123999999998, 2024-09-17T21:31:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50830 2024-09-17T21:31:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:31:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:31:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481907+0000", 2024-09-17T21:31:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:31:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481909+0000", 2024-09-17T21:31:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T21:31:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481915+0000", 2024-09-17T21:31:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.02e-06 2024-09-17T21:31:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481920+0000", 2024-09-17T21:31:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6799999999999998e-06 2024-09-17T21:31:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529793+0000", 2024-09-17T21:31:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047873042999999997 2024-09-17T21:31:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529870+0000", 2024-09-17T21:31:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6967999999999999e-05 2024-09-17T21:31:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529968+0000", 2024-09-17T21:31:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8085999999999996e-05 2024-09-17T21:31:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50831 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 18087936~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 79.990449162000004, 2024-09-17T21:31:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048663304999999997, 2024-09-17T21:31:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50831 2024-09-17T21:31:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:31:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481929+0000", 2024-09-17T21:31:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2189999999999998e-06 2024-09-17T21:31:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481935+0000", 2024-09-17T21:31:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.383e-06 2024-09-17T21:31:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481939+0000", 2024-09-17T21:31:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2479999999999998e-06 2024-09-17T21:31:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530095+0000", 2024-09-17T21:31:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048156617999999998 2024-09-17T21:31:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530159+0000", 2024-09-17T21:31:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3394999999999999e-05 2024-09-17T21:31:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530590+0000", 2024-09-17T21:31:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043101199999999999 2024-09-17T21:31:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50832 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 19660800~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:31:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 79.990430515, 2024-09-17T21:31:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049339466999999998, 2024-09-17T21:31:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50832 2024-09-17T21:31:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:31:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:31:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481946+0000", 2024-09-17T21:31:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481951+0000", 2024-09-17T21:31:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1479999999999997e-06 2024-09-17T21:31:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481956+0000", 2024-09-17T21:31:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7339999999999996e-06 2024-09-17T21:31:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481961+0000", 2024-09-17T21:31:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4889999999999997e-06 2024-09-17T21:31:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530630+0000", 2024-09-17T21:31:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048668596000000001 2024-09-17T21:31:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530720+0000", 2024-09-17T21:31:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0163999999999999e-05 2024-09-17T21:31:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531285+0000", 2024-09-17T21:31:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056493199999999998 2024-09-17T21:31:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50833 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:31:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 79.990408371000001, 2024-09-17T21:31:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049908069999999999, 2024-09-17T21:31:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50833 2024-09-17T21:31:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:31:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:31:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481968+0000", 2024-09-17T21:31:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5600000000000001e-07 2024-09-17T21:31:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481970+0000", 2024-09-17T21:31:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:31:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481975+0000", 2024-09-17T21:31:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8499999999999999e-06 2024-09-17T21:31:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481980+0000", 2024-09-17T21:31:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-17T21:31:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531337+0000", 2024-09-17T21:31:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049356874000000002 2024-09-17T21:31:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531392+0000", 2024-09-17T21:31:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4826e-05 2024-09-17T21:31:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531875+0000", 2024-09-17T21:31:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048345099999999998 2024-09-17T21:31:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50834 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 24903680~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:31:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 79.990383602999998, 2024-09-17T21:31:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050534015000000002, 2024-09-17T21:31:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50834 2024-09-17T21:31:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:31:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:31:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481993+0000", 2024-09-17T21:31:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:31:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481995+0000", 2024-09-17T21:31:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.199e-06 2024-09-17T21:31:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482001+0000", 2024-09-17T21:31:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.772e-06 2024-09-17T21:31:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482006+0000", 2024-09-17T21:31:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0660000000000003e-06 2024-09-17T21:31:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531923+0000", 2024-09-17T21:31:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049917200000000002 2024-09-17T21:31:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532004+0000", 2024-09-17T21:31:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0758000000000002e-05 2024-09-17T21:31:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532526+0000", 2024-09-17T21:31:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00052258899999999995 2024-09-17T21:31:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50835 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 28573696~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:31:31.540 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 79.990363619999997, 2024-09-17T21:31:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051204701999999998, 2024-09-17T21:31:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50835 2024-09-17T21:31:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:31:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:31:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482013+0000", 2024-09-17T21:31:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:31:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482015+0000", 2024-09-17T21:31:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.272e-06 2024-09-17T21:31:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482020+0000", 2024-09-17T21:31:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3040000000000004e-06 2024-09-17T21:31:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482025+0000", 2024-09-17T21:31:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5650000000000003e-06 2024-09-17T21:31:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532569+0000", 2024-09-17T21:31:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050544661999999997 2024-09-17T21:31:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532649+0000", 2024-09-17T21:31:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9640000000000003e-05 2024-09-17T21:31:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533217+0000", 2024-09-17T21:31:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056784100000000003 2024-09-17T21:31:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50836 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 30146560~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 79.990344003000004, 2024-09-17T21:31:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051875589, 2024-09-17T21:31:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50836 2024-09-17T21:31:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3700000000000001e-07 2024-09-17T21:31:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482034+0000", 2024-09-17T21:31:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0059999999999999e-06 2024-09-17T21:31:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482040+0000", 2024-09-17T21:31:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:31:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482044+0000", 2024-09-17T21:31:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:31:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533260+0000", 2024-09-17T21:31:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051215416 2024-09-17T21:31:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533348+0000", 2024-09-17T21:31:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8695000000000004e-05 2024-09-17T21:31:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533907+0000", 2024-09-17T21:31:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055902199999999995 2024-09-17T21:31:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50844 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:31:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 79.990180577000004, 2024-09-17T21:31:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0475644, 2024-09-17T21:31:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50844 2024-09-17T21:31:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:31:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:31:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482196+0000", 2024-09-17T21:31:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.39e-07 2024-09-17T21:31:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482198+0000", 2024-09-17T21:31:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0099999999999998e-06 2024-09-17T21:31:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482204+0000", 2024-09-17T21:31:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.93e-06 2024-09-17T21:31:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482208+0000", 2024-09-17T21:31:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7439999999999998e-06 2024-09-17T21:31:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529318+0000", 2024-09-17T21:31:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047110051999999999 2024-09-17T21:31:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529396+0000", 2024-09-17T21:31:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7712000000000004e-05 2024-09-17T21:31:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529760+0000", 2024-09-17T21:31:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00036351299999999999 2024-09-17T21:31:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50845 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 21757952~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:31:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 79.990161474000004, 2024-09-17T21:31:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048084625999999998, 2024-09-17T21:31:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50845 2024-09-17T21:31:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:31:31.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:31:31.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482215+0000", 2024-09-17T21:31:31.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:31:31.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482216+0000", 2024-09-17T21:31:31.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.823e-06 2024-09-17T21:31:31.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482222+0000", 2024-09-17T21:31:31.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.507e-06 2024-09-17T21:31:31.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482227+0000", 2024-09-17T21:31:31.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6609999999999999e-06 2024-09-17T21:31:31.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529792+0000", 2024-09-17T21:31:31.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047565075999999998 2024-09-17T21:31:31.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529871+0000", 2024-09-17T21:31:31.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8855000000000003e-05 2024-09-17T21:31:31.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530299+0000", 2024-09-17T21:31:31.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000428307 2024-09-17T21:31:31.567 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.567 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.567 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.567 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50846 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 25427968~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.567 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:31.568 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 79.990139087000003, 2024-09-17T21:31:31.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048735142000000002, 2024-09-17T21:31:31.568 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.568 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.568 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.568 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.568 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.569 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.569 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50846 2024-09-17T21:31:31.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.569 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:31.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:31.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:31.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:31:31.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482239+0000", 2024-09-17T21:31:31.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.871e-06 2024-09-17T21:31:31.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482244+0000", 2024-09-17T21:31:31.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.49e-06 2024-09-17T21:31:31.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482249+0000", 2024-09-17T21:31:31.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6269999999999999e-06 2024-09-17T21:31:31.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530347+0000", 2024-09-17T21:31:31.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048097721000000003 2024-09-17T21:31:31.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530437+0000", 2024-09-17T21:31:31.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0604000000000002e-05 2024-09-17T21:31:31.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530972+0000", 2024-09-17T21:31:31.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053445000000000001 2024-09-17T21:31:31.574 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.574 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.574 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.574 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50847 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 27000832~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.575 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:31.575 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 79.990121191, 2024-09-17T21:31:31.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049308668, 2024-09-17T21:31:31.575 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.575 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.575 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.575 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.575 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.575 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.575 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50847 2024-09-17T21:31:31.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.575 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:31.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:31.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:31.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:31:31.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482257+0000", 2024-09-17T21:31:31.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.846e-06 2024-09-17T21:31:31.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482264+0000", 2024-09-17T21:31:31.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8170000000000001e-06 2024-09-17T21:31:31.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482283+0000", 2024-09-17T21:31:31.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9718e-05 2024-09-17T21:31:31.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531018+0000", 2024-09-17T21:31:31.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048734497000000002 2024-09-17T21:31:31.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531080+0000", 2024-09-17T21:31:31.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1795e-05 2024-09-17T21:31:31.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531563+0000", 2024-09-17T21:31:31.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048366499999999998 2024-09-17T21:31:31.579 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.579 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.579 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.580 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50848 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 30670848~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.580 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:31:31.580 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 79.990084648999996, 2024-09-17T21:31:31.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049893337000000003, 2024-09-17T21:31:31.580 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.580 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.580 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.580 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.580 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.581 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.581 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50848 2024-09-17T21:31:31.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.581 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:31:31.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482292+0000", 2024-09-17T21:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T21:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482294+0000", 2024-09-17T21:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3209999999999999e-06 2024-09-17T21:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482300+0000", 2024-09-17T21:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7189999999999998e-06 2024-09-17T21:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482305+0000", 2024-09-17T21:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T21:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531606+0000", 2024-09-17T21:31:31.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049301273999999999 2024-09-17T21:31:31.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531696+0000", 2024-09-17T21:31:31.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0161999999999995e-05 2024-09-17T21:31:31.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532184+0000", 2024-09-17T21:31:31.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048806400000000002 2024-09-17T21:31:31.585 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.585 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.585 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.585 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50849 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 32243712~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:31.585 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:31:31.585 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 79.990065208000004, 2024-09-17T21:31:31.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050444607000000002, 2024-09-17T21:31:31.585 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.585 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.585 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.585 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.586 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.586 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.586 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50849 2024-09-17T21:31:31.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.586 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:31:31.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:31:31.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482311+0000", 2024-09-17T21:31:31.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:31.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482316+0000", 2024-09-17T21:31:31.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1209999999999998e-06 2024-09-17T21:31:31.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482321+0000", 2024-09-17T21:31:31.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4779999999999999e-06 2024-09-17T21:31:31.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482326+0000", 2024-09-17T21:31:31.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7099999999999998e-06 2024-09-17T21:31:31.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532221+0000", 2024-09-17T21:31:31.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049894936000000001 2024-09-17T21:31:31.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532287+0000", 2024-09-17T21:31:31.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5914000000000006e-05 2024-09-17T21:31:31.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532755+0000", 2024-09-17T21:31:31.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000468107 2024-09-17T21:31:31.589 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.590 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.590 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.590 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.590 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.590 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:31.592 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:31:31.593 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is set 2024-09-17T21:31:31.594 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:31.589+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:31.694 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:31.692+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:31.744 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:31.745 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:31.745 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:31.745 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:31.752 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:31:31.793 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:31.791+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:31.855 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:31.855 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:31.855 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:31.855 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:31.855 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:31.863 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:31:31.894 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:31.892+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:31.922 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:31:31.971 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "pg_update_log_missing(3.5 epoch 45/37 rep_tid 844 entries 45'73 (0'0) error 3:a65c40ae:::rbd_data.1120b21c3309.0000000000000126:head by client.4384.0:48900 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 45'70)", 2024-09-17T21:31:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:31:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.920805795000007, 2024-09-17T21:31:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019554365000000001, 2024-09-17T21:31:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T21:31:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:31:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:31:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047112+0000", 2024-09-17T21:31:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:31:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047121+0000", 2024-09-17T21:31:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4379999999999995e-06 2024-09-17T21:31:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047139+0000", 2024-09-17T21:31:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8235000000000001e-05 2024-09-17T21:31:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047303+0000", 2024-09-17T21:31:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016389400000000001 2024-09-17T21:31:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047396+0000", 2024-09-17T21:31:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3355000000000003e-05 2024-09-17T21:31:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066666+0000", 2024-09-17T21:31:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019269649999999999 2024-09-17T21:31:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48906 3.4 e45/25 3:2503fe27:::rbd_data.1120b21c3309.0000000000000127:head v 45'148, mlcod=45'146)", 2024-09-17T21:31:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.918679616000006, 2024-09-17T21:31:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018169739000000001, 2024-09-17T21:31:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:31:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049244+0000", 2024-09-17T21:31:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8390000000000003e-06 2024-09-17T21:31:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066411+0000", 2024-09-17T21:31:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017166818 2024-09-17T21:31:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066426+0000", 2024-09-17T21:31:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4888999999999999e-05 2024-09-17T21:31:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066611+0000", 2024-09-17T21:31:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018464499999999999 2024-09-17T21:31:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066630+0000", 2024-09-17T21:31:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9201000000000001e-05 2024-09-17T21:31:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067373+0000", 2024-09-17T21:31:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00074322600000000004 2024-09-17T21:31:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067407+0000", 2024-09-17T21:31:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4357999999999999e-05 2024-09-17T21:31:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50206 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'301, mlcod=45'299)", 2024-09-17T21:31:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:31:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 80.789202771999996, 2024-09-17T21:31:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068285810000000002, 2024-09-17T21:31:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:31:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:31:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178716+0000", 2024-09-17T21:31:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.438e-06 2024-09-17T21:31:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192044+0000", 2024-09-17T21:31:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013328370000000001 2024-09-17T21:31:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192069+0000", 2024-09-17T21:31:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4496000000000001e-05 2024-09-17T21:31:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192085+0000", 2024-09-17T21:31:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5758999999999999e-05 2024-09-17T21:31:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192167+0000", 2024-09-17T21:31:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2842000000000004e-05 2024-09-17T21:31:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192201+0000", 2024-09-17T21:31:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3127999999999999e-05 2024-09-17T21:31:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246938+0000", 2024-09-17T21:31:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054736982000000003 2024-09-17T21:31:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247000+0000", 2024-09-17T21:31:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2794999999999998e-05 2024-09-17T21:31:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50207 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'303, mlcod=45'299)", 2024-09-17T21:31:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:31:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 80.746255723000004, 2024-09-17T21:31:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026003433999999999, 2024-09-17T21:31:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:31:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:31:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221663+0000", 2024-09-17T21:31:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.271e-06 2024-09-17T21:31:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221672+0000", 2024-09-17T21:31:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6810000000000004e-06 2024-09-17T21:31:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221684+0000", 2024-09-17T21:31:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:31:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221691+0000", 2024-09-17T21:31:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:31:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221743+0000", 2024-09-17T21:31:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2868999999999998e-05 2024-09-17T21:31:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221765+0000", 2024-09-17T21:31:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1316999999999999e-05 2024-09-17T21:31:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247616+0000", 2024-09-17T21:31:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025851504000000001 2024-09-17T21:31:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247665+0000", 2024-09-17T21:31:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8739000000000002e-05 2024-09-17T21:31:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57808 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 196608~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.189956683000005, 2024-09-17T21:31:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017751475999999999, 2024-09-17T21:31:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57808 2024-09-17T21:31:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:31:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777968+0000", 2024-09-17T21:31:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8530000000000003e-06 2024-09-17T21:31:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777975+0000", 2024-09-17T21:31:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4410000000000004e-06 2024-09-17T21:31:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777980+0000", 2024-09-17T21:31:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7210000000000002e-06 2024-09-17T21:31:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795323+0000", 2024-09-17T21:31:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017343319999999999 2024-09-17T21:31:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795410+0000", 2024-09-17T21:31:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6567e-05 2024-09-17T21:31:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795712+0000", 2024-09-17T21:31:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030219200000000001 2024-09-17T21:31:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57809 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 458752~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:31:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.189930830999998, 2024-09-17T21:31:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018092869000000001, 2024-09-17T21:31:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57809 2024-09-17T21:31:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:31:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:31:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777987+0000", 2024-09-17T21:31:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T21:31:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777989+0000", 2024-09-17T21:31:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8649999999999999e-06 2024-09-17T21:31:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777994+0000", 2024-09-17T21:31:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4199999999999998e-06 2024-09-17T21:31:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778004+0000", 2024-09-17T21:31:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6900000000000004e-06 2024-09-17T21:31:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795745+0000", 2024-09-17T21:31:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017740859000000001 2024-09-17T21:31:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795815+0000", 2024-09-17T21:31:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0219e-05 2024-09-17T21:31:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796079+0000", 2024-09-17T21:31:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000264446 2024-09-17T21:31:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57810 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 720896~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:31:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.189907104, 2024-09-17T21:31:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018361253000000001, 2024-09-17T21:31:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57810 2024-09-17T21:31:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:31:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:31:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778011+0000", 2024-09-17T21:31:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:31:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778012+0000", 2024-09-17T21:31:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.872e-06 2024-09-17T21:31:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778018+0000", 2024-09-17T21:31:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8150000000000002e-06 2024-09-17T21:31:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778023+0000", 2024-09-17T21:31:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3939999999999998e-06 2024-09-17T21:31:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796106+0000", 2024-09-17T21:31:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018083446999999999 2024-09-17T21:31:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796159+0000", 2024-09-17T21:31:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3029999999999999e-05 2024-09-17T21:31:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796371+0000", 2024-09-17T21:31:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021231700000000001 2024-09-17T21:31:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57811 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.189888683999996, 2024-09-17T21:31:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018666035000000001, 2024-09-17T21:31:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57811 2024-09-17T21:31:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778031+0000", 2024-09-17T21:31:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.796e-06 2024-09-17T21:31:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778036+0000", 2024-09-17T21:31:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3290000000000001e-06 2024-09-17T21:31:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778041+0000", 2024-09-17T21:31:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5839999999999996e-06 2024-09-17T21:31:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796409+0000", 2024-09-17T21:31:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018368703 2024-09-17T21:31:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796483+0000", 2024-09-17T21:31:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3751999999999995e-05 2024-09-17T21:31:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796695+0000", 2024-09-17T21:31:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021153 2024-09-17T21:31:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57812 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1245184~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.189869787000006, 2024-09-17T21:31:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019000064000000001, 2024-09-17T21:31:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57812 2024-09-17T21:31:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:31:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778050+0000", 2024-09-17T21:31:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7e-06 2024-09-17T21:31:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778055+0000", 2024-09-17T21:31:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4049999999999999e-06 2024-09-17T21:31:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778059+0000", 2024-09-17T21:31:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1479999999999999e-06 2024-09-17T21:31:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796734+0000", 2024-09-17T21:31:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018674399000000001 2024-09-17T21:31:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796786+0000", 2024-09-17T21:31:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2879e-05 2024-09-17T21:31:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797048+0000", 2024-09-17T21:31:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026118400000000003 2024-09-17T21:31:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57813 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1507328~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:31:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.189851895000004, 2024-09-17T21:31:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019279392999999999, 2024-09-17T21:31:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57813 2024-09-17T21:31:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:31:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:31:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778066+0000", 2024-09-17T21:31:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6199999999999999e-07 2024-09-17T21:31:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778068+0000", 2024-09-17T21:31:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7990000000000001e-06 2024-09-17T21:31:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778074+0000", 2024-09-17T21:31:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:31:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778078+0000", 2024-09-17T21:31:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2579999999999997e-06 2024-09-17T21:31:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797077+0000", 2024-09-17T21:31:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018999180000000001 2024-09-17T21:31:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797123+0000", 2024-09-17T21:31:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5834999999999997e-05 2024-09-17T21:31:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797345+0000", 2024-09-17T21:31:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022191100000000001 2024-09-17T21:31:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57814 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:31:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.189833023000006, 2024-09-17T21:31:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019604947000000001, 2024-09-17T21:31:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57814 2024-09-17T21:31:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:31:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:31:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778085+0000", 2024-09-17T21:31:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:31:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778086+0000", 2024-09-17T21:31:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7850000000000001e-06 2024-09-17T21:31:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778093+0000", 2024-09-17T21:31:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6449999999999999e-06 2024-09-17T21:31:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778098+0000", 2024-09-17T21:31:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:31:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797383+0000", 2024-09-17T21:31:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019284879000000001 2024-09-17T21:31:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797431+0000", 2024-09-17T21:31:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8856000000000001e-05 2024-09-17T21:31:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797689+0000", 2024-09-17T21:31:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025785800000000002 2024-09-17T21:31:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57815 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2031616~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.189812355000001, 2024-09-17T21:31:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019910231, 2024-09-17T21:31:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57815 2024-09-17T21:31:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:31:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778118+0000", 2024-09-17T21:31:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2967e-05 2024-09-17T21:31:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778125+0000", 2024-09-17T21:31:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.601e-06 2024-09-17T21:31:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778130+0000", 2024-09-17T21:31:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1839999999999998e-06 2024-09-17T21:31:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797720+0000", 2024-09-17T21:31:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019589766000000002 2024-09-17T21:31:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797765+0000", 2024-09-17T21:31:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5281999999999998e-05 2024-09-17T21:31:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798015+0000", 2024-09-17T21:31:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025006200000000001 2024-09-17T21:31:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57816 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2293760~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.189779760999997, 2024-09-17T21:31:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020175322999999998, 2024-09-17T21:31:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57816 2024-09-17T21:31:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:31:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778140+0000", 2024-09-17T21:31:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T21:31:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778145+0000", 2024-09-17T21:31:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7320000000000003e-06 2024-09-17T21:31:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778154+0000", 2024-09-17T21:31:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6969999999999999e-06 2024-09-17T21:31:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798049+0000", 2024-09-17T21:31:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019894603 2024-09-17T21:31:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798106+0000", 2024-09-17T21:31:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7207000000000003e-05 2024-09-17T21:31:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798313+0000", 2024-09-17T21:31:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206861 2024-09-17T21:31:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57817 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2555904~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.189756514999999, 2024-09-17T21:31:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020466314999999999, 2024-09-17T21:31:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57817 2024-09-17T21:31:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2399999999999999e-07 2024-09-17T21:31:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778163+0000", 2024-09-17T21:31:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.204e-06 2024-09-17T21:31:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778169+0000", 2024-09-17T21:31:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:31:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778173+0000", 2024-09-17T21:31:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:31:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798342+0000", 2024-09-17T21:31:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020168416000000002 2024-09-17T21:31:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798388+0000", 2024-09-17T21:31:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6171000000000002e-05 2024-09-17T21:31:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798627+0000", 2024-09-17T21:31:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000239058 2024-09-17T21:31:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57818 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2818048~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:31:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.189737817999998, 2024-09-17T21:31:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020767240999999999, 2024-09-17T21:31:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57818 2024-09-17T21:31:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:31:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:31:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778180+0000", 2024-09-17T21:31:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:31:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778182+0000", 2024-09-17T21:31:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7269999999999999e-06 2024-09-17T21:31:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778188+0000", 2024-09-17T21:31:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6080000000000001e-06 2024-09-17T21:31:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778202+0000", 2024-09-17T21:31:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3369e-05 2024-09-17T21:31:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798660+0000", 2024-09-17T21:31:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020458171000000001 2024-09-17T21:31:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798704+0000", 2024-09-17T21:31:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4273000000000001e-05 2024-09-17T21:31:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798947+0000", 2024-09-17T21:31:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242651 2024-09-17T21:31:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57819 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:31:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.189706948999998, 2024-09-17T21:31:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020892340999999998, 2024-09-17T21:31:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57819 2024-09-17T21:31:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:31:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:31:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778211+0000", 2024-09-17T21:31:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:31:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778217+0000", 2024-09-17T21:31:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T21:31:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778224+0000", 2024-09-17T21:31:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3520000000000003e-06 2024-09-17T21:31:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778228+0000", 2024-09-17T21:31:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7480000000000001e-06 2024-09-17T21:31:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798977+0000", 2024-09-17T21:31:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020748571 2024-09-17T21:31:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799035+0000", 2024-09-17T21:31:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7908999999999997e-05 2024-09-17T21:31:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799103+0000", 2024-09-17T21:31:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7794999999999997e-05 2024-09-17T21:31:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57820 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.189681757000002, 2024-09-17T21:31:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021016658000000001, 2024-09-17T21:31:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57820 2024-09-17T21:31:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:31:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778238+0000", 2024-09-17T21:31:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.945e-06 2024-09-17T21:31:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778244+0000", 2024-09-17T21:31:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6520000000000003e-06 2024-09-17T21:31:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778248+0000", 2024-09-17T21:31:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5920000000000002e-06 2024-09-17T21:31:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799131+0000", 2024-09-17T21:31:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020883288999999999 2024-09-17T21:31:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799185+0000", 2024-09-17T21:31:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3223000000000002e-05 2024-09-17T21:31:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799252+0000", 2024-09-17T21:31:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7576999999999994e-05 2024-09-17T21:31:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57821 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.189663412000002, 2024-09-17T21:31:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021153528000000001, 2024-09-17T21:31:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57821 2024-09-17T21:31:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-07 2024-09-17T21:31:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778256+0000", 2024-09-17T21:31:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.604e-06 2024-09-17T21:31:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778262+0000", 2024-09-17T21:31:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7629999999999997e-06 2024-09-17T21:31:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778266+0000", 2024-09-17T21:31:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:31:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799284+0000", 2024-09-17T21:31:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021017312 2024-09-17T21:31:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799326+0000", 2024-09-17T21:31:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2639000000000003e-05 2024-09-17T21:31:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799407+0000", 2024-09-17T21:31:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1077999999999999e-05 2024-09-17T21:31:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57822 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:31:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.189645102, 2024-09-17T21:31:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021268966, 2024-09-17T21:31:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57822 2024-09-17T21:31:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:31:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:31:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778273+0000", 2024-09-17T21:31:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:31:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778274+0000", 2024-09-17T21:31:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.598e-06 2024-09-17T21:31:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778283+0000", 2024-09-17T21:31:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8279999999999992e-06 2024-09-17T21:31:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778289+0000", 2024-09-17T21:31:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5609999999999998e-06 2024-09-17T21:31:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799432+0000", 2024-09-17T21:31:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021143862999999999 2024-09-17T21:31:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799473+0000", 2024-09-17T21:31:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1075999999999998e-05 2024-09-17T21:31:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799541+0000", 2024-09-17T21:31:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7657999999999995e-05 2024-09-17T21:31:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57823 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.189622141000001, 2024-09-17T21:31:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021379770999999999, 2024-09-17T21:31:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57823 2024-09-17T21:31:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:31:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778297+0000", 2024-09-17T21:31:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:31:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778303+0000", 2024-09-17T21:31:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4559999999999999e-06 2024-09-17T21:31:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778307+0000", 2024-09-17T21:31:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6229999999999996e-06 2024-09-17T21:31:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799560+0000", 2024-09-17T21:31:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021252931999999999 2024-09-17T21:31:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799609+0000", 2024-09-17T21:31:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8301999999999999e-05 2024-09-17T21:31:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799675+0000", 2024-09-17T21:31:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6205999999999996e-05 2024-09-17T21:31:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.072 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:32.074 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:31:32.074 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:32.075 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:32.075 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:32.075 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:32.076 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:32.071+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:32.076 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:32.077 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:32.093 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:32.093 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:32.110 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:32.111 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:32.127 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:32.127 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:32.144 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:32.144 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:32.161 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:32.161 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:32.174 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:32.172+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:32.182 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:32.182 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:32.191 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:31:32.275 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:32.273+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:32.334 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:32.335 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:32.343 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:31:32.376 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:32.374+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:32.429 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-17T21:31:32.429 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:31:32.430 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T21:31:32.430 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T21:31:32.477 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:32.474+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:32.500 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:32.500 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:32.510 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:31:32.578 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:32.575+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:32.618 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:32.618 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:32.627 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:31:32.678 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:32.676+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:32.730 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:32.730 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:32.733 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:31:32.733 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:32.733 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 123, 2024-09-17T21:31:32.733 INFO:teuthology.orchestra.run.smithi129.stdout: "fsid": "f6d3a80c-8a1f-4716-b25d-e97b28890c0d", 2024-09-17T21:31:32.733 INFO:teuthology.orchestra.run.smithi129.stdout: "created": "2024-09-17T21:29:12.606637+0000", 2024-09-17T21:31:32.733 INFO:teuthology.orchestra.run.smithi129.stdout: "modified": "2024-09-17T21:31:32.407094+0000", 2024-09-17T21:31:32.734 INFO:teuthology.orchestra.run.smithi129.stdout: "last_up_change": "2024-09-17T21:30:27.721456+0000", 2024-09-17T21:31:32.734 INFO:teuthology.orchestra.run.smithi129.stdout: "last_in_change": "2024-09-17T21:31:10.410190+0000", 2024-09-17T21:31:32.734 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T21:31:32.734 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_num": 5806080, 2024-09-17T21:31:32.734 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_set": [ 2024-09-17T21:31:32.734 INFO:teuthology.orchestra.run.smithi129.stdout: "nodeep-scrub", 2024-09-17T21:31:32.734 INFO:teuthology.orchestra.run.smithi129.stdout: "noscrub", 2024-09-17T21:31:32.734 INFO:teuthology.orchestra.run.smithi129.stdout: "pglog_hardlimit", 2024-09-17T21:31:32.734 INFO:teuthology.orchestra.run.smithi129.stdout: "purged_snapdirs", 2024-09-17T21:31:32.734 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_deletes", 2024-09-17T21:31:32.734 INFO:teuthology.orchestra.run.smithi129.stdout: "sortbitwise" 2024-09-17T21:31:32.734 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:31:32.734 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_version": 6, 2024-09-17T21:31:32.735 INFO:teuthology.orchestra.run.smithi129.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T21:31:32.735 INFO:teuthology.orchestra.run.smithi129.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T21:31:32.735 INFO:teuthology.orchestra.run.smithi129.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T21:31:32.735 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_snapshot": "", 2024-09-17T21:31:32.735 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_max": 3, 2024-09-17T21:31:32.735 INFO:teuthology.orchestra.run.smithi129.stdout: "max_osd": 8, 2024-09-17T21:31:32.735 INFO:teuthology.orchestra.run.smithi129.stdout: "require_min_compat_client": "luminous", 2024-09-17T21:31:32.735 INFO:teuthology.orchestra.run.smithi129.stdout: "min_compat_client": "jewel", 2024-09-17T21:31:32.735 INFO:teuthology.orchestra.run.smithi129.stdout: "require_osd_release": "squid", 2024-09-17T21:31:32.735 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_crimson": false, 2024-09-17T21:31:32.735 INFO:teuthology.orchestra.run.smithi129.stdout: "pools": [ 2024-09-17T21:31:32.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.735 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 1, 2024-09-17T21:31:32.736 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": ".mgr", 2024-09-17T21:31:32.736 INFO:teuthology.orchestra.run.smithi129.stdout: "create_time": "2024-09-17T21:29:26.876763+0000", 2024-09-17T21:31:32.736 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-17T21:31:32.736 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_names": "hashpspool", 2024-09-17T21:31:32.736 INFO:teuthology.orchestra.run.smithi129.stdout: "type": 1, 2024-09-17T21:31:32.736 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 2, 2024-09-17T21:31:32.736 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": 1, 2024-09-17T21:31:32.736 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_rule": 0, 2024-09-17T21:31:32.736 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:31:32.736 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:31:32.736 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:31:32.736 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:31:32.736 INFO:teuthology.orchestra.run.smithi129.stdout: "is_stretch_pool": false, 2024-09-17T21:31:32.737 INFO:teuthology.orchestra.run.smithi129.stdout: "object_hash": 2, 2024-09-17T21:31:32.737 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:31:32.737 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": 1, 2024-09-17T21:31:32.737 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num": 1, 2024-09-17T21:31:32.737 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num_target": 1, 2024-09-17T21:31:32.737 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_target": 1, 2024-09-17T21:31:32.737 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_pending": 1, 2024-09-17T21:31:32.737 INFO:teuthology.orchestra.run.smithi129.stdout: "last_pg_merge_meta": { 2024-09-17T21:31:32.737 INFO:teuthology.orchestra.run.smithi129.stdout: "source_pgid": "0.0", 2024-09-17T21:31:32.737 INFO:teuthology.orchestra.run.smithi129.stdout: "ready_epoch": 0, 2024-09-17T21:31:32.737 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_started": 0, 2024-09-17T21:31:32.737 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_clean": 0, 2024-09-17T21:31:32.737 INFO:teuthology.orchestra.run.smithi129.stdout: "source_version": "0'0", 2024-09-17T21:31:32.738 INFO:teuthology.orchestra.run.smithi129.stdout: "target_version": "0'0" 2024-09-17T21:31:32.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.738 INFO:teuthology.orchestra.run.smithi129.stdout: "last_change": "19", 2024-09-17T21:31:32.738 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend": "0", 2024-09-17T21:31:32.738 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:31:32.738 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:31:32.738 INFO:teuthology.orchestra.run.smithi129.stdout: "auid": 0, 2024-09-17T21:31:32.738 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:31:32.738 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_seq": 0, 2024-09-17T21:31:32.738 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_epoch": 0, 2024-09-17T21:31:32.738 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_snaps": [], 2024-09-17T21:31:32.738 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps": "[]", 2024-09-17T21:31:32.738 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_bytes": 0, 2024-09-17T21:31:32.738 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_objects": 0, 2024-09-17T21:31:32.739 INFO:teuthology.orchestra.run.smithi129.stdout: "tiers": [], 2024-09-17T21:31:32.739 INFO:teuthology.orchestra.run.smithi129.stdout: "tier_of": -1, 2024-09-17T21:31:32.739 INFO:teuthology.orchestra.run.smithi129.stdout: "read_tier": -1, 2024-09-17T21:31:32.739 INFO:teuthology.orchestra.run.smithi129.stdout: "write_tier": -1, 2024-09-17T21:31:32.739 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_mode": "none", 2024-09-17T21:31:32.739 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_bytes": 0, 2024-09-17T21:31:32.739 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_objects": 0, 2024-09-17T21:31:32.739 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:31:32.739 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:31:32.739 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:31:32.739 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_flush_age": 0, 2024-09-17T21:31:32.739 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_evict_age": 0, 2024-09-17T21:31:32.739 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profile": "", 2024-09-17T21:31:32.739 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_params": { 2024-09-17T21:31:32.740 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "none" 2024-09-17T21:31:32.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.740 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_period": 0, 2024-09-17T21:31:32.740 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_count": 0, 2024-09-17T21:31:32.740 INFO:teuthology.orchestra.run.smithi129.stdout: "use_gmt_hitset": true, 2024-09-17T21:31:32.740 INFO:teuthology.orchestra.run.smithi129.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:31:32.740 INFO:teuthology.orchestra.run.smithi129.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:31:32.740 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:31:32.740 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:31:32.740 INFO:teuthology.orchestra.run.smithi129.stdout: "grade_table": [], 2024-09-17T21:31:32.740 INFO:teuthology.orchestra.run.smithi129.stdout: "stripe_width": 0, 2024-09-17T21:31:32.740 INFO:teuthology.orchestra.run.smithi129.stdout: "expected_num_objects": 0, 2024-09-17T21:31:32.740 INFO:teuthology.orchestra.run.smithi129.stdout: "fast_read": false, 2024-09-17T21:31:32.741 INFO:teuthology.orchestra.run.smithi129.stdout: "options": { 2024-09-17T21:31:32.741 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_max": 32, 2024-09-17T21:31:32.741 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_min": 1 2024-09-17T21:31:32.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.741 INFO:teuthology.orchestra.run.smithi129.stdout: "application_metadata": { 2024-09-17T21:31:32.741 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr": {} 2024-09-17T21:31:32.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.741 INFO:teuthology.orchestra.run.smithi129.stdout: "read_balance": { 2024-09-17T21:31:32.741 INFO:teuthology.orchestra.run.smithi129.stdout: "score_type": "Fair distribution", 2024-09-17T21:31:32.741 INFO:teuthology.orchestra.run.smithi129.stdout: "score_acting": 8, 2024-09-17T21:31:32.741 INFO:teuthology.orchestra.run.smithi129.stdout: "score_stable": 8, 2024-09-17T21:31:32.741 INFO:teuthology.orchestra.run.smithi129.stdout: "optimal_score": 0.25, 2024-09-17T21:31:32.741 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_acting": 2, 2024-09-17T21:31:32.741 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_stable": 2, 2024-09-17T21:31:32.742 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity_weighted": 0.90080642700195312, 2024-09-17T21:31:32.742 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-17T21:31:32.742 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-17T21:31:32.742 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.742 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 2, 2024-09-17T21:31:32.742 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": "rbd", 2024-09-17T21:31:32.742 INFO:teuthology.orchestra.run.smithi129.stdout: "create_time": "2024-09-17T21:29:31.078466+0000", 2024-09-17T21:31:32.742 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 8193, 2024-09-17T21:31:32.742 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:31:32.742 INFO:teuthology.orchestra.run.smithi129.stdout: "type": 1, 2024-09-17T21:31:32.742 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 2, 2024-09-17T21:31:32.742 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": 1, 2024-09-17T21:31:32.743 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_rule": 0, 2024-09-17T21:31:32.743 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:31:32.743 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:31:32.743 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:31:32.743 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:31:32.743 INFO:teuthology.orchestra.run.smithi129.stdout: "is_stretch_pool": false, 2024-09-17T21:31:32.743 INFO:teuthology.orchestra.run.smithi129.stdout: "object_hash": 2, 2024-09-17T21:31:32.743 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:31:32.743 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": 8, 2024-09-17T21:31:32.743 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num": 8, 2024-09-17T21:31:32.743 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num_target": 8, 2024-09-17T21:31:32.744 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_target": 8, 2024-09-17T21:31:32.744 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_pending": 8, 2024-09-17T21:31:32.744 INFO:teuthology.orchestra.run.smithi129.stdout: "last_pg_merge_meta": { 2024-09-17T21:31:32.744 INFO:teuthology.orchestra.run.smithi129.stdout: "source_pgid": "0.0", 2024-09-17T21:31:32.744 INFO:teuthology.orchestra.run.smithi129.stdout: "ready_epoch": 0, 2024-09-17T21:31:32.744 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_started": 0, 2024-09-17T21:31:32.744 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_clean": 0, 2024-09-17T21:31:32.744 INFO:teuthology.orchestra.run.smithi129.stdout: "source_version": "0'0", 2024-09-17T21:31:32.744 INFO:teuthology.orchestra.run.smithi129.stdout: "target_version": "0'0" 2024-09-17T21:31:32.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.744 INFO:teuthology.orchestra.run.smithi129.stdout: "last_change": "23", 2024-09-17T21:31:32.744 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend": "0", 2024-09-17T21:31:32.744 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:31:32.744 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:31:32.745 INFO:teuthology.orchestra.run.smithi129.stdout: "auid": 0, 2024-09-17T21:31:32.745 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:31:32.745 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_seq": 2, 2024-09-17T21:31:32.745 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_epoch": 23, 2024-09-17T21:31:32.745 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_snaps": [], 2024-09-17T21:31:32.745 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps": "[]", 2024-09-17T21:31:32.745 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_bytes": 0, 2024-09-17T21:31:32.745 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_objects": 0, 2024-09-17T21:31:32.745 INFO:teuthology.orchestra.run.smithi129.stdout: "tiers": [], 2024-09-17T21:31:32.745 INFO:teuthology.orchestra.run.smithi129.stdout: "tier_of": -1, 2024-09-17T21:31:32.745 INFO:teuthology.orchestra.run.smithi129.stdout: "read_tier": -1, 2024-09-17T21:31:32.745 INFO:teuthology.orchestra.run.smithi129.stdout: "write_tier": -1, 2024-09-17T21:31:32.745 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_mode": "none", 2024-09-17T21:31:32.746 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_bytes": 0, 2024-09-17T21:31:32.746 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_objects": 0, 2024-09-17T21:31:32.746 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:31:32.746 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:31:32.746 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:31:32.746 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_flush_age": 0, 2024-09-17T21:31:32.746 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_evict_age": 0, 2024-09-17T21:31:32.746 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profile": "", 2024-09-17T21:31:32.746 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_params": { 2024-09-17T21:31:32.746 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "none" 2024-09-17T21:31:32.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.746 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_period": 0, 2024-09-17T21:31:32.746 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_count": 0, 2024-09-17T21:31:32.746 INFO:teuthology.orchestra.run.smithi129.stdout: "use_gmt_hitset": true, 2024-09-17T21:31:32.747 INFO:teuthology.orchestra.run.smithi129.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:31:32.747 INFO:teuthology.orchestra.run.smithi129.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:31:32.747 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:31:32.747 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:31:32.747 INFO:teuthology.orchestra.run.smithi129.stdout: "grade_table": [], 2024-09-17T21:31:32.747 INFO:teuthology.orchestra.run.smithi129.stdout: "stripe_width": 0, 2024-09-17T21:31:32.747 INFO:teuthology.orchestra.run.smithi129.stdout: "expected_num_objects": 0, 2024-09-17T21:31:32.747 INFO:teuthology.orchestra.run.smithi129.stdout: "fast_read": false, 2024-09-17T21:31:32.747 INFO:teuthology.orchestra.run.smithi129.stdout: "options": {}, 2024-09-17T21:31:32.747 INFO:teuthology.orchestra.run.smithi129.stdout: "application_metadata": { 2024-09-17T21:31:32.747 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd": {} 2024-09-17T21:31:32.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.747 INFO:teuthology.orchestra.run.smithi129.stdout: "read_balance": { 2024-09-17T21:31:32.748 INFO:teuthology.orchestra.run.smithi129.stdout: "score_type": "Fair distribution", 2024-09-17T21:31:32.748 INFO:teuthology.orchestra.run.smithi129.stdout: "score_acting": 1.9900000095367432, 2024-09-17T21:31:32.748 INFO:teuthology.orchestra.run.smithi129.stdout: "score_stable": 1.9900000095367432, 2024-09-17T21:31:32.748 INFO:teuthology.orchestra.run.smithi129.stdout: "optimal_score": 0.87999999523162842, 2024-09-17T21:31:32.748 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_acting": 1.75, 2024-09-17T21:31:32.748 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_stable": 1.75, 2024-09-17T21:31:32.748 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity_weighted": 0.90080642700195312, 2024-09-17T21:31:32.748 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-17T21:31:32.748 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-17T21:31:32.748 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.748 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 3, 2024-09-17T21:31:32.749 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": "pool_client.0", 2024-09-17T21:31:32.749 INFO:teuthology.orchestra.run.smithi129.stdout: "create_time": "2024-09-17T21:29:39.892418+0000", 2024-09-17T21:31:32.749 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 8193, 2024-09-17T21:31:32.749 INFO:teuthology.orchestra.run.smithi129.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:31:32.749 INFO:teuthology.orchestra.run.smithi129.stdout: "type": 1, 2024-09-17T21:31:32.749 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 2, 2024-09-17T21:31:32.749 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": 1, 2024-09-17T21:31:32.749 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_rule": 0, 2024-09-17T21:31:32.749 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:31:32.749 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:31:32.749 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:31:32.749 INFO:teuthology.orchestra.run.smithi129.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:31:32.749 INFO:teuthology.orchestra.run.smithi129.stdout: "is_stretch_pool": false, 2024-09-17T21:31:32.749 INFO:teuthology.orchestra.run.smithi129.stdout: "object_hash": 2, 2024-09-17T21:31:32.750 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:31:32.750 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": 32, 2024-09-17T21:31:32.750 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num": 32, 2024-09-17T21:31:32.750 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_placement_num_target": 32, 2024-09-17T21:31:32.750 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_target": 32, 2024-09-17T21:31:32.750 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num_pending": 32, 2024-09-17T21:31:32.750 INFO:teuthology.orchestra.run.smithi129.stdout: "last_pg_merge_meta": { 2024-09-17T21:31:32.750 INFO:teuthology.orchestra.run.smithi129.stdout: "source_pgid": "0.0", 2024-09-17T21:31:32.750 INFO:teuthology.orchestra.run.smithi129.stdout: "ready_epoch": 0, 2024-09-17T21:31:32.750 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_started": 0, 2024-09-17T21:31:32.750 INFO:teuthology.orchestra.run.smithi129.stdout: "last_epoch_clean": 0, 2024-09-17T21:31:32.750 INFO:teuthology.orchestra.run.smithi129.stdout: "source_version": "0'0", 2024-09-17T21:31:32.750 INFO:teuthology.orchestra.run.smithi129.stdout: "target_version": "0'0" 2024-09-17T21:31:32.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.751 INFO:teuthology.orchestra.run.smithi129.stdout: "last_change": "123", 2024-09-17T21:31:32.751 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend": "0", 2024-09-17T21:31:32.751 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:31:32.751 INFO:teuthology.orchestra.run.smithi129.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:31:32.751 INFO:teuthology.orchestra.run.smithi129.stdout: "auid": 0, 2024-09-17T21:31:32.751 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:31:32.751 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_seq": 47, 2024-09-17T21:31:32.751 INFO:teuthology.orchestra.run.smithi129.stdout: "snap_epoch": 123, 2024-09-17T21:31:32.751 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_snaps": [], 2024-09-17T21:31:32.751 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps": "[]", 2024-09-17T21:31:32.751 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_bytes": 0, 2024-09-17T21:31:32.751 INFO:teuthology.orchestra.run.smithi129.stdout: "quota_max_objects": 0, 2024-09-17T21:31:32.751 INFO:teuthology.orchestra.run.smithi129.stdout: "tiers": [], 2024-09-17T21:31:32.751 INFO:teuthology.orchestra.run.smithi129.stdout: "tier_of": -1, 2024-09-17T21:31:32.752 INFO:teuthology.orchestra.run.smithi129.stdout: "read_tier": -1, 2024-09-17T21:31:32.752 INFO:teuthology.orchestra.run.smithi129.stdout: "write_tier": -1, 2024-09-17T21:31:32.752 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_mode": "none", 2024-09-17T21:31:32.752 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_bytes": 0, 2024-09-17T21:31:32.752 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_objects": 0, 2024-09-17T21:31:32.752 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:31:32.752 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:31:32.752 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:31:32.752 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_flush_age": 0, 2024-09-17T21:31:32.752 INFO:teuthology.orchestra.run.smithi129.stdout: "cache_min_evict_age": 0, 2024-09-17T21:31:32.753 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profile": "", 2024-09-17T21:31:32.753 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_params": { 2024-09-17T21:31:32.753 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "none" 2024-09-17T21:31:32.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.753 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_period": 0, 2024-09-17T21:31:32.753 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_count": 0, 2024-09-17T21:31:32.753 INFO:teuthology.orchestra.run.smithi129.stdout: "use_gmt_hitset": true, 2024-09-17T21:31:32.753 INFO:teuthology.orchestra.run.smithi129.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:31:32.753 INFO:teuthology.orchestra.run.smithi129.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:31:32.753 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:31:32.753 INFO:teuthology.orchestra.run.smithi129.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:31:32.753 INFO:teuthology.orchestra.run.smithi129.stdout: "grade_table": [], 2024-09-17T21:31:32.753 INFO:teuthology.orchestra.run.smithi129.stdout: "stripe_width": 0, 2024-09-17T21:31:32.753 INFO:teuthology.orchestra.run.smithi129.stdout: "expected_num_objects": 0, 2024-09-17T21:31:32.754 INFO:teuthology.orchestra.run.smithi129.stdout: "fast_read": false, 2024-09-17T21:31:32.754 INFO:teuthology.orchestra.run.smithi129.stdout: "options": {}, 2024-09-17T21:31:32.754 INFO:teuthology.orchestra.run.smithi129.stdout: "application_metadata": { 2024-09-17T21:31:32.754 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd": {} 2024-09-17T21:31:32.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.754 INFO:teuthology.orchestra.run.smithi129.stdout: "read_balance": { 2024-09-17T21:31:32.754 INFO:teuthology.orchestra.run.smithi129.stdout: "score_type": "Fair distribution", 2024-09-17T21:31:32.754 INFO:teuthology.orchestra.run.smithi129.stdout: "score_acting": 1.5, 2024-09-17T21:31:32.754 INFO:teuthology.orchestra.run.smithi129.stdout: "score_stable": 1.5, 2024-09-17T21:31:32.754 INFO:teuthology.orchestra.run.smithi129.stdout: "optimal_score": 1, 2024-09-17T21:31:32.754 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_acting": 1.5, 2024-09-17T21:31:32.754 INFO:teuthology.orchestra.run.smithi129.stdout: "raw_score_stable": 1.5, 2024-09-17T21:31:32.754 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity_weighted": 0.90080642700195312, 2024-09-17T21:31:32.754 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-17T21:31:32.755 INFO:teuthology.orchestra.run.smithi129.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-17T21:31:32.755 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.755 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.755 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:31:32.755 INFO:teuthology.orchestra.run.smithi129.stdout: "osds": [ 2024-09-17T21:31:32.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.755 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 0, 2024-09-17T21:31:32.755 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "ad067b51-9402-4f69-a0f0-08fbf068dcfc", 2024-09-17T21:31:32.755 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:31:32.755 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:31:32.755 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:31:32.755 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:31:32.755 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:31:32.755 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:31:32.756 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 13, 2024-09-17T21:31:32.756 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 105, 2024-09-17T21:31:32.756 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:31:32.756 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:31:32.756 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:31:32.756 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.756 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.756 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6800", 2024-09-17T21:31:32.756 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:31:32.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.756 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.757 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6801", 2024-09-17T21:31:32.757 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:31:32.757 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.757 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.757 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:31:32.757 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.757 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.757 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6802", 2024-09-17T21:31:32.757 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:31:32.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.757 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.758 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6803", 2024-09-17T21:31:32.758 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:31:32.758 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.758 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.758 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:31:32.758 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.758 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.758 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6806", 2024-09-17T21:31:32.758 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:31:32.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.759 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.759 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6807", 2024-09-17T21:31:32.759 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:31:32.759 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.759 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.759 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:31:32.759 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.759 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.759 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6804", 2024-09-17T21:31:32.759 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:31:32.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.760 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.760 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6805", 2024-09-17T21:31:32.760 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3635367421 2024-09-17T21:31:32.760 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.760 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.760 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6801/3635367421", 2024-09-17T21:31:32.760 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6803/3635367421", 2024-09-17T21:31:32.760 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6807/3635367421", 2024-09-17T21:31:32.760 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6805/3635367421", 2024-09-17T21:31:32.760 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:31:32.761 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:31:32.761 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:31:32.761 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.761 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 1, 2024-09-17T21:31:32.761 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "30385fc0-ef5f-45f5-a3ec-37e890d4826e", 2024-09-17T21:31:32.761 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:31:32.761 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:31:32.761 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:31:32.761 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:31:32.761 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 15, 2024-09-17T21:31:32.761 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 47, 2024-09-17T21:31:32.762 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 63, 2024-09-17T21:31:32.762 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 104, 2024-09-17T21:31:32.762 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 48, 2024-09-17T21:31:32.762 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:31:32.762 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:31:32.762 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.762 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.762 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6818", 2024-09-17T21:31:32.762 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:31:32.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.762 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.762 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6819", 2024-09-17T21:31:32.763 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:31:32.763 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.763 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.763 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:31:32.763 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.763 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.763 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6820", 2024-09-17T21:31:32.763 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:31:32.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.763 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.763 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6821", 2024-09-17T21:31:32.764 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:31:32.764 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.764 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.764 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:31:32.764 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.764 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.764 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6824", 2024-09-17T21:31:32.764 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:31:32.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.764 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.764 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6825", 2024-09-17T21:31:32.765 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:31:32.765 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.765 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.765 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:31:32.765 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.765 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.765 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6822", 2024-09-17T21:31:32.765 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:31:32.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.765 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.765 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6823", 2024-09-17T21:31:32.766 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 842542464 2024-09-17T21:31:32.766 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.766 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.766 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6819/842542464", 2024-09-17T21:31:32.766 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6821/842542464", 2024-09-17T21:31:32.766 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6825/842542464", 2024-09-17T21:31:32.766 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6823/842542464", 2024-09-17T21:31:32.766 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:31:32.766 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:31:32.766 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:31:32.766 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.767 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 2, 2024-09-17T21:31:32.767 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "76be9b2b-b63f-402a-9ab5-2564bd671ac3", 2024-09-17T21:31:32.767 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:31:32.767 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:31:32.767 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:31:32.767 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 0.21673583984375, 2024-09-17T21:31:32.767 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:31:32.767 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:31:32.767 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:31:32.767 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 103, 2024-09-17T21:31:32.767 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:31:32.767 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:31:32.767 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:31:32.768 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.768 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.768 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6808", 2024-09-17T21:31:32.768 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:31:32.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.768 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.768 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6809", 2024-09-17T21:31:32.768 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:31:32.768 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.768 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.769 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:31:32.769 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.769 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.769 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6810", 2024-09-17T21:31:32.769 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:31:32.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.769 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.769 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6811", 2024-09-17T21:31:32.769 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:31:32.769 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.770 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.770 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:31:32.770 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.770 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.770 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6814", 2024-09-17T21:31:32.770 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:31:32.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.770 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.770 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6815", 2024-09-17T21:31:32.770 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:31:32.770 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.771 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.771 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:31:32.771 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.771 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.771 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6812", 2024-09-17T21:31:32.771 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:31:32.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.771 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.771 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6813", 2024-09-17T21:31:32.771 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1790706405 2024-09-17T21:31:32.772 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.772 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.772 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6809/1790706405", 2024-09-17T21:31:32.772 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6811/1790706405", 2024-09-17T21:31:32.772 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6815/1790706405", 2024-09-17T21:31:32.772 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6813/1790706405", 2024-09-17T21:31:32.772 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:31:32.772 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:31:32.772 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:31:32.772 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.772 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 3, 2024-09-17T21:31:32.773 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "d6c34e3b-1ac6-415c-9c56-38ae77a5c9b0", 2024-09-17T21:31:32.773 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:31:32.773 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:31:32.773 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:31:32.773 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 0.989715576171875, 2024-09-17T21:31:32.773 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:31:32.773 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:31:32.773 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:31:32.773 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 118, 2024-09-17T21:31:32.773 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:31:32.773 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:31:32.773 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:31:32.773 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.774 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.774 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6826", 2024-09-17T21:31:32.774 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:31:32.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.774 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.774 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6827", 2024-09-17T21:31:32.774 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:31:32.774 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.774 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.774 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:31:32.774 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.775 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.775 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6828", 2024-09-17T21:31:32.775 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:31:32.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.775 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.775 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6829", 2024-09-17T21:31:32.775 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:31:32.775 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.775 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.775 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:31:32.776 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.776 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.776 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6832", 2024-09-17T21:31:32.776 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:31:32.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.776 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.776 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6833", 2024-09-17T21:31:32.776 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:31:32.776 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.776 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.777 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:31:32.777 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.777 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.777 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6830", 2024-09-17T21:31:32.777 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:31:32.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.777 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.777 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6831", 2024-09-17T21:31:32.777 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 393361453 2024-09-17T21:31:32.777 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.777 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.778 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.129:6827/393361453", 2024-09-17T21:31:32.778 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.129:6829/393361453", 2024-09-17T21:31:32.778 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.129:6833/393361453", 2024-09-17T21:31:32.778 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.129:6831/393361453", 2024-09-17T21:31:32.778 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:31:32.778 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:31:32.778 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:31:32.778 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.778 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 4, 2024-09-17T21:31:32.778 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "7d722e8c-c497-4f0c-9f61-d9826f8c93d8", 2024-09-17T21:31:32.778 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:31:32.779 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:31:32.779 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:31:32.779 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:31:32.779 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:31:32.779 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:31:32.779 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:31:32.779 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 103, 2024-09-17T21:31:32.779 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:31:32.779 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:31:32.779 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:31:32.779 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.779 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.779 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6808", 2024-09-17T21:31:32.780 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:31:32.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.780 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.780 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6809", 2024-09-17T21:31:32.780 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:31:32.780 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.780 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.780 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:31:32.780 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.780 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.781 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6810", 2024-09-17T21:31:32.781 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:31:32.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.781 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.781 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6811", 2024-09-17T21:31:32.781 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:31:32.781 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.781 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.781 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:31:32.781 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.782 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.782 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6814", 2024-09-17T21:31:32.782 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:31:32.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.782 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.782 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6815", 2024-09-17T21:31:32.782 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:31:32.782 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.782 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.782 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:31:32.782 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.783 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.783 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6812", 2024-09-17T21:31:32.783 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:31:32.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.783 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.783 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6813", 2024-09-17T21:31:32.783 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2337546183 2024-09-17T21:31:32.783 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.783 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.783 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6809/2337546183", 2024-09-17T21:31:32.783 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6811/2337546183", 2024-09-17T21:31:32.784 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6815/2337546183", 2024-09-17T21:31:32.784 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6813/2337546183", 2024-09-17T21:31:32.784 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:31:32.784 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:31:32.784 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:31:32.784 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.784 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 5, 2024-09-17T21:31:32.784 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "954f801b-3ac9-410a-a375-eb1c533744de", 2024-09-17T21:31:32.784 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:31:32.784 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:31:32.784 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:31:32.784 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:31:32.785 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:31:32.785 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:31:32.785 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:31:32.785 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 105, 2024-09-17T21:31:32.785 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:31:32.785 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:31:32.785 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:31:32.785 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.785 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.785 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6816", 2024-09-17T21:31:32.785 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:31:32.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.786 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.786 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6817", 2024-09-17T21:31:32.786 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:31:32.786 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.786 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.786 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:31:32.786 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.786 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.786 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6818", 2024-09-17T21:31:32.786 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:31:32.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.787 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.787 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6819", 2024-09-17T21:31:32.787 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:31:32.787 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.787 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.787 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:31:32.787 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.787 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.787 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6822", 2024-09-17T21:31:32.787 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:31:32.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.788 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.788 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6823", 2024-09-17T21:31:32.788 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:31:32.788 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.788 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.788 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:31:32.788 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.788 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.788 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6820", 2024-09-17T21:31:32.788 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:31:32.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.789 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.789 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6821", 2024-09-17T21:31:32.789 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1510320351 2024-09-17T21:31:32.789 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.789 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.789 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6817/1510320351", 2024-09-17T21:31:32.789 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6819/1510320351", 2024-09-17T21:31:32.789 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6823/1510320351", 2024-09-17T21:31:32.789 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6821/1510320351", 2024-09-17T21:31:32.789 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:31:32.790 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:31:32.790 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:31:32.790 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.790 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 6, 2024-09-17T21:31:32.790 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "21837515-65a2-4f21-96d6-67cfc4589358", 2024-09-17T21:31:32.790 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:31:32.790 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:31:32.790 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:31:32.790 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:31:32.790 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:31:32.790 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:31:32.791 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:31:32.791 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 48, 2024-09-17T21:31:32.791 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:31:32.791 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:31:32.791 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:31:32.791 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.791 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.791 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6824", 2024-09-17T21:31:32.791 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:31:32.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.791 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.791 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6825", 2024-09-17T21:31:32.792 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:31:32.792 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.792 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.792 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:31:32.792 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.792 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.792 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6826", 2024-09-17T21:31:32.792 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:31:32.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.793 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.793 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6827", 2024-09-17T21:31:32.793 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:31:32.793 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.793 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.793 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:31:32.793 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.793 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.793 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6830", 2024-09-17T21:31:32.793 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:31:32.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.794 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.794 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6831", 2024-09-17T21:31:32.794 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:31:32.794 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.794 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.794 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:31:32.794 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.794 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.794 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6828", 2024-09-17T21:31:32.794 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:31:32.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.795 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.795 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6829", 2024-09-17T21:31:32.795 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 471043930 2024-09-17T21:31:32.795 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.795 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.795 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6825/471043930", 2024-09-17T21:31:32.795 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6827/471043930", 2024-09-17T21:31:32.795 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6831/471043930", 2024-09-17T21:31:32.795 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6829/471043930", 2024-09-17T21:31:32.795 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:31:32.795 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:31:32.795 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:31:32.796 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.796 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 7, 2024-09-17T21:31:32.796 INFO:teuthology.orchestra.run.smithi129.stdout: "uuid": "67af7eea-8803-425d-ae50-30a6ab91a711", 2024-09-17T21:31:32.796 INFO:teuthology.orchestra.run.smithi129.stdout: "up": 1, 2024-09-17T21:31:32.796 INFO:teuthology.orchestra.run.smithi129.stdout: "in": 1, 2024-09-17T21:31:32.796 INFO:teuthology.orchestra.run.smithi129.stdout: "weight": 1, 2024-09-17T21:31:32.796 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_affinity": 1, 2024-09-17T21:31:32.796 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_begin": 0, 2024-09-17T21:31:32.796 INFO:teuthology.orchestra.run.smithi129.stdout: "last_clean_end": 0, 2024-09-17T21:31:32.796 INFO:teuthology.orchestra.run.smithi129.stdout: "up_from": 15, 2024-09-17T21:31:32.796 INFO:teuthology.orchestra.run.smithi129.stdout: "up_thru": 104, 2024-09-17T21:31:32.797 INFO:teuthology.orchestra.run.smithi129.stdout: "down_at": 0, 2024-09-17T21:31:32.797 INFO:teuthology.orchestra.run.smithi129.stdout: "lost_at": 0, 2024-09-17T21:31:32.797 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrs": { 2024-09-17T21:31:32.797 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.797 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.797 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6800", 2024-09-17T21:31:32.797 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:31:32.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.797 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.797 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6801", 2024-09-17T21:31:32.797 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:31:32.797 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.798 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.798 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addrs": { 2024-09-17T21:31:32.798 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.798 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.798 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6802", 2024-09-17T21:31:32.798 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:31:32.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.798 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.798 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6803", 2024-09-17T21:31:32.798 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:31:32.798 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.799 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.799 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addrs": { 2024-09-17T21:31:32.799 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.799 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.799 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6806", 2024-09-17T21:31:32.799 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:31:32.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.799 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.799 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6807", 2024-09-17T21:31:32.799 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:31:32.800 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.800 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.800 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addrs": { 2024-09-17T21:31:32.800 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-17T21:31:32.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.800 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-17T21:31:32.800 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6804", 2024-09-17T21:31:32.800 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:31:32.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.800 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-17T21:31:32.800 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.171:6805", 2024-09-17T21:31:32.801 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1770619177 2024-09-17T21:31:32.801 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.801 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.801 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "172.21.15.171:6801/1770619177", 2024-09-17T21:31:32.801 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "172.21.15.171:6803/1770619177", 2024-09-17T21:31:32.801 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_back_addr": "172.21.15.171:6807/1770619177", 2024-09-17T21:31:32.801 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_front_addr": "172.21.15.171:6805/1770619177", 2024-09-17T21:31:32.801 INFO:teuthology.orchestra.run.smithi129.stdout: "state": [ 2024-09-17T21:31:32.801 INFO:teuthology.orchestra.run.smithi129.stdout: "exists", 2024-09-17T21:31:32.801 INFO:teuthology.orchestra.run.smithi129.stdout: "up" 2024-09-17T21:31:32.801 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.801 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.801 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:31:32.802 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_xinfo": [ 2024-09-17T21:31:32.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.802 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 0, 2024-09-17T21:31:32.802 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:31:32.802 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:31:32.802 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:31:32.802 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:31:32.802 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:31:32.802 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:21.445879+0000", 2024-09-17T21:31:32.802 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:31:32.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.802 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 1, 2024-09-17T21:31:32.803 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:31:32.803 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:31:32.803 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:31:32.803 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:31:32.804 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:31:32.804 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.240967+0000", 2024-09-17T21:31:32.804 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 48 2024-09-17T21:31:32.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.804 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 2, 2024-09-17T21:31:32.804 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:31:32.804 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:31:32.804 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:31:32.804 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:31:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:31:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.039097+0000", 2024-09-17T21:31:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:31:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 3, 2024-09-17T21:31:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:31:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:31:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:31:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:31:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:31:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.372014+0000", 2024-09-17T21:31:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:31:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 4, 2024-09-17T21:31:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:31:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:31:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:31:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:31:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:31:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.049543+0000", 2024-09-17T21:31:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:31:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 5, 2024-09-17T21:31:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:31:32.807 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:31:32.807 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:31:32.807 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:31:32.807 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:31:32.807 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.009384+0000", 2024-09-17T21:31:32.807 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:31:32.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.807 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 6, 2024-09-17T21:31:32.807 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:31:32.807 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:31:32.807 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:31:32.807 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:31:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:31:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.048598+0000", 2024-09-17T21:31:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:31:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: "osd": 7, 2024-09-17T21:31:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: "down_stamp": "0.000000", 2024-09-17T21:31:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_probability": 0, 2024-09-17T21:31:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: "laggy_interval": 0, 2024-09-17T21:31:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: "features": 4540719139924082687, 2024-09-17T21:31:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: "old_weight": 0, 2024-09-17T21:31:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: "last_purged_snaps_scrub": "2024-09-17T21:29:22.046345+0000", 2024-09-17T21:31:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: "dead_epoch": 0 2024-09-17T21:31:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:31:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_upmap": [], 2024-09-17T21:31:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_upmap_items": [], 2024-09-17T21:31:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_upmap_primaries": [], 2024-09-17T21:31:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_temp": [], 2024-09-17T21:31:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "primary_temp": [], 2024-09-17T21:31:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "blocklist": {}, 2024-09-17T21:31:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "range_blocklist": {}, 2024-09-17T21:31:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_profiles": { 2024-09-17T21:31:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "default": { 2024-09-17T21:31:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "crush-failure-domain": "osd", 2024-09-17T21:31:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "k": "2", 2024-09-17T21:31:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: "m": "1", 2024-09-17T21:31:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: "plugin": "jerasure", 2024-09-17T21:31:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: "technique": "reed_sol_van" 2024-09-17T21:31:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: "removed_snaps_queue": [ 2024-09-17T21:31:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 3, 2024-09-17T21:31:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: "snaps": [ 2024-09-17T21:31:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: "begin": 16, 2024-09-17T21:31:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: "length": 7 2024-09-17T21:31:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:31:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: "new_removed_snaps": [ 2024-09-17T21:31:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 3, 2024-09-17T21:31:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: "snaps": [ 2024-09-17T21:31:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: "begin": 16, 2024-09-17T21:31:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: "length": 1 2024-09-17T21:31:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:31:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: "new_purged_snaps": [ 2024-09-17T21:31:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: "pool": 3, 2024-09-17T21:31:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: "snaps": [ 2024-09-17T21:31:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: "begin": 23, 2024-09-17T21:31:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: "length": 5 2024-09-17T21:31:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T21:31:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_node_flags": {}, 2024-09-17T21:31:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: "device_class_flags": {}, 2024-09-17T21:31:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_mode": { 2024-09-17T21:31:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_mode_enabled": false, 2024-09-17T21:31:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_bucket_count": 0, 2024-09-17T21:31:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: "degraded_stretch_mode": 0, 2024-09-17T21:31:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: "recovering_stretch_mode": 0, 2024-09-17T21:31:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: "stretch_mode_bucket": 0 2024-09-17T21:31:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:32.813 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:32.815 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:31:32.815 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:32.812+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:32.816 INFO:tasks.thrashosds.thrasher:j is {'epoch': 123, 'fsid': 'f6d3a80c-8a1f-4716-b25d-e97b28890c0d', 'created': '2024-09-17T21:29:12.606637+0000', 'modified': '2024-09-17T21:31:32.407094+0000', 'last_up_change': '2024-09-17T21:30:27.721456+0000', 'last_in_change': '2024-09-17T21:31:10.410190+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T21:29:26.876763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9008064270019531, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T21:29:31.078466+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9008064270019531, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T21:29:39.892418+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 47, 'snap_epoch': 123, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9008064270019531, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.8999999761581421}}], 'osds': [{'osd': 0, 'uuid': 'ad067b51-9402-4f69-a0f0-08fbf068dcfc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 105, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6800', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6801', 'nonce': 3635367421}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6802', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6803', 'nonce': 3635367421}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6806', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6807', 'nonce': 3635367421}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6804', 'nonce': 3635367421}, {'type': 'v1', 'addr': '172.21.15.129:6805', 'nonce': 3635367421}]}, 'public_addr': '172.21.15.129:6801/3635367421', 'cluster_addr': '172.21.15.129:6803/3635367421', 'heartbeat_back_addr': '172.21.15.129:6807/3635367421', 'heartbeat_front_addr': '172.21.15.129:6805/3635367421', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '30385fc0-ef5f-45f5-a3ec-37e890d4826e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 47, 'up_from': 63, 'up_thru': 104, 'down_at': 48, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6818', 'nonce': 842542464}, {'type': 'v1', 'addr': '172.21.15.129:6819', 'nonce': 842542464}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6820', 'nonce': 842542464}, {'type': 'v1', 'addr': '172.21.15.129:6821', 'nonce': 842542464}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6824', 'nonce': 842542464}, {'type': 'v1', 'addr': '172.21.15.129:6825', 'nonce': 842542464}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6822', 'nonce': 842542464}, {'type': 'v1', 'addr': '172.21.15.129:6823', 'nonce': 842542464}]}, 'public_addr': '172.21.15.129:6819/842542464', 'cluster_addr': '172.21.15.129:6821/842542464', 'heartbeat_back_addr': '172.21.15.129:6825/842542464', 'heartbeat_front_addr': '172.21.15.129:6823/842542464', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '76be9b2b-b63f-402a-9ab5-2564bd671ac3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.21673583984375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6808', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6809', 'nonce': 1790706405}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6810', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6811', 'nonce': 1790706405}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6814', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6815', 'nonce': 1790706405}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6812', 'nonce': 1790706405}, {'type': 'v1', 'addr': '172.21.15.129:6813', 'nonce': 1790706405}]}, 'public_addr': '172.21.15.129:6809/1790706405', 'cluster_addr': '172.21.15.129:6811/1790706405', 'heartbeat_back_addr': '172.21.15.129:6815/1790706405', 'heartbeat_front_addr': '172.21.15.129:6813/1790706405', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'd6c34e3b-1ac6-415c-9c56-38ae77a5c9b0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.989715576171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 118, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6826', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6827', 'nonce': 393361453}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6828', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6829', 'nonce': 393361453}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6832', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6833', 'nonce': 393361453}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6830', 'nonce': 393361453}, {'type': 'v1', 'addr': '172.21.15.129:6831', 'nonce': 393361453}]}, 'public_addr': '172.21.15.129:6827/393361453', 'cluster_addr': '172.21.15.129:6829/393361453', 'heartbeat_back_addr': '172.21.15.129:6833/393361453', 'heartbeat_front_addr': '172.21.15.129:6831/393361453', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '7d722e8c-c497-4f0c-9f61-d9826f8c93d8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6808', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6809', 'nonce': 2337546183}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6810', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6811', 'nonce': 2337546183}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6814', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6815', 'nonce': 2337546183}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6812', 'nonce': 2337546183}, {'type': 'v1', 'addr': '172.21.15.171:6813', 'nonce': 2337546183}]}, 'public_addr': '172.21.15.171:6809/2337546183', 'cluster_addr': '172.21.15.171:6811/2337546183', 'heartbeat_back_addr': '172.21.15.171:6815/2337546183', 'heartbeat_front_addr': '172.21.15.171:6813/2337546183', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '954f801b-3ac9-410a-a375-eb1c533744de', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 105, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6816', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6817', 'nonce': 1510320351}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6818', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6819', 'nonce': 1510320351}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6822', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6823', 'nonce': 1510320351}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6820', 'nonce': 1510320351}, {'type': 'v1', 'addr': '172.21.15.171:6821', 'nonce': 1510320351}]}, 'public_addr': '172.21.15.171:6817/1510320351', 'cluster_addr': '172.21.15.171:6819/1510320351', 'heartbeat_back_addr': '172.21.15.171:6823/1510320351', 'heartbeat_front_addr': '172.21.15.171:6821/1510320351', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '21837515-65a2-4f21-96d6-67cfc4589358', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6824', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6825', 'nonce': 471043930}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6826', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6827', 'nonce': 471043930}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6830', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6831', 'nonce': 471043930}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6828', 'nonce': 471043930}, {'type': 'v1', 'addr': '172.21.15.171:6829', 'nonce': 471043930}]}, 'public_addr': '172.21.15.171:6825/471043930', 'cluster_addr': '172.21.15.171:6827/471043930', 'heartbeat_back_addr': '172.21.15.171:6831/471043930', 'heartbeat_front_addr': '172.21.15.171:6829/471043930', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '67af7eea-8803-425d-ae50-30a6ab91a711', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 104, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6800', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6801', 'nonce': 1770619177}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6802', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6803', 'nonce': 1770619177}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6806', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6807', 'nonce': 1770619177}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6804', 'nonce': 1770619177}, {'type': 'v1', 'addr': '172.21.15.171:6805', 'nonce': 1770619177}]}, 'public_addr': '172.21.15.171:6801/1770619177', 'cluster_addr': '172.21.15.171:6803/1770619177', 'heartbeat_back_addr': '172.21.15.171:6807/1770619177', 'heartbeat_front_addr': '172.21.15.171:6805/1770619177', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:21.445879+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.240967+0000', 'dead_epoch': 48}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.039097+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.372014+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.049543+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.009384+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.048598+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:29:22.046345+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 16, 'length': 7}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 16, 'length': 1}]}], 'new_purged_snaps': [{'pool': 3, 'snaps': [{'begin': 23, 'length': 5}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T21:31:32.816 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:31:32.915 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:32.912+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:32.960 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:32.960 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:32.969 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:31:33.015 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:33.013+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:33.072 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:33.072 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:33.081 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:31:33.116 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:33.114+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:33.139 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:31:33.140 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-17T21:31:33.151 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-17T21:31:32.881620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636575338,"num_objects":1400,"num_object_clones":292,"num_object_copies":2800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1400,"num_whiteouts":872,"num_read":155460,"num_read_kb":890618,"num_write":20993,"num_write_kb":454059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1208,"num_bytes_recovered":117790844,"num_keys_recovered":327,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17769,"ondisk_log_size":17769,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":393372,"kb_used_data":46412,"kb_used_omap":31,"kb_used_meta":346656,"kb_avail":754581348,"statfs":{"total":773094113280,"available":772691300352,"internally_reserved":0,"allocated":47525888,"data_stored":334107715,"data_compressed":4158286,"data_compressed_allocated":23842816,"data_compressed_original":315326332,"omap_allocated":32080,"internal_metadata":354976432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-169470489,"num_objects":-72,"num_object_clones":-5,"num_object_copies":-144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-72,"num_whiteouts":-4,"num_read":1074,"num_read_kb":918,"num_write":216,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000828"},"pg_stats":[{"pgid":"3.1f","version":"118'769","reported_seq":4934,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T21:31:28.412820+0000","last_change":"2024-09-17T21:31:28.413231+0000","last_active":"2024-09-17T21:31:28.412820+0000","last_peered":"2024-09-17T21:31:28.412820+0000","last_clean":"2024-09-17T21:31:28.412820+0000","last_became_active":"2024-09-17T21:31:15.457506+0000","last_became_peered":"2024-09-17T21:31:15.457506+0000","last_unstale":"2024-09-17T21:31:28.412820+0000","last_undegraded":"2024-09-17T21:31:28.412820+0000","last_fullsized":"2024-09-17T21:31:28.412820+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:34:50.761573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011036640000000001,"stat_sum":{"num_bytes":42120264,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":29,"num_read":6543,"num_read_kb":63318,"num_write":1132,"num_write_kb":32899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":27321072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.1e","version":"121'515","reported_seq":3995,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.683430+0000","last_change":"2024-09-17T21:31:30.585078+0000","last_active":"2024-09-17T21:31:30.683430+0000","last_peered":"2024-09-17T21:31:30.683430+0000","last_clean":"2024-09-17T21:31:30.683430+0000","last_became_active":"2024-09-17T21:29:41.472161+0000","last_became_peered":"2024-09-17T21:29:41.472161+0000","last_unstale":"2024-09-17T21:31:30.683430+0000","last_undegraded":"2024-09-17T21:31:30.683430+0000","last_fullsized":"2024-09-17T21:31:30.683430+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:37:42.349032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011372279999999999,"stat_sum":{"num_bytes":15573325,"num_objects":38,"num_object_clones":6,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":24,"num_read":3786,"num_read_kb":31402,"num_write":514,"num_write_kb":11787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.1d","version":"121'728","reported_seq":5218,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.547822+0000","last_change":"2024-09-17T21:31:30.547822+0000","last_active":"2024-09-17T21:31:30.547822+0000","last_peered":"2024-09-17T21:31:30.547822+0000","last_clean":"2024-09-17T21:31:30.547822+0000","last_became_active":"2024-09-17T21:31:14.449672+0000","last_became_peered":"2024-09-17T21:31:14.449672+0000","last_unstale":"2024-09-17T21:31:30.547822+0000","last_undegraded":"2024-09-17T21:31:30.547822+0000","last_fullsized":"2024-09-17T21:31:30.547822+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:01:03.346353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085039800000000004,"stat_sum":{"num_bytes":4841743,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":22,"num_read":6550,"num_read_kb":51684,"num_write":862,"num_write_kb":18867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":15268754,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.1c","version":"121'379","reported_seq":2335,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.660987+0000","last_change":"2024-09-17T21:31:30.547450+0000","last_active":"2024-09-17T21:31:30.660987+0000","last_peered":"2024-09-17T21:31:30.660987+0000","last_clean":"2024-09-17T21:31:30.660987+0000","last_became_active":"2024-09-17T21:31:14.451037+0000","last_became_peered":"2024-09-17T21:31:14.451037+0000","last_unstale":"2024-09-17T21:31:30.660987+0000","last_undegraded":"2024-09-17T21:31:30.660987+0000","last_fullsized":"2024-09-17T21:31:30.660987+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:16.354886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061458200000000002,"stat_sum":{"num_bytes":15973472,"num_objects":45,"num_object_clones":7,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":33,"num_read":2353,"num_read_kb":12588,"num_write":406,"num_write_kb":9114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":604511,"num_keys_recovered":142,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.1b","version":"116'541","reported_seq":8223,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T21:31:28.413216+0000","last_change":"2024-09-17T21:31:28.413453+0000","last_active":"2024-09-17T21:31:28.413216+0000","last_peered":"2024-09-17T21:31:28.413216+0000","last_clean":"2024-09-17T21:31:28.413216+0000","last_became_active":"2024-09-17T21:29:41.476041+0000","last_became_peered":"2024-09-17T21:29:41.476041+0000","last_unstale":"2024-09-17T21:31:28.413216+0000","last_undegraded":"2024-09-17T21:31:28.413216+0000","last_fullsized":"2024-09-17T21:31:28.413216+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:04:17.774093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001183559,"stat_sum":{"num_bytes":34369769,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":21,"num_read":8595,"num_read_kb":53352,"num_write":555,"num_write_kb":24870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.1a","version":"120'596","reported_seq":6292,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.667037+0000","last_change":"2024-09-17T21:31:30.528171+0000","last_active":"2024-09-17T21:31:30.667037+0000","last_peered":"2024-09-17T21:31:30.667037+0000","last_clean":"2024-09-17T21:31:30.667037+0000","last_became_active":"2024-09-17T21:30:28.747031+0000","last_became_peered":"2024-09-17T21:30:28.747031+0000","last_unstale":"2024-09-17T21:31:30.667037+0000","last_undegraded":"2024-09-17T21:31:30.667037+0000","last_fullsized":"2024-09-17T21:31:30.667037+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:08.794508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066349400000000004,"stat_sum":{"num_bytes":8532781,"num_objects":45,"num_object_clones":7,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":28,"num_read":6929,"num_read_kb":21534,"num_write":643,"num_write_kb":6721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":235969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.19","version":"120'392","reported_seq":4603,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T21:31:31.552007+0000","last_change":"2024-09-17T21:31:31.552007+0000","last_active":"2024-09-17T21:31:31.552007+0000","last_peered":"2024-09-17T21:31:31.552007+0000","last_clean":"2024-09-17T21:31:31.552007+0000","last_became_active":"2024-09-17T21:30:28.745634+0000","last_became_peered":"2024-09-17T21:30:28.745634+0000","last_unstale":"2024-09-17T21:31:31.552007+0000","last_undegraded":"2024-09-17T21:31:31.552007+0000","last_fullsized":"2024-09-17T21:31:31.552007+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:41:47.716346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055059900000000003,"stat_sum":{"num_bytes":66467073,"num_objects":53,"num_object_clones":11,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":32,"num_read":4545,"num_read_kb":56991,"num_write":416,"num_write_kb":45225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":36017677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.18","version":"121'449","reported_seq":4255,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.685002+0000","last_change":"2024-09-17T21:31:30.584348+0000","last_active":"2024-09-17T21:31:30.685002+0000","last_peered":"2024-09-17T21:31:30.685002+0000","last_clean":"2024-09-17T21:31:30.685002+0000","last_became_active":"2024-09-17T21:29:41.472468+0000","last_became_peered":"2024-09-17T21:29:41.472468+0000","last_unstale":"2024-09-17T21:31:30.685002+0000","last_undegraded":"2024-09-17T21:31:30.685002+0000","last_fullsized":"2024-09-17T21:31:30.685002+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:28:54.691978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065759599999999998,"stat_sum":{"num_bytes":20887633,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":22,"num_read":4574,"num_read_kb":27498,"num_write":460,"num_write_kb":13091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.17","version":"116'743","reported_seq":2855,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T21:31:28.412876+0000","last_change":"2024-09-17T21:31:28.413583+0000","last_active":"2024-09-17T21:31:28.412876+0000","last_peered":"2024-09-17T21:31:28.412876+0000","last_clean":"2024-09-17T21:31:28.412876+0000","last_became_active":"2024-09-17T21:31:15.456240+0000","last_became_peered":"2024-09-17T21:31:15.456240+0000","last_unstale":"2024-09-17T21:31:28.412876+0000","last_undegraded":"2024-09-17T21:31:28.412876+0000","last_fullsized":"2024-09-17T21:31:28.412876+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:55.250950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014852159999999999,"stat_sum":{"num_bytes":36559163,"num_objects":47,"num_object_clones":9,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":27,"num_read":4507,"num_read_kb":22459,"num_write":912,"num_write_kb":30080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":9394579,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.16","version":"118'1053","reported_seq":2826,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.547132+0000","last_change":"2024-09-17T21:31:30.547345+0000","last_active":"2024-09-17T21:31:30.547132+0000","last_peered":"2024-09-17T21:31:30.547132+0000","last_clean":"2024-09-17T21:31:30.547132+0000","last_became_active":"2024-09-17T21:31:15.456612+0000","last_became_peered":"2024-09-17T21:31:15.456612+0000","last_unstale":"2024-09-17T21:31:30.547132+0000","last_undegraded":"2024-09-17T21:31:30.547132+0000","last_fullsized":"2024-09-17T21:31:30.547132+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:00:35.393613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048292799999999998,"stat_sum":{"num_bytes":22710030,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":28,"num_read":5550,"num_read_kb":20731,"num_write":1637,"num_write_kb":12341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3063082,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.15","version":"120'490","reported_seq":3028,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.605975+0000","last_change":"2024-09-17T21:31:30.606255+0000","last_active":"2024-09-17T21:31:30.605975+0000","last_peered":"2024-09-17T21:31:30.605975+0000","last_clean":"2024-09-17T21:31:30.605975+0000","last_became_active":"2024-09-17T21:29:41.478346+0000","last_became_peered":"2024-09-17T21:29:41.478346+0000","last_unstale":"2024-09-17T21:31:30.605975+0000","last_undegraded":"2024-09-17T21:31:30.605975+0000","last_fullsized":"2024-09-17T21:31:30.605975+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:51:56.844922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064411799999999995,"stat_sum":{"num_bytes":6551552,"num_objects":49,"num_object_clones":10,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":37,"num_read":3628,"num_read_kb":9299,"num_write":424,"num_write_kb":8355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.14","version":"114'253","reported_seq":2704,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.527972+0000","last_change":"2024-09-17T21:31:30.528195+0000","last_active":"2024-09-17T21:31:30.527972+0000","last_peered":"2024-09-17T21:31:30.527972+0000","last_clean":"2024-09-17T21:31:30.527972+0000","last_became_active":"2024-09-17T21:29:41.477611+0000","last_became_peered":"2024-09-17T21:29:41.477611+0000","last_unstale":"2024-09-17T21:31:30.527972+0000","last_undegraded":"2024-09-17T21:31:30.527972+0000","last_fullsized":"2024-09-17T21:31:30.527972+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:37:24.749141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065445,"stat_sum":{"num_bytes":25486539,"num_objects":59,"num_object_clones":19,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":34,"num_read":2151,"num_read_kb":14078,"num_write":346,"num_write_kb":12746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":216,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.540386+0000","last_change":"2024-09-17T21:29:35.002656+0000","last_active":"2024-09-17T21:31:30.540386+0000","last_peered":"2024-09-17T21:31:30.540386+0000","last_clean":"2024-09-17T21:31:30.540386+0000","last_became_active":"2024-09-17T21:29:32.971677+0000","last_became_peered":"2024-09-17T21:29:32.971677+0000","last_unstale":"2024-09-17T21:31:30.540386+0000","last_undegraded":"2024-09-17T21:31:30.540386+0000","last_fullsized":"2024-09-17T21:31:30.540386+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:53.781657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041121300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"119'739","reported_seq":4043,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T21:31:28.498320+0000","last_change":"2024-09-17T21:31:28.412713+0000","last_active":"2024-09-17T21:31:28.498320+0000","last_peered":"2024-09-17T21:31:28.498320+0000","last_clean":"2024-09-17T21:31:28.498320+0000","last_became_active":"2024-09-17T21:30:31.755039+0000","last_became_peered":"2024-09-17T21:30:31.755039+0000","last_unstale":"2024-09-17T21:31:28.498320+0000","last_undegraded":"2024-09-17T21:31:28.498320+0000","last_fullsized":"2024-09-17T21:31:28.498320+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:08:43.128601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067204700000000005,"stat_sum":{"num_bytes":13597315,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":23,"num_read":6524,"num_read_kb":17928,"num_write":902,"num_write_kb":8580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":448871,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.6","version":"0'0","reported_seq":152,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T21:31:31.551557+0000","last_change":"2024-09-17T21:30:28.745759+0000","last_active":"2024-09-17T21:31:31.551557+0000","last_peered":"2024-09-17T21:31:31.551557+0000","last_clean":"2024-09-17T21:31:31.551557+0000","last_became_active":"2024-09-17T21:30:28.744742+0000","last_became_peered":"2024-09-17T21:30:28.744742+0000","last_unstale":"2024-09-17T21:31:31.551557+0000","last_undegraded":"2024-09-17T21:31:31.551557+0000","last_fullsized":"2024-09-17T21:31:31.551557+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:37:02.014566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"120'555","reported_seq":2617,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.584928+0000","last_change":"2024-09-17T21:31:30.585119+0000","last_active":"2024-09-17T21:31:30.584928+0000","last_peered":"2024-09-17T21:31:30.584928+0000","last_clean":"2024-09-17T21:31:30.584928+0000","last_became_active":"2024-09-17T21:29:41.472095+0000","last_became_peered":"2024-09-17T21:29:41.472095+0000","last_unstale":"2024-09-17T21:31:30.584928+0000","last_undegraded":"2024-09-17T21:31:30.584928+0000","last_fullsized":"2024-09-17T21:31:30.584928+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:08:28.875448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074059699999999996,"stat_sum":{"num_bytes":3324600,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":28,"num_read":3707,"num_read_kb":13068,"num_write":684,"num_write_kb":8995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"2.5","version":"0'0","reported_seq":216,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.605806+0000","last_change":"2024-09-17T21:29:35.078854+0000","last_active":"2024-09-17T21:31:30.605806+0000","last_peered":"2024-09-17T21:31:30.605806+0000","last_clean":"2024-09-17T21:31:30.605806+0000","last_became_active":"2024-09-17T21:29:33.071665+0000","last_became_peered":"2024-09-17T21:29:33.071665+0000","last_unstale":"2024-09-17T21:31:30.605806+0000","last_undegraded":"2024-09-17T21:31:30.605806+0000","last_fullsized":"2024-09-17T21:31:30.605806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.415687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047969899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"118'769","reported_seq":4707,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T21:31:31.552053+0000","last_change":"2024-09-17T21:31:31.557058+0000","last_active":"2024-09-17T21:31:31.552053+0000","last_peered":"2024-09-17T21:31:31.552053+0000","last_clean":"2024-09-17T21:31:31.552053+0000","last_became_active":"2024-09-17T21:30:31.752750+0000","last_became_peered":"2024-09-17T21:30:31.752750+0000","last_unstale":"2024-09-17T21:31:31.552053+0000","last_undegraded":"2024-09-17T21:31:31.552053+0000","last_fullsized":"2024-09-17T21:31:31.552053+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:29.230043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052776940000000003,"stat_sum":{"num_bytes":10185981,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":33,"num_read":6239,"num_read_kb":32674,"num_write":1032,"num_write_kb":17419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3231045,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"2.4","version":"0'0","reported_seq":152,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T21:31:31.551652+0000","last_change":"2024-09-17T21:30:28.748319+0000","last_active":"2024-09-17T21:31:31.551652+0000","last_peered":"2024-09-17T21:31:31.551652+0000","last_clean":"2024-09-17T21:31:31.551652+0000","last_became_active":"2024-09-17T21:30:28.747728+0000","last_became_peered":"2024-09-17T21:30:28.747728+0000","last_unstale":"2024-09-17T21:31:31.551652+0000","last_undegraded":"2024-09-17T21:31:31.551652+0000","last_fullsized":"2024-09-17T21:31:31.551652+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:58:18.954005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"120'484","reported_seq":4695,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.547389+0000","last_change":"2024-09-17T21:31:30.547592+0000","last_active":"2024-09-17T21:31:30.547389+0000","last_peered":"2024-09-17T21:31:30.547389+0000","last_clean":"2024-09-17T21:31:30.547389+0000","last_became_active":"2024-09-17T21:31:14.455957+0000","last_became_peered":"2024-09-17T21:31:14.455957+0000","last_unstale":"2024-09-17T21:31:30.547389+0000","last_undegraded":"2024-09-17T21:31:30.547389+0000","last_fullsized":"2024-09-17T21:31:30.547389+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:50:16.621639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070133000000000001,"stat_sum":{"num_bytes":12781273,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":27,"num_read":4982,"num_read_kb":31631,"num_write":552,"num_write_kb":9789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":1545182,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.3","version":"121'727","reported_seq":4606,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.649342+0000","last_change":"2024-09-17T21:31:30.527896+0000","last_active":"2024-09-17T21:31:30.649342+0000","last_peered":"2024-09-17T21:31:30.649342+0000","last_clean":"2024-09-17T21:31:30.649342+0000","last_became_active":"2024-09-17T21:29:41.475273+0000","last_became_peered":"2024-09-17T21:29:41.475273+0000","last_unstale":"2024-09-17T21:31:30.649342+0000","last_undegraded":"2024-09-17T21:31:30.649342+0000","last_fullsized":"2024-09-17T21:31:30.649342+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:16:04.837427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393867,"stat_sum":{"num_bytes":25366835,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":22,"num_read":5975,"num_read_kb":17795,"num_write":886,"num_write_kb":6598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"2.2","version":"23'2","reported_seq":53,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.546711+0000","last_change":"2024-09-17T21:31:11.424633+0000","last_active":"2024-09-17T21:31:30.546711+0000","last_peered":"2024-09-17T21:31:30.546711+0000","last_clean":"2024-09-17T21:31:30.546711+0000","last_became_active":"2024-09-17T21:31:11.421737+0000","last_became_peered":"2024-09-17T21:31:11.421737+0000","last_unstale":"2024-09-17T21:31:30.546711+0000","last_undegraded":"2024-09-17T21:31:30.546711+0000","last_fullsized":"2024-09-17T21:31:30.546711+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:07:58.025026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"120'399","reported_seq":2831,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T21:31:31.551745+0000","last_change":"2024-09-17T21:31:31.557048+0000","last_active":"2024-09-17T21:31:31.551745+0000","last_peered":"2024-09-17T21:31:31.551745+0000","last_clean":"2024-09-17T21:31:31.551745+0000","last_became_active":"2024-09-17T21:30:28.746690+0000","last_became_peered":"2024-09-17T21:30:28.746690+0000","last_unstale":"2024-09-17T21:31:31.551745+0000","last_undegraded":"2024-09-17T21:31:31.551745+0000","last_fullsized":"2024-09-17T21:31:31.551745+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:34:18.537148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055675209999999998,"stat_sum":{"num_bytes":6526166,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":21,"num_read":3361,"num_read_kb":13328,"num_write":385,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1513,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.583737+0000","last_change":"2024-09-17T21:31:28.419063+0000","last_active":"2024-09-17T21:31:30.583737+0000","last_peered":"2024-09-17T21:31:30.583737+0000","last_clean":"2024-09-17T21:31:30.583737+0000","last_became_active":"2024-09-17T21:31:28.418800+0000","last_became_peered":"2024-09-17T21:31:28.418800+0000","last_unstale":"2024-09-17T21:31:30.583737+0000","last_undegraded":"2024-09-17T21:31:30.583737+0000","last_fullsized":"2024-09-17T21:31:30.583737+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:26:55.627280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"117'400","reported_seq":3773,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T21:31:28.411985+0000","last_change":"2024-09-17T21:31:28.412342+0000","last_active":"2024-09-17T21:31:28.411985+0000","last_peered":"2024-09-17T21:31:28.411985+0000","last_clean":"2024-09-17T21:31:28.411985+0000","last_became_active":"2024-09-17T21:29:41.473964+0000","last_became_peered":"2024-09-17T21:29:41.473964+0000","last_unstale":"2024-09-17T21:31:28.411985+0000","last_undegraded":"2024-09-17T21:31:28.411985+0000","last_fullsized":"2024-09-17T21:31:28.411985+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:50:05.391091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058759199999999995,"stat_sum":{"num_bytes":44960803,"num_objects":51,"num_object_clones":11,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":31,"num_read":3796,"num_read_kb":64519,"num_write":569,"num_write_kb":39811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.0","version":"0'0","reported_seq":225,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.605580+0000","last_change":"2024-09-17T21:30:28.746557+0000","last_active":"2024-09-17T21:31:30.605580+0000","last_peered":"2024-09-17T21:31:30.605580+0000","last_clean":"2024-09-17T21:31:30.605580+0000","last_became_active":"2024-09-17T21:30:28.746261+0000","last_became_peered":"2024-09-17T21:30:28.746261+0000","last_unstale":"2024-09-17T21:31:30.605580+0000","last_undegraded":"2024-09-17T21:31:30.605580+0000","last_fullsized":"2024-09-17T21:31:30.605580+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:42.591689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044115199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"114'313","reported_seq":4034,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.627061+0000","last_change":"2024-09-17T21:31:30.584686+0000","last_active":"2024-09-17T21:31:30.627061+0000","last_peered":"2024-09-17T21:31:30.627061+0000","last_clean":"2024-09-17T21:31:30.627061+0000","last_became_active":"2024-09-17T21:31:14.459936+0000","last_became_peered":"2024-09-17T21:31:14.459936+0000","last_unstale":"2024-09-17T21:31:30.627061+0000","last_undegraded":"2024-09-17T21:31:30.627061+0000","last_fullsized":"2024-09-17T21:31:30.627061+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:35:51.758625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088687200000000003,"stat_sum":{"num_bytes":6402535,"num_objects":41,"num_object_clones":7,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":28,"num_read":3921,"num_read_kb":72695,"num_write":296,"num_write_kb":18067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":13048770,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"2.3","version":"21'1","reported_seq":53,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.546716+0000","last_change":"2024-09-17T21:31:11.427558+0000","last_active":"2024-09-17T21:31:30.546716+0000","last_peered":"2024-09-17T21:31:30.546716+0000","last_clean":"2024-09-17T21:31:30.546716+0000","last_became_active":"2024-09-17T21:31:11.422076+0000","last_became_peered":"2024-09-17T21:31:11.422076+0000","last_unstale":"2024-09-17T21:31:30.546716+0000","last_undegraded":"2024-09-17T21:31:30.546716+0000","last_fullsized":"2024-09-17T21:31:30.546716+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:37:14.137712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":308,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.605496+0000","last_change":"2024-09-17T21:29:28.554672+0000","last_active":"2024-09-17T21:31:30.605496+0000","last_peered":"2024-09-17T21:31:30.605496+0000","last_clean":"2024-09-17T21:31:30.605496+0000","last_became_active":"2024-09-17T21:29:28.553733+0000","last_became_peered":"2024-09-17T21:29:28.553733+0000","last_unstale":"2024-09-17T21:31:30.605496+0000","last_undegraded":"2024-09-17T21:31:30.605496+0000","last_fullsized":"2024-09-17T21:31:30.605496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_clean_scrub_stamp":"2024-09-17T21:29:26.925225+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:36:42.585477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"121'673","reported_seq":4392,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.584293+0000","last_change":"2024-09-17T21:31:30.584293+0000","last_active":"2024-09-17T21:31:30.584293+0000","last_peered":"2024-09-17T21:31:30.584293+0000","last_clean":"2024-09-17T21:31:30.584293+0000","last_became_active":"2024-09-17T21:30:31.753839+0000","last_became_peered":"2024-09-17T21:30:31.753839+0000","last_unstale":"2024-09-17T21:31:30.584293+0000","last_undegraded":"2024-09-17T21:31:30.584293+0000","last_fullsized":"2024-09-17T21:31:30.584293+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:34:40.322271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058563300000000001,"stat_sum":{"num_bytes":18601446,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":19,"num_read":5086,"num_read_kb":35050,"num_write":861,"num_write_kb":14883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2410412,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.9","version":"121'394","reported_seq":3424,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.527899+0000","last_change":"2024-09-17T21:31:30.527899+0000","last_active":"2024-09-17T21:31:30.527899+0000","last_peered":"2024-09-17T21:31:30.527899+0000","last_clean":"2024-09-17T21:31:30.527899+0000","last_became_active":"2024-09-17T21:29:41.477965+0000","last_became_peered":"2024-09-17T21:29:41.477965+0000","last_unstale":"2024-09-17T21:31:30.527899+0000","last_undegraded":"2024-09-17T21:31:30.527899+0000","last_fullsized":"2024-09-17T21:31:30.527899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:55:50.754201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042662400000000001,"stat_sum":{"num_bytes":3851650,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":16,"num_read":3777,"num_read_kb":28830,"num_write":421,"num_write_kb":12788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.a","version":"120'562","reported_seq":3903,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.540565+0000","last_change":"2024-09-17T21:31:30.540727+0000","last_active":"2024-09-17T21:31:30.540565+0000","last_peered":"2024-09-17T21:31:30.540565+0000","last_clean":"2024-09-17T21:31:30.540565+0000","last_became_active":"2024-09-17T21:29:41.473420+0000","last_became_peered":"2024-09-17T21:29:41.473420+0000","last_unstale":"2024-09-17T21:31:30.540565+0000","last_undegraded":"2024-09-17T21:31:30.540565+0000","last_fullsized":"2024-09-17T21:31:30.540565+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:43:55.454418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029901400000000001,"stat_sum":{"num_bytes":41958903,"num_objects":46,"num_object_clones":8,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":30,"num_read":4566,"num_read_kb":15555,"num_write":593,"num_write_kb":6172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.b","version":"121'479","reported_seq":6931,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.608428+0000","last_change":"2024-09-17T21:31:30.584668+0000","last_active":"2024-09-17T21:31:30.608428+0000","last_peered":"2024-09-17T21:31:30.608428+0000","last_clean":"2024-09-17T21:31:30.608428+0000","last_became_active":"2024-09-17T21:29:41.472354+0000","last_became_peered":"2024-09-17T21:29:41.472354+0000","last_unstale":"2024-09-17T21:31:30.608428+0000","last_undegraded":"2024-09-17T21:31:30.608428+0000","last_fullsized":"2024-09-17T21:31:30.608428+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:04:15.917030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086885300000000005,"stat_sum":{"num_bytes":10121961,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":27,"num_read":7481,"num_read_kb":30147,"num_write":427,"num_write_kb":3983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.c","version":"120'595","reported_seq":4511,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.643043+0000","last_change":"2024-09-17T21:31:30.547687+0000","last_active":"2024-09-17T21:31:30.643043+0000","last_peered":"2024-09-17T21:31:30.643043+0000","last_clean":"2024-09-17T21:31:30.643043+0000","last_became_active":"2024-09-17T21:31:15.457822+0000","last_became_peered":"2024-09-17T21:31:15.457822+0000","last_unstale":"2024-09-17T21:31:30.643043+0000","last_undegraded":"2024-09-17T21:31:30.643043+0000","last_fullsized":"2024-09-17T21:31:30.643043+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:59:48.565511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071968300000000004,"stat_sum":{"num_bytes":40055691,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":25,"num_read":6874,"num_read_kb":17224,"num_write":537,"num_write_kb":5160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":2355433,"num_keys_recovered":58,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.d","version":"118'596","reported_seq":4353,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.606336+0000","last_change":"2024-09-17T21:31:30.606916+0000","last_active":"2024-09-17T21:31:30.606336+0000","last_peered":"2024-09-17T21:31:30.606336+0000","last_clean":"2024-09-17T21:31:30.606336+0000","last_became_active":"2024-09-17T21:31:14.457648+0000","last_became_peered":"2024-09-17T21:31:14.457648+0000","last_unstale":"2024-09-17T21:31:30.606336+0000","last_undegraded":"2024-09-17T21:31:30.606336+0000","last_fullsized":"2024-09-17T21:31:30.606336+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:11:00.841458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001250625,"stat_sum":{"num_bytes":8734461,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":28,"num_read":5354,"num_read_kb":12719,"num_write":703,"num_write_kb":4578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2843955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.e","version":"119'555","reported_seq":4222,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.607225+0000","last_change":"2024-09-17T21:31:30.607225+0000","last_active":"2024-09-17T21:31:30.607225+0000","last_peered":"2024-09-17T21:31:30.607225+0000","last_clean":"2024-09-17T21:31:30.607225+0000","last_became_active":"2024-09-17T21:29:41.473736+0000","last_became_peered":"2024-09-17T21:29:41.473736+0000","last_unstale":"2024-09-17T21:31:30.607225+0000","last_undegraded":"2024-09-17T21:31:30.607225+0000","last_fullsized":"2024-09-17T21:31:30.607225+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:11:42.486292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579980000000001,"stat_sum":{"num_bytes":14966550,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":27,"num_read":5040,"num_read_kb":17594,"num_write":685,"num_write_kb":5927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.f","version":"117'831","reported_seq":4269,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.637030+0000","last_change":"2024-09-17T21:31:30.606887+0000","last_active":"2024-09-17T21:31:30.637030+0000","last_peered":"2024-09-17T21:31:30.637030+0000","last_clean":"2024-09-17T21:31:30.637030+0000","last_became_active":"2024-09-17T21:29:41.473394+0000","last_became_peered":"2024-09-17T21:29:41.473394+0000","last_unstale":"2024-09-17T21:31:30.637030+0000","last_undegraded":"2024-09-17T21:31:30.637030+0000","last_fullsized":"2024-09-17T21:31:30.637030+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:06:01.584691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011078629999999999,"stat_sum":{"num_bytes":28438550,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":36,"num_read":5831,"num_read_kb":8272,"num_write":1158,"num_write_kb":21754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.10","version":"121'607","reported_seq":2427,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.614537+0000","last_change":"2024-09-17T21:31:30.541613+0000","last_active":"2024-09-17T21:31:30.614537+0000","last_peered":"2024-09-17T21:31:30.614537+0000","last_clean":"2024-09-17T21:31:30.614537+0000","last_became_active":"2024-09-17T21:29:41.471342+0000","last_became_peered":"2024-09-17T21:29:41.471342+0000","last_unstale":"2024-09-17T21:31:30.614537+0000","last_undegraded":"2024-09-17T21:31:30.614537+0000","last_fullsized":"2024-09-17T21:31:30.614537+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:56:49.615664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117961,"stat_sum":{"num_bytes":5168422,"num_objects":36,"num_object_clones":5,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":25,"num_read":3239,"num_read_kb":14667,"num_write":837,"num_write_kb":8817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.11","version":"120'371","reported_seq":4570,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.667462+0000","last_change":"2024-09-17T21:31:30.606577+0000","last_active":"2024-09-17T21:31:30.667462+0000","last_peered":"2024-09-17T21:31:30.667462+0000","last_clean":"2024-09-17T21:31:30.667462+0000","last_became_active":"2024-09-17T21:29:41.473252+0000","last_became_peered":"2024-09-17T21:29:41.473252+0000","last_unstale":"2024-09-17T21:31:30.667462+0000","last_undegraded":"2024-09-17T21:31:30.667462+0000","last_fullsized":"2024-09-17T21:31:30.667462+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:48:51.477428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010211319999999999,"stat_sum":{"num_bytes":19528578,"num_objects":46,"num_object_clones":9,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":29,"num_read":4586,"num_read_kb":34435,"num_write":358,"num_write_kb":15074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.12","version":"118'352","reported_seq":2102,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T21:31:28.412289+0000","last_change":"2024-09-17T21:31:28.412547+0000","last_active":"2024-09-17T21:31:28.412289+0000","last_peered":"2024-09-17T21:31:28.412289+0000","last_clean":"2024-09-17T21:31:28.412289+0000","last_became_active":"2024-09-17T21:29:41.477019+0000","last_became_peered":"2024-09-17T21:29:41.477019+0000","last_unstale":"2024-09-17T21:31:28.412289+0000","last_undegraded":"2024-09-17T21:31:28.412289+0000","last_fullsized":"2024-09-17T21:31:28.412289+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:42:06.092104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072698499999999998,"stat_sum":{"num_bytes":10197439,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":25,"num_read":1869,"num_read_kb":5226,"num_write":344,"num_write_kb":2424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.13","version":"121'385","reported_seq":3793,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T21:31:30.606191+0000","last_change":"2024-09-17T21:31:30.606191+0000","last_active":"2024-09-17T21:31:30.606191+0000","last_peered":"2024-09-17T21:31:30.606191+0000","last_clean":"2024-09-17T21:31:30.606191+0000","last_became_active":"2024-09-17T21:29:41.473182+0000","last_became_peered":"2024-09-17T21:29:41.473182+0000","last_unstale":"2024-09-17T21:31:30.606191+0000","last_undegraded":"2024-09-17T21:31:30.606191+0000","last_fullsized":"2024-09-17T21:31:30.606191+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:02:28.544778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063295500000000004,"stat_sum":{"num_bytes":11142448,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":35,"num_read":3475,"num_read_kb":22272,"num_write":353,"num_write_kb":9441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":635984951,"num_objects":1396,"num_object_clones":292,"num_object_copies":2792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1396,"num_whiteouts":872,"num_read":155394,"num_read_kb":890563,"num_write":20890,"num_write_kb":452429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1205,"num_bytes_recovered":117790825,"num_keys_recovered":327,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35663872,"data_stored":325742755,"data_compressed":4047276,"data_compressed_allocated":23490560,"data_compressed_original":312414076,"omap_allocated":10476,"internal_metadata":0},"log_size":17694,"ondisk_log_size":17694,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55080,"kb_used_data":5832,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94316760,"statfs":{"total":96636764160,"available":96580362240,"internally_reserved":0,"allocated":5971968,"data_stored":21478869,"data_compressed":491811,"data_compressed_allocated":2437120,"data_compressed_original":18554168,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509468,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45856,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325984,"statfs":{"total":96636764160,"available":96589807616,"internally_reserved":0,"allocated":2236416,"data_stored":4201053,"data_compressed":116000,"data_compressed_allocated":368640,"data_compressed_original":3073567,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45616,"kb_used_data":6108,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326224,"statfs":{"total":96636764160,"available":96590053376,"internally_reserved":0,"allocated":6254592,"data_stored":51548084,"data_compressed":684029,"data_compressed_allocated":3231744,"data_compressed_original":48884839,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61580,"kb_used_data":8184,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310260,"statfs":{"total":96636764160,"available":96573706240,"internally_reserved":0,"allocated":8380416,"data_stored":57628189,"data_compressed":736645,"data_compressed_allocated":4403200,"data_compressed_original":54213944,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509469,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38088,"kb_used_data":6884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333752,"statfs":{"total":96636764160,"available":96597762048,"internally_reserved":0,"allocated":7049216,"data_stored":48721730,"data_compressed":659502,"data_compressed_allocated":3854336,"data_compressed_original":46264320,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41744,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330096,"statfs":{"total":96636764160,"available":96594018304,"internally_reserved":0,"allocated":2289664,"data_stored":3777914,"data_compressed":61754,"data_compressed_allocated":331776,"data_compressed_original":2564096,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":63,"seq":270582939662,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43076,"kb_used_data":6120,"kb_used_omap":20,"kb_used_meta":36907,"kb_avail":94328764,"statfs":{"total":96636764160,"available":96592654336,"internally_reserved":0,"allocated":6266880,"data_stored":52955602,"data_compressed":337775,"data_compressed_allocated":3776512,"data_compressed_original":51322880,"omap_allocated":20925,"internal_metadata":37793347},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62332,"kb_used_data":8864,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94309508,"statfs":{"total":96636764160,"available":96572936192,"internally_reserved":0,"allocated":9076736,"data_stored":93796274,"data_compressed":1070770,"data_compressed_allocated":5439488,"data_compressed_original":90448518,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7606272,"data_stored":92537826,"data_compressed":1065801,"data_compressed_allocated":5402624,"data_compressed_original":89875078,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4685824,"data_stored":51386270,"data_compressed":295171,"data_compressed_allocated":3645440,"data_compressed_original":50438144,"omap_allocated":10476,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":864256,"data_stored":3103607,"data_compressed":61754,"data_compressed_allocated":331776,"data_compressed_original":2564096,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5550080,"data_stored":47824701,"data_compressed":637565,"data_compressed_allocated":3809280,"data_compressed_original":46055424,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6926336,"data_stored":56713936,"data_compressed":732000,"data_compressed_allocated":4378624,"data_compressed_original":53976376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4825088,"data_stored":50873758,"data_compressed":684029,"data_compressed_allocated":3231744,"data_compressed_original":48884839,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":3526682,"data_compressed":116000,"data_compressed_allocated":368640,"data_compressed_original":3073567,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4403200,"data_stored":19775975,"data_compressed":454956,"data_compressed_allocated":2322432,"data_compressed_original":17546552,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:31:33.154 INFO:tasks.thrashosds.thrasher:Setting 3.1c to [0, 3, 6, 5] 2024-09-17T21:31:33.154 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1c', '0', '3', '6', '5'] 2024-09-17T21:31:33.154 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1c 0 3 6 5 2024-09-17T21:31:33.183 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:33.184 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:33.192 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:31:33.236 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:33.234+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:33.296 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:33.296 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:33.305 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:31:33.318 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:33.316+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:33.408 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:33.408 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:33.416 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:31:33.419 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:33.417+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:33.520 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:33.517+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:33.520 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:33.520 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:33.529 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:31:33.551 INFO:teuthology.orchestra.run.smithi129.stderr:set 3.1c pg_upmap_items mapping to [0->3,6->5] 2024-09-17T21:31:33.594 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:31:33.656 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:33.654+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:33.687 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:33.687 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:33.697 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:31:33.741 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:33.739+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:33.801 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:33.801 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:33.809 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:31:33.850 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:33.848+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:33.923 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:33.921+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:33.952 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:33.952 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:33.961 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:31:34.045 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:34.043+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:34.106 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:34.106 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:34.114 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:31:34.158 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:34.156+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:34.173 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:31:34.220 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:34.220 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:34.225 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:34.223+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:34.229 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:31:34.297 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:34.297 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:34.314 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:34.315 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:34.327 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:34.324+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:34.336 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:34.336 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:34.347 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:34.347 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:34.347 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:34.347 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:34.347 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:34.360 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:31:34.363 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:34.363 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:34.380 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:34.380 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:34.397 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:34.397 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:34.414 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:34.414 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:34.428 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:34.426+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:34.430 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:34.430 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:34.469 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:34.469 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:34.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:34.469 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97778 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 245760~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:31:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.49982412, 2024-09-17T21:31:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050933728999999997, 2024-09-17T21:31:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97778 2024-09-17T21:31:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:31:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:31:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966899+0000", 2024-09-17T21:31:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:31:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966900+0000", 2024-09-17T21:31:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.384e-06 2024-09-17T21:31:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966905+0000", 2024-09-17T21:31:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6530000000000002e-06 2024-09-17T21:31:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966909+0000", 2024-09-17T21:31:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2889999999999999e-06 2024-09-17T21:31:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017585+0000", 2024-09-17T21:31:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050675691000000002 2024-09-17T21:31:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017642+0000", 2024-09-17T21:31:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7167999999999999e-05 2024-09-17T21:31:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017832+0000", 2024-09-17T21:31:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019022 2024-09-17T21:31:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97779 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:31:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.499807052000001, 2024-09-17T21:31:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051245027999999998, 2024-09-17T21:31:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97779 2024-09-17T21:31:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:31:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:31:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966916+0000", 2024-09-17T21:31:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:31:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966918+0000", 2024-09-17T21:31:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8369999999999999e-06 2024-09-17T21:31:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966924+0000", 2024-09-17T21:31:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2260000000000002e-06 2024-09-17T21:31:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966928+0000", 2024-09-17T21:31:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3819999999999997e-06 2024-09-17T21:31:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017866+0000", 2024-09-17T21:31:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050938079999999997 2024-09-17T21:31:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017945+0000", 2024-09-17T21:31:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8275000000000005e-05 2024-09-17T21:31:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018161+0000", 2024-09-17T21:31:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021579499999999999 2024-09-17T21:31:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97780 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 344064~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.499782774000003, 2024-09-17T21:31:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051535301999999998, 2024-09-17T21:31:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97780 2024-09-17T21:31:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:31:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966942+0000", 2024-09-17T21:31:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7439999999999999e-06 2024-09-17T21:31:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966946+0000", 2024-09-17T21:31:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5510000000000001e-06 2024-09-17T21:31:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966951+0000", 2024-09-17T21:31:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:31:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018191+0000", 2024-09-17T21:31:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051240290000000001 2024-09-17T21:31:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018269+0000", 2024-09-17T21:31:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7416999999999994e-05 2024-09-17T21:31:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018475+0000", 2024-09-17T21:31:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206486 2024-09-17T21:31:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97789 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 40960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.499762605000001, 2024-09-17T21:31:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051862143999999999, 2024-09-17T21:31:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97789 2024-09-17T21:31:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5899999999999996e-07 2024-09-17T21:31:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966962+0000", 2024-09-17T21:31:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9700000000000002e-06 2024-09-17T21:31:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966969+0000", 2024-09-17T21:31:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6939999999999998e-06 2024-09-17T21:31:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966980+0000", 2024-09-17T21:31:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0679999999999999e-05 2024-09-17T21:31:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018513+0000", 2024-09-17T21:31:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051533420000000003 2024-09-17T21:31:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018595+0000", 2024-09-17T21:31:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1260999999999993e-05 2024-09-17T21:31:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018822+0000", 2024-09-17T21:31:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022756 2024-09-17T21:31:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97790 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 90112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:31:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.499732107, 2024-09-17T21:31:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052173479000000002, 2024-09-17T21:31:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97790 2024-09-17T21:31:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:31:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:31:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966991+0000", 2024-09-17T21:31:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9500000000000003e-07 2024-09-17T21:31:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966993+0000", 2024-09-17T21:31:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.238e-06 2024-09-17T21:31:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967000+0000", 2024-09-17T21:31:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9500000000000004e-06 2024-09-17T21:31:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967005+0000", 2024-09-17T21:31:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:31:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018852+0000", 2024-09-17T21:31:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051847250999999997 2024-09-17T21:31:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018927+0000", 2024-09-17T21:31:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4944000000000006e-05 2024-09-17T21:31:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019164+0000", 2024-09-17T21:31:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000236803 2024-09-17T21:31:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97791 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 139264~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:31:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.499710362999998, 2024-09-17T21:31:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052464594000000003, 2024-09-17T21:31:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97791 2024-09-17T21:31:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:31:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:31:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967013+0000", 2024-09-17T21:31:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:31:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967014+0000", 2024-09-17T21:31:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8679999999999999e-06 2024-09-17T21:31:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967020+0000", 2024-09-17T21:31:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.322e-06 2024-09-17T21:31:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967024+0000", 2024-09-17T21:31:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2660000000000003e-06 2024-09-17T21:31:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019193+0000", 2024-09-17T21:31:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052169010000000002 2024-09-17T21:31:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019269+0000", 2024-09-17T21:31:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6149999999999994e-05 2024-09-17T21:31:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019477+0000", 2024-09-17T21:31:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020753699999999999 2024-09-17T21:31:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97792 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 188416~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.499692623999998, 2024-09-17T21:31:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052764958000000001, 2024-09-17T21:31:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97792 2024-09-17T21:31:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0900000000000002e-07 2024-09-17T21:31:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967032+0000", 2024-09-17T21:31:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9740000000000001e-06 2024-09-17T21:31:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967039+0000", 2024-09-17T21:31:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3679999999999998e-06 2024-09-17T21:31:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967043+0000", 2024-09-17T21:31:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3379999999999998e-06 2024-09-17T21:31:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019505+0000", 2024-09-17T21:31:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052462264000000002 2024-09-17T21:31:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019582+0000", 2024-09-17T21:31:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7146999999999998e-05 2024-09-17T21:31:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019795+0000", 2024-09-17T21:31:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000212458 2024-09-17T21:31:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97793 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 237568~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:31:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.499671124000002, 2024-09-17T21:31:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053070436999999998, 2024-09-17T21:31:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97793 2024-09-17T21:31:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:31:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:31:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967052+0000", 2024-09-17T21:31:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7500000000000001e-07 2024-09-17T21:31:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967054+0000", 2024-09-17T21:31:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:31:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967059+0000", 2024-09-17T21:31:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5840000000000002e-06 2024-09-17T21:31:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967064+0000", 2024-09-17T21:31:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:31:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019826+0000", 2024-09-17T21:31:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052762396000000003 2024-09-17T21:31:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019900+0000", 2024-09-17T21:31:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3594e-05 2024-09-17T21:31:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020122+0000", 2024-09-17T21:31:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022185999999999999 2024-09-17T21:31:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97794 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 286720~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.499650987999999, 2024-09-17T21:31:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053453734000000003, 2024-09-17T21:31:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97794 2024-09-17T21:31:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2199999999999999e-07 2024-09-17T21:31:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967074+0000", 2024-09-17T21:31:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8419999999999999e-06 2024-09-17T21:31:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967080+0000", 2024-09-17T21:31:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5660000000000003e-06 2024-09-17T21:31:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967120+0000", 2024-09-17T21:31:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9504e-05 2024-09-17T21:31:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020189+0000", 2024-09-17T21:31:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053069144999999998 2024-09-17T21:31:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020287+0000", 2024-09-17T21:31:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8259999999999998e-05 2024-09-17T21:31:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020525+0000", 2024-09-17T21:31:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023799499999999999 2024-09-17T21:31:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97795 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 335872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.499592853999999, 2024-09-17T21:31:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053707207999999999, 2024-09-17T21:31:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97795 2024-09-17T21:31:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:31:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967139+0000", 2024-09-17T21:31:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5790000000000004e-06 2024-09-17T21:31:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967145+0000", 2024-09-17T21:31:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3550000000000001e-06 2024-09-17T21:31:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967155+0000", 2024-09-17T21:31:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8859999999999999e-06 2024-09-17T21:31:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020555+0000", 2024-09-17T21:31:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053400359000000001 2024-09-17T21:31:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020630+0000", 2024-09-17T21:31:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4894999999999994e-05 2024-09-17T21:31:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020837+0000", 2024-09-17T21:31:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020670500000000001 2024-09-17T21:31:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97796 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 385024~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.499560594000002, 2024-09-17T21:31:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053990227000000002, 2024-09-17T21:31:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97796 2024-09-17T21:31:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:31:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967164+0000", 2024-09-17T21:31:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0370000000000001e-06 2024-09-17T21:31:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967171+0000", 2024-09-17T21:31:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2190000000000001e-06 2024-09-17T21:31:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967175+0000", 2024-09-17T21:31:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2599999999999999e-06 2024-09-17T21:31:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020866+0000", 2024-09-17T21:31:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053691539000000003 2024-09-17T21:31:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020942+0000", 2024-09-17T21:31:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5487000000000004e-05 2024-09-17T21:31:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021152+0000", 2024-09-17T21:31:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021021599999999999 2024-09-17T21:31:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97805 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 8192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.499541913999998, 2024-09-17T21:31:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054331771000000001, 2024-09-17T21:31:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97805 2024-09-17T21:31:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:31:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967183+0000", 2024-09-17T21:31:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6840000000000001e-06 2024-09-17T21:31:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967189+0000", 2024-09-17T21:31:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0299999999999999e-06 2024-09-17T21:31:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967193+0000", 2024-09-17T21:31:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-17T21:31:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021182+0000", 2024-09-17T21:31:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053989319000000001 2024-09-17T21:31:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021261+0000", 2024-09-17T21:31:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8460000000000004e-05 2024-09-17T21:31:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021512+0000", 2024-09-17T21:31:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025158500000000002 2024-09-17T21:31:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97806 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.499523910999997, 2024-09-17T21:31:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054654343000000001, 2024-09-17T21:31:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97806 2024-09-17T21:31:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:31:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967201+0000", 2024-09-17T21:31:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.691e-06 2024-09-17T21:31:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967206+0000", 2024-09-17T21:31:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7780000000000004e-06 2024-09-17T21:31:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967224+0000", 2024-09-17T21:31:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7357999999999999e-05 2024-09-17T21:31:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021546+0000", 2024-09-17T21:31:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054322237000000002 2024-09-17T21:31:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021618+0000", 2024-09-17T21:31:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2007000000000004e-05 2024-09-17T21:31:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021853+0000", 2024-09-17T21:31:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023495800000000001 2024-09-17T21:31:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97807 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 106496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:31:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.499491511000002, 2024-09-17T21:31:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055014183000000001, 2024-09-17T21:31:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97807 2024-09-17T21:31:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:31:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:31:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967232+0000", 2024-09-17T21:31:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:31:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967234+0000", 2024-09-17T21:31:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.283e-06 2024-09-17T21:31:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967240+0000", 2024-09-17T21:31:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.144e-06 2024-09-17T21:31:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967252+0000", 2024-09-17T21:31:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2322e-05 2024-09-17T21:31:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021882+0000", 2024-09-17T21:31:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054629952000000002 2024-09-17T21:31:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021952+0000", 2024-09-17T21:31:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9919999999999995e-05 2024-09-17T21:31:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022245+0000", 2024-09-17T21:31:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00029302100000000003 2024-09-17T21:31:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97808 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 155648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.499462911999998, 2024-09-17T21:31:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055447485999999997, 2024-09-17T21:31:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97808 2024-09-17T21:31:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4299999999999998e-07 2024-09-17T21:31:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967267+0000", 2024-09-17T21:31:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9820000000000002e-06 2024-09-17T21:31:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967273+0000", 2024-09-17T21:31:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:31:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967277+0000", 2024-09-17T21:31:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5050000000000001e-06 2024-09-17T21:31:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022297+0000", 2024-09-17T21:31:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055020131999999999 2024-09-17T21:31:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022379+0000", 2024-09-17T21:31:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1810000000000004e-05 2024-09-17T21:31:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022707+0000", 2024-09-17T21:31:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00032784300000000003 2024-09-17T21:31:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97809 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:31:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.499439397000003, 2024-09-17T21:31:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056012109999999997, 2024-09-17T21:31:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97809 2024-09-17T21:31:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:31:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:31:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967284+0000", 2024-09-17T21:31:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:31:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967285+0000", 2024-09-17T21:31:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.826e-06 2024-09-17T21:31:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967291+0000", 2024-09-17T21:31:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:31:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967302+0000", 2024-09-17T21:31:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1114999999999999e-05 2024-09-17T21:31:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022901+0000", 2024-09-17T21:31:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055599335 2024-09-17T21:31:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022990+0000", 2024-09-17T21:31:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8671999999999994e-05 2024-09-17T21:31:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023295+0000", 2024-09-17T21:31:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030554900000000001 2024-09-17T21:31:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97810 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 253952~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.499412968999998, 2024-09-17T21:31:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056355668999999997, 2024-09-17T21:31:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97810 2024-09-17T21:31:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0200000000000002e-07 2024-09-17T21:31:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967313+0000", 2024-09-17T21:31:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4549999999999998e-06 2024-09-17T21:31:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967319+0000", 2024-09-17T21:31:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3080000000000004e-06 2024-09-17T21:31:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967324+0000", 2024-09-17T21:31:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0159999999999999e-06 2024-09-17T21:31:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023342+0000", 2024-09-17T21:31:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056018635999999997 2024-09-17T21:31:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023423+0000", 2024-09-17T21:31:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0473e-05 2024-09-17T21:31:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023665+0000", 2024-09-17T21:31:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242279 2024-09-17T21:31:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97811 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 303104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.499386711, 2024-09-17T21:31:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056743978, 2024-09-17T21:31:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97811 2024-09-17T21:31:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1300000000000001e-07 2024-09-17T21:31:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967338+0000", 2024-09-17T21:31:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9740000000000001e-06 2024-09-17T21:31:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967346+0000", 2024-09-17T21:31:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8129999999999996e-06 2024-09-17T21:31:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967351+0000", 2024-09-17T21:31:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6129999999999997e-06 2024-09-17T21:31:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023750+0000", 2024-09-17T21:31:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056399796000000002 2024-09-17T21:31:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023839+0000", 2024-09-17T21:31:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.852e-05 2024-09-17T21:31:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024080+0000", 2024-09-17T21:31:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024084900000000001 2024-09-17T21:31:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97812 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 352256~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:31:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.499366094999999, 2024-09-17T21:31:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.057024455000000002, 2024-09-17T21:31:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97812 2024-09-17T21:31:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:31:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:31:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967357+0000", 2024-09-17T21:31:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:31:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967359+0000", 2024-09-17T21:31:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:31:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967365+0000", 2024-09-17T21:31:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:31:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967369+0000", 2024-09-17T21:31:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5029999999999999e-06 2024-09-17T21:31:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024109+0000", 2024-09-17T21:31:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056740038 2024-09-17T21:31:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024175+0000", 2024-09-17T21:31:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5106999999999999e-05 2024-09-17T21:31:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024381+0000", 2024-09-17T21:31:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020629700000000001 2024-09-17T21:31:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:112033 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-09-17T21:31:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:31:11.421210+0000", 2024-09-17T21:31:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.045512329000001, 2024-09-17T21:31:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0104142869999999, 2024-09-17T21:31:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 112033 2024-09-17T21:31:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421210+0000", 2024-09-17T21:31:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421210+0000", 2024-09-17T21:31:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421212+0000", 2024-09-17T21:31:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.376e-06 2024-09-17T21:31:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421217+0000", 2024-09-17T21:31:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.259e-06 2024-09-17T21:31:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421228+0000", 2024-09-17T21:31:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0944000000000001e-05 2024-09-17T21:31:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421238+0000", 2024-09-17T21:31:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:31:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421626+0000", 2024-09-17T21:31:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00038804099999999999 2024-09-17T21:31:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:31:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421637+0000", 2024-09-17T21:31:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1783000000000001e-05 2024-09-17T21:31:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:12.431245+0000", 2024-09-17T21:31:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.00960765 2024-09-17T21:31:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:12.431413+0000", 2024-09-17T21:31:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000167784 2024-09-17T21:31:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:12.431624+0000", 2024-09-17T21:31:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000211704 2024-09-17T21:31:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.557 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:34.558 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:31:34.559 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:34.556+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:34.563 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is unset 2024-09-17T21:31:34.659 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:34.656+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:34.681 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:34.681 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:34.681 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:34.681 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:34.691 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:31:34.759 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:34.757+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:34.795 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:34.795 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:34.795 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:34.795 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:34.795 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:34.805 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:31:34.889 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:34.887+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:34.951 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:37817 3.17 3:eab85a8d:::rbd_header.1120e62eea89:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T21:31:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:31:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 93.252753197999994, 2024-09-17T21:31:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86635970399999995, 2024-09-17T21:31:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37817 2024-09-17T21:31:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:31:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695456+0000", 2024-09-17T21:31:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695457+0000", 2024-09-17T21:31:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5799999999999998e-07 2024-09-17T21:31:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695461+0000", 2024-09-17T21:31:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1110000000000001e-06 2024-09-17T21:31:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695471+0000", 2024-09-17T21:31:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0441e-05 2024-09-17T21:31:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695477+0000", 2024-09-17T21:31:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7599999999999999e-06 2024-09-17T21:31:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695533+0000", 2024-09-17T21:31:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6570999999999998e-05 2024-09-17T21:31:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:31:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:01.695543+0000", 2024-09-17T21:31:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9979999999999998e-06 2024-09-17T21:31:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561472+0000", 2024-09-17T21:31:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.86592841899999995 2024-09-17T21:31:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561610+0000", 2024-09-17T21:31:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013854699999999999 2024-09-17T21:31:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:02.561816+0000", 2024-09-17T21:31:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020509900000000001 2024-09-17T21:31:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50202 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:31:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.840582408000003, 2024-09-17T21:31:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.076620261999999995, 2024-09-17T21:31:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50202 2024-09-17T21:31:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:31:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107627+0000", 2024-09-17T21:31:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.107629+0000", 2024-09-17T21:31:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3709999999999998e-06 2024-09-17T21:31:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119765+0000", 2024-09-17T21:31:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012135683 2024-09-17T21:31:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119787+0000", 2024-09-17T21:31:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2748999999999999e-05 2024-09-17T21:31:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119797+0000", 2024-09-17T21:31:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8169999999999994e-06 2024-09-17T21:31:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119872+0000", 2024-09-17T21:31:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270999999999994e-05 2024-09-17T21:31:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119987+0000", 2024-09-17T21:31:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011557699999999999 2024-09-17T21:31:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.121766+0000", 2024-09-17T21:31:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00177853 2024-09-17T21:31:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178307+0000", 2024-09-17T21:31:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056541559999999998 2024-09-17T21:31:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184097+0000", 2024-09-17T21:31:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057899029999999999 2024-09-17T21:31:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184172+0000", 2024-09-17T21:31:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5226000000000001e-05 2024-09-17T21:31:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184247+0000", 2024-09-17T21:31:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4574999999999997e-05 2024-09-17T21:31:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50203 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head [truncate 20971520] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:31:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.828382503, 2024-09-17T21:31:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065009779000000004, 2024-09-17T21:31:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50203 2024-09-17T21:31:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:31:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119827+0000", 2024-09-17T21:31:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119828+0000", 2024-09-17T21:31:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6390000000000001e-06 2024-09-17T21:31:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119832+0000", 2024-09-17T21:31:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7699999999999999e-06 2024-09-17T21:31:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119845+0000", 2024-09-17T21:31:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3093e-05 2024-09-17T21:31:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.119852+0000", 2024-09-17T21:31:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6950000000000003e-06 2024-09-17T21:31:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.160945+0000", 2024-09-17T21:31:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041093076999999999 2024-09-17T21:31:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161071+0000", 2024-09-17T21:31:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012569799999999999 2024-09-17T21:31:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.161283+0000", 2024-09-17T21:31:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021201999999999999 2024-09-17T21:31:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.179360+0000", 2024-09-17T21:31:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018077784999999999 2024-09-17T21:31:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184726+0000", 2024-09-17T21:31:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053652960000000003 2024-09-17T21:31:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184787+0000", 2024-09-17T21:31:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1402000000000006e-05 2024-09-17T21:31:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.184836+0000", 2024-09-17T21:31:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9304000000000001e-05 2024-09-17T21:31:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50206 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:31:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.791300962999998, 2024-09-17T21:31:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.090361968000000001, 2024-09-17T21:31:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50206 2024-09-17T21:31:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:31:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156908+0000", 2024-09-17T21:31:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.156911+0000", 2024-09-17T21:31:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-17T21:31:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173670+0000", 2024-09-17T21:31:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016758907 2024-09-17T21:31:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173696+0000", 2024-09-17T21:31:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6329000000000002e-05 2024-09-17T21:31:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173708+0000", 2024-09-17T21:31:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2512e-05 2024-09-17T21:31:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173788+0000", 2024-09-17T21:31:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9332999999999994e-05 2024-09-17T21:31:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173918+0000", 2024-09-17T21:31:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013063300000000001 2024-09-17T21:31:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.176228+0000", 2024-09-17T21:31:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002309757 2024-09-17T21:31:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.245164+0000", 2024-09-17T21:31:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068936301000000005 2024-09-17T21:31:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247176+0000", 2024-09-17T21:31:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020117590000000001 2024-09-17T21:31:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:34.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247215+0000", 2024-09-17T21:31:34.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9094000000000002e-05 2024-09-17T21:31:34.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247270+0000", 2024-09-17T21:31:34.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4793999999999997e-05 2024-09-17T21:31:34.974 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.974 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.974 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.975 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50207 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head [truncate 21556327] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:34.975 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:31:34.975 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.774451983000006, 2024-09-17T21:31:34.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.074055363999999999, 2024-09-17T21:31:34.975 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.975 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.975 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:34.975 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.975 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.975 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.975 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50207 2024-09-17T21:31:34.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.975 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:31:34.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173757+0000", 2024-09-17T21:31:34.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173759+0000", 2024-09-17T21:31:34.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8160000000000001e-06 2024-09-17T21:31:34.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173763+0000", 2024-09-17T21:31:34.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5360000000000003e-06 2024-09-17T21:31:34.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173771+0000", 2024-09-17T21:31:34.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8849999999999999e-06 2024-09-17T21:31:34.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.173777+0000", 2024-09-17T21:31:34.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7799999999999997e-06 2024-09-17T21:31:34.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221363+0000", 2024-09-17T21:31:34.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047585493 2024-09-17T21:31:34.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221430+0000", 2024-09-17T21:31:34.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7129000000000001e-05 2024-09-17T21:31:34.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:34.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221550+0000", 2024-09-17T21:31:34.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120658 2024-09-17T21:31:34.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T21:31:34.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246113+0000", 2024-09-17T21:31:34.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02456262 2024-09-17T21:31:34.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:34.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247744+0000", 2024-09-17T21:31:34.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001630692 2024-09-17T21:31:34.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:34.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247777+0000", 2024-09-17T21:31:34.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3015000000000002e-05 2024-09-17T21:31:34.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247812+0000", 2024-09-17T21:31:34.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.574e-05 2024-09-17T21:31:34.980 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.980 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.980 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.980 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50828 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:34.981 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:34.981 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.466340051000003, 2024-09-17T21:31:34.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047324185999999997, 2024-09-17T21:31:34.981 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.981 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.981 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.981 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.981 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.981 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.981 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50828 2024-09-17T21:31:34.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.981 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:34.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:34.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481869+0000", 2024-09-17T21:31:34.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.46e-07 2024-09-17T21:31:34.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481871+0000", 2024-09-17T21:31:34.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T21:31:34.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481878+0000", 2024-09-17T21:31:34.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:31:34.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481882+0000", 2024-09-17T21:31:34.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:31:34.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529038+0000", 2024-09-17T21:31:34.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047156034999999999 2024-09-17T21:31:34.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529099+0000", 2024-09-17T21:31:34.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0316000000000003e-05 2024-09-17T21:31:34.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529193+0000", 2024-09-17T21:31:34.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4369000000000004e-05 2024-09-17T21:31:34.984 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.985 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.985 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.985 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50829 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 12845056~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:34.985 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:34.985 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.466321257000004, 2024-09-17T21:31:34.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047887596999999997, 2024-09-17T21:31:34.985 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.985 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.985 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.985 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.985 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.985 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.986 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50829 2024-09-17T21:31:34.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.986 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:34.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:34.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481888+0000", 2024-09-17T21:31:34.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:31:34.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481890+0000", 2024-09-17T21:31:34.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.877e-06 2024-09-17T21:31:34.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481895+0000", 2024-09-17T21:31:34.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:31:34.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481900+0000", 2024-09-17T21:31:34.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4440000000000001e-06 2024-09-17T21:31:34.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529225+0000", 2024-09-17T21:31:34.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047325117999999999 2024-09-17T21:31:34.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529292+0000", 2024-09-17T21:31:34.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7100000000000005e-05 2024-09-17T21:31:34.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529775+0000", 2024-09-17T21:31:34.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048322799999999999 2024-09-17T21:31:34.989 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.989 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.989 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.989 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50830 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:34.989 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:31:34.989 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.466302854999995, 2024-09-17T21:31:34.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048062123999999998, 2024-09-17T21:31:34.989 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.990 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.990 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.990 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.990 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.990 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.990 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50830 2024-09-17T21:31:34.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.990 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:31:34.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481906+0000", 2024-09-17T21:31:34.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481907+0000", 2024-09-17T21:31:34.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:31:34.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481909+0000", 2024-09-17T21:31:34.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T21:31:34.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481915+0000", 2024-09-17T21:31:34.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.02e-06 2024-09-17T21:31:34.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481920+0000", 2024-09-17T21:31:34.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6799999999999998e-06 2024-09-17T21:31:34.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529793+0000", 2024-09-17T21:31:34.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047873042999999997 2024-09-17T21:31:34.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529870+0000", 2024-09-17T21:31:34.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6967999999999999e-05 2024-09-17T21:31:34.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529968+0000", 2024-09-17T21:31:34.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8085999999999996e-05 2024-09-17T21:31:34.993 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.993 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.993 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.994 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50831 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 18087936~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:34.994 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:34.994 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.466282508999996, 2024-09-17T21:31:34.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048663304999999997, 2024-09-17T21:31:34.994 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.994 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.994 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.994 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.994 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.994 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.994 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50831 2024-09-17T21:31:34.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.994 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:34.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:34.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:34.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481927+0000", 2024-09-17T21:31:34.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:31:34.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:34.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481929+0000", 2024-09-17T21:31:34.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2189999999999998e-06 2024-09-17T21:31:34.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:34.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481935+0000", 2024-09-17T21:31:34.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.383e-06 2024-09-17T21:31:34.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:34.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481939+0000", 2024-09-17T21:31:34.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2479999999999998e-06 2024-09-17T21:31:34.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:34.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530095+0000", 2024-09-17T21:31:34.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048156617999999998 2024-09-17T21:31:34.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:34.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530159+0000", 2024-09-17T21:31:34.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3394999999999999e-05 2024-09-17T21:31:34.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:34.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530590+0000", 2024-09-17T21:31:34.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043101199999999999 2024-09-17T21:31:34.998 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.998 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:34.998 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:34.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.998 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50832 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 19660800~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:34.998 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:31:34.998 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.466263862000005, 2024-09-17T21:31:34.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049339466999999998, 2024-09-17T21:31:34.998 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:34.998 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:34.998 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:34.998 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:34.999 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:34.999 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:34.999 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50832 2024-09-17T21:31:34.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.999 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:34.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:34.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:31:34.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:34.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:34.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:34.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:34.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481945+0000", 2024-09-17T21:31:34.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481946+0000", 2024-09-17T21:31:35.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:35.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481951+0000", 2024-09-17T21:31:35.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1479999999999997e-06 2024-09-17T21:31:35.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481956+0000", 2024-09-17T21:31:35.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7339999999999996e-06 2024-09-17T21:31:35.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481961+0000", 2024-09-17T21:31:35.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4889999999999997e-06 2024-09-17T21:31:35.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530630+0000", 2024-09-17T21:31:35.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048668596000000001 2024-09-17T21:31:35.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530720+0000", 2024-09-17T21:31:35.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0163999999999999e-05 2024-09-17T21:31:35.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531285+0000", 2024-09-17T21:31:35.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056493199999999998 2024-09-17T21:31:35.002 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.002 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.002 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.002 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50833 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 23330816~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.002 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:31:35.002 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.466241718000006, 2024-09-17T21:31:35.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049908069999999999, 2024-09-17T21:31:35.003 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.003 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.003 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.003 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.003 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.003 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.003 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50833 2024-09-17T21:31:35.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.003 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:31:35.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481967+0000", 2024-09-17T21:31:35.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481968+0000", 2024-09-17T21:31:35.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5600000000000001e-07 2024-09-17T21:31:35.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481970+0000", 2024-09-17T21:31:35.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9539999999999998e-06 2024-09-17T21:31:35.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481975+0000", 2024-09-17T21:31:35.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8499999999999999e-06 2024-09-17T21:31:35.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481980+0000", 2024-09-17T21:31:35.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-17T21:31:35.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531337+0000", 2024-09-17T21:31:35.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049356874000000002 2024-09-17T21:31:35.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531392+0000", 2024-09-17T21:31:35.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4826e-05 2024-09-17T21:31:35.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531875+0000", 2024-09-17T21:31:35.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048345099999999998 2024-09-17T21:31:35.009 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.009 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.009 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.009 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50834 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 24903680~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.010 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:31:35.010 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.466216950000003, 2024-09-17T21:31:35.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050534015000000002, 2024-09-17T21:31:35.010 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.010 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.010 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.010 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.010 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.010 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.010 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50834 2024-09-17T21:31:35.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.010 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:31:35.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481992+0000", 2024-09-17T21:31:35.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481993+0000", 2024-09-17T21:31:35.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:31:35.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.481995+0000", 2024-09-17T21:31:35.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.199e-06 2024-09-17T21:31:35.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482001+0000", 2024-09-17T21:31:35.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.772e-06 2024-09-17T21:31:35.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482006+0000", 2024-09-17T21:31:35.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0660000000000003e-06 2024-09-17T21:31:35.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531923+0000", 2024-09-17T21:31:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049917200000000002 2024-09-17T21:31:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532004+0000", 2024-09-17T21:31:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0758000000000002e-05 2024-09-17T21:31:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532526+0000", 2024-09-17T21:31:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00052258899999999995 2024-09-17T21:31:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50835 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 28573696~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:31:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.466196967000002, 2024-09-17T21:31:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051204701999999998, 2024-09-17T21:31:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50835 2024-09-17T21:31:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:31:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482012+0000", 2024-09-17T21:31:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482013+0000", 2024-09-17T21:31:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:31:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482015+0000", 2024-09-17T21:31:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.272e-06 2024-09-17T21:31:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482020+0000", 2024-09-17T21:31:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3040000000000004e-06 2024-09-17T21:31:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482025+0000", 2024-09-17T21:31:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5650000000000003e-06 2024-09-17T21:31:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532569+0000", 2024-09-17T21:31:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050544661999999997 2024-09-17T21:31:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532649+0000", 2024-09-17T21:31:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9640000000000003e-05 2024-09-17T21:31:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533217+0000", 2024-09-17T21:31:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056784100000000003 2024-09-17T21:31:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50836 3.1f 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:10 [sparse-read 30146560~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.466177349999995, 2024-09-17T21:31:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051875589, 2024-09-17T21:31:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50836 2024-09-17T21:31:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482032+0000", 2024-09-17T21:31:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3700000000000001e-07 2024-09-17T21:31:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482034+0000", 2024-09-17T21:31:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0059999999999999e-06 2024-09-17T21:31:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482040+0000", 2024-09-17T21:31:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:31:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482044+0000", 2024-09-17T21:31:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:31:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533260+0000", 2024-09-17T21:31:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051215416 2024-09-17T21:31:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533348+0000", 2024-09-17T21:31:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8695000000000004e-05 2024-09-17T21:31:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533907+0000", 2024-09-17T21:31:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055902199999999995 2024-09-17T21:31:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50844 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 20185088~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:31:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.466013923999995, 2024-09-17T21:31:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0475644, 2024-09-17T21:31:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50844 2024-09-17T21:31:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:31:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482195+0000", 2024-09-17T21:31:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482196+0000", 2024-09-17T21:31:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.39e-07 2024-09-17T21:31:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482198+0000", 2024-09-17T21:31:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0099999999999998e-06 2024-09-17T21:31:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482204+0000", 2024-09-17T21:31:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.93e-06 2024-09-17T21:31:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482208+0000", 2024-09-17T21:31:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7439999999999998e-06 2024-09-17T21:31:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529318+0000", 2024-09-17T21:31:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047110051999999999 2024-09-17T21:31:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529396+0000", 2024-09-17T21:31:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7712000000000004e-05 2024-09-17T21:31:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529760+0000", 2024-09-17T21:31:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00036351299999999999 2024-09-17T21:31:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50845 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 21757952~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:31:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.465994820999995, 2024-09-17T21:31:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048084625999999998, 2024-09-17T21:31:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50845 2024-09-17T21:31:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:31:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482214+0000", 2024-09-17T21:31:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482215+0000", 2024-09-17T21:31:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:31:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482216+0000", 2024-09-17T21:31:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.823e-06 2024-09-17T21:31:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482222+0000", 2024-09-17T21:31:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.507e-06 2024-09-17T21:31:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482227+0000", 2024-09-17T21:31:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6609999999999999e-06 2024-09-17T21:31:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529792+0000", 2024-09-17T21:31:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.047565075999999998 2024-09-17T21:31:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.529871+0000", 2024-09-17T21:31:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8855000000000003e-05 2024-09-17T21:31:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530299+0000", 2024-09-17T21:31:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000428307 2024-09-17T21:31:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50846 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 25427968~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.465972433999994, 2024-09-17T21:31:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048735142000000002, 2024-09-17T21:31:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50846 2024-09-17T21:31:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482237+0000", 2024-09-17T21:31:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:31:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482239+0000", 2024-09-17T21:31:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.871e-06 2024-09-17T21:31:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482244+0000", 2024-09-17T21:31:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.49e-06 2024-09-17T21:31:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482249+0000", 2024-09-17T21:31:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6269999999999999e-06 2024-09-17T21:31:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530347+0000", 2024-09-17T21:31:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048097721000000003 2024-09-17T21:31:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530437+0000", 2024-09-17T21:31:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0604000000000002e-05 2024-09-17T21:31:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530972+0000", 2024-09-17T21:31:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053445000000000001 2024-09-17T21:31:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50847 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 27000832~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.465954538000005, 2024-09-17T21:31:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049308668, 2024-09-17T21:31:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50847 2024-09-17T21:31:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482255+0000", 2024-09-17T21:31:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:31:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482257+0000", 2024-09-17T21:31:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.846e-06 2024-09-17T21:31:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482264+0000", 2024-09-17T21:31:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8170000000000001e-06 2024-09-17T21:31:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482283+0000", 2024-09-17T21:31:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9718e-05 2024-09-17T21:31:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531018+0000", 2024-09-17T21:31:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.048734497000000002 2024-09-17T21:31:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531080+0000", 2024-09-17T21:31:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1795e-05 2024-09-17T21:31:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531563+0000", 2024-09-17T21:31:35.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048366499999999998 2024-09-17T21:31:35.041 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.041 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.041 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.041 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50848 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 30670848~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.041 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:31:35.041 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.465917996000002, 2024-09-17T21:31:35.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049893337000000003, 2024-09-17T21:31:35.041 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.041 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.041 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.041 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.042 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.042 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.042 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50848 2024-09-17T21:31:35.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.042 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:31:35.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482291+0000", 2024-09-17T21:31:35.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482292+0000", 2024-09-17T21:31:35.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T21:31:35.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482294+0000", 2024-09-17T21:31:35.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3209999999999999e-06 2024-09-17T21:31:35.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482300+0000", 2024-09-17T21:31:35.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7189999999999998e-06 2024-09-17T21:31:35.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482305+0000", 2024-09-17T21:31:35.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T21:31:35.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531606+0000", 2024-09-17T21:31:35.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049301273999999999 2024-09-17T21:31:35.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531696+0000", 2024-09-17T21:31:35.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0161999999999995e-05 2024-09-17T21:31:35.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532184+0000", 2024-09-17T21:31:35.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00048806400000000002 2024-09-17T21:31:35.045 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.045 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.045 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.045 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50849 3.1 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:10 [sparse-read 32243712~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.045 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:31:35.046 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 83.465898554999995, 2024-09-17T21:31:35.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050444607000000002, 2024-09-17T21:31:35.046 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.046 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.046 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.046 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50849 2024-09-17T21:31:35.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.046 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:31:35.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482310+0000", 2024-09-17T21:31:35.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482311+0000", 2024-09-17T21:31:35.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:35.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482316+0000", 2024-09-17T21:31:35.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1209999999999998e-06 2024-09-17T21:31:35.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482321+0000", 2024-09-17T21:31:35.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4779999999999999e-06 2024-09-17T21:31:35.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.482326+0000", 2024-09-17T21:31:35.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7099999999999998e-06 2024-09-17T21:31:35.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532221+0000", 2024-09-17T21:31:35.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.049894936000000001 2024-09-17T21:31:35.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532287+0000", 2024-09-17T21:31:35.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5914000000000006e-05 2024-09-17T21:31:35.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532755+0000", 2024-09-17T21:31:35.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000468107 2024-09-17T21:31:35.049 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.049 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.050 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.050 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.050 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.050 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:35.051 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:31:35.052 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:35.049+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:35.152 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:35.150+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:35.195 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:35.196 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:35.196 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:35.196 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:35.204 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:31:35.289 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:35.286+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:35.347 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:35.348 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:35.348 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:35.348 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:35.348 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:35.354 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:35.352+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:35.356 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:31:35.456 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:35.453+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:35.502 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "pg_update_log_missing(3.5 epoch 45/37 rep_tid 844 entries 45'73 (0'0) error 3:a65c40ae:::rbd_data.1120b21c3309.0000000000000126:head by client.4384.0:48900 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 45'70)", 2024-09-17T21:31:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:31:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.452105278000005, 2024-09-17T21:31:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019554365000000001, 2024-09-17T21:31:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T21:31:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:31:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:31:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047112+0000", 2024-09-17T21:31:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:31:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047121+0000", 2024-09-17T21:31:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4379999999999995e-06 2024-09-17T21:31:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047139+0000", 2024-09-17T21:31:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8235000000000001e-05 2024-09-17T21:31:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047303+0000", 2024-09-17T21:31:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016389400000000001 2024-09-17T21:31:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047396+0000", 2024-09-17T21:31:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3355000000000003e-05 2024-09-17T21:31:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066666+0000", 2024-09-17T21:31:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019269649999999999 2024-09-17T21:31:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48906 3.4 e45/25 3:2503fe27:::rbd_data.1120b21c3309.0000000000000127:head v 45'148, mlcod=45'146)", 2024-09-17T21:31:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.449979099000004, 2024-09-17T21:31:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018169739000000001, 2024-09-17T21:31:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:31:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049244+0000", 2024-09-17T21:31:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8390000000000003e-06 2024-09-17T21:31:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066411+0000", 2024-09-17T21:31:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017166818 2024-09-17T21:31:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066426+0000", 2024-09-17T21:31:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4888999999999999e-05 2024-09-17T21:31:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066611+0000", 2024-09-17T21:31:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018464499999999999 2024-09-17T21:31:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066630+0000", 2024-09-17T21:31:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9201000000000001e-05 2024-09-17T21:31:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067373+0000", 2024-09-17T21:31:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00074322600000000004 2024-09-17T21:31:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067407+0000", 2024-09-17T21:31:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4357999999999999e-05 2024-09-17T21:31:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50206 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'301, mlcod=45'299)", 2024-09-17T21:31:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:31:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.320502254999994, 2024-09-17T21:31:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068285810000000002, 2024-09-17T21:31:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:31:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:31:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178716+0000", 2024-09-17T21:31:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.438e-06 2024-09-17T21:31:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192044+0000", 2024-09-17T21:31:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013328370000000001 2024-09-17T21:31:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192069+0000", 2024-09-17T21:31:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4496000000000001e-05 2024-09-17T21:31:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192085+0000", 2024-09-17T21:31:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5758999999999999e-05 2024-09-17T21:31:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192167+0000", 2024-09-17T21:31:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2842000000000004e-05 2024-09-17T21:31:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192201+0000", 2024-09-17T21:31:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3127999999999999e-05 2024-09-17T21:31:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246938+0000", 2024-09-17T21:31:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054736982000000003 2024-09-17T21:31:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247000+0000", 2024-09-17T21:31:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2794999999999998e-05 2024-09-17T21:31:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50207 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'303, mlcod=45'299)", 2024-09-17T21:31:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:31:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.277555206000002, 2024-09-17T21:31:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026003433999999999, 2024-09-17T21:31:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:31:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:31:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221663+0000", 2024-09-17T21:31:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.271e-06 2024-09-17T21:31:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221672+0000", 2024-09-17T21:31:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6810000000000004e-06 2024-09-17T21:31:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221684+0000", 2024-09-17T21:31:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:31:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221691+0000", 2024-09-17T21:31:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:31:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221743+0000", 2024-09-17T21:31:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2868999999999998e-05 2024-09-17T21:31:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221765+0000", 2024-09-17T21:31:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1316999999999999e-05 2024-09-17T21:31:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247616+0000", 2024-09-17T21:31:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025851504000000001 2024-09-17T21:31:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247665+0000", 2024-09-17T21:31:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8739000000000002e-05 2024-09-17T21:31:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57808 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 196608~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.721256166000003, 2024-09-17T21:31:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017751475999999999, 2024-09-17T21:31:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57808 2024-09-17T21:31:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:31:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777968+0000", 2024-09-17T21:31:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8530000000000003e-06 2024-09-17T21:31:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777975+0000", 2024-09-17T21:31:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4410000000000004e-06 2024-09-17T21:31:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777980+0000", 2024-09-17T21:31:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7210000000000002e-06 2024-09-17T21:31:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795323+0000", 2024-09-17T21:31:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017343319999999999 2024-09-17T21:31:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795410+0000", 2024-09-17T21:31:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6567e-05 2024-09-17T21:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795712+0000", 2024-09-17T21:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030219200000000001 2024-09-17T21:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57809 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 458752~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:31:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.721230313999996, 2024-09-17T21:31:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018092869000000001, 2024-09-17T21:31:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57809 2024-09-17T21:31:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:31:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:31:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777987+0000", 2024-09-17T21:31:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T21:31:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777989+0000", 2024-09-17T21:31:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8649999999999999e-06 2024-09-17T21:31:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777994+0000", 2024-09-17T21:31:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4199999999999998e-06 2024-09-17T21:31:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778004+0000", 2024-09-17T21:31:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6900000000000004e-06 2024-09-17T21:31:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795745+0000", 2024-09-17T21:31:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017740859000000001 2024-09-17T21:31:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795815+0000", 2024-09-17T21:31:35.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0219e-05 2024-09-17T21:31:35.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796079+0000", 2024-09-17T21:31:35.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000264446 2024-09-17T21:31:35.533 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.533 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.533 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.534 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57810 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 720896~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:35.534 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:31:35.534 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.721206586999998, 2024-09-17T21:31:35.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018361253000000001, 2024-09-17T21:31:35.534 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.534 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.534 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.534 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.534 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.534 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.534 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57810 2024-09-17T21:31:35.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.535 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:31:35.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:31:35.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778011+0000", 2024-09-17T21:31:35.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:31:35.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778012+0000", 2024-09-17T21:31:35.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.872e-06 2024-09-17T21:31:35.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778018+0000", 2024-09-17T21:31:35.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8150000000000002e-06 2024-09-17T21:31:35.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778023+0000", 2024-09-17T21:31:35.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3939999999999998e-06 2024-09-17T21:31:35.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796106+0000", 2024-09-17T21:31:35.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018083446999999999 2024-09-17T21:31:35.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796159+0000", 2024-09-17T21:31:35.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3029999999999999e-05 2024-09-17T21:31:35.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796371+0000", 2024-09-17T21:31:35.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021231700000000001 2024-09-17T21:31:35.538 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.539 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.539 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.539 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57811 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:35.539 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:35.539 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.721188166999994, 2024-09-17T21:31:35.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018666035000000001, 2024-09-17T21:31:35.539 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.539 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.539 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.539 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.540 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.540 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.540 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57811 2024-09-17T21:31:35.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.540 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:35.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:35.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:35.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:35.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778031+0000", 2024-09-17T21:31:35.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.796e-06 2024-09-17T21:31:35.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778036+0000", 2024-09-17T21:31:35.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3290000000000001e-06 2024-09-17T21:31:35.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778041+0000", 2024-09-17T21:31:35.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5839999999999996e-06 2024-09-17T21:31:35.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796409+0000", 2024-09-17T21:31:35.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018368703 2024-09-17T21:31:35.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796483+0000", 2024-09-17T21:31:35.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3751999999999995e-05 2024-09-17T21:31:35.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796695+0000", 2024-09-17T21:31:35.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021153 2024-09-17T21:31:35.544 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.544 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.544 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.544 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57812 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1245184~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:35.544 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:35.544 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.721169270000004, 2024-09-17T21:31:35.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019000064000000001, 2024-09-17T21:31:35.545 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.545 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.545 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.545 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57812 2024-09-17T21:31:35.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.545 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:35.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:35.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:35.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:31:35.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778050+0000", 2024-09-17T21:31:35.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7e-06 2024-09-17T21:31:35.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778055+0000", 2024-09-17T21:31:35.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4049999999999999e-06 2024-09-17T21:31:35.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778059+0000", 2024-09-17T21:31:35.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1479999999999999e-06 2024-09-17T21:31:35.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796734+0000", 2024-09-17T21:31:35.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018674399000000001 2024-09-17T21:31:35.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796786+0000", 2024-09-17T21:31:35.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2879e-05 2024-09-17T21:31:35.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797048+0000", 2024-09-17T21:31:35.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026118400000000003 2024-09-17T21:31:35.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.549 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.549 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57813 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1507328~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:35.549 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:31:35.550 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.721151378000002, 2024-09-17T21:31:35.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019279392999999999, 2024-09-17T21:31:35.550 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.550 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.550 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.550 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57813 2024-09-17T21:31:35.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.550 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:31:35.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:31:35.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778066+0000", 2024-09-17T21:31:35.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6199999999999999e-07 2024-09-17T21:31:35.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778068+0000", 2024-09-17T21:31:35.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7990000000000001e-06 2024-09-17T21:31:35.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778074+0000", 2024-09-17T21:31:35.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:31:35.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778078+0000", 2024-09-17T21:31:35.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2579999999999997e-06 2024-09-17T21:31:35.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797077+0000", 2024-09-17T21:31:35.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018999180000000001 2024-09-17T21:31:35.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797123+0000", 2024-09-17T21:31:35.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5834999999999997e-05 2024-09-17T21:31:35.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797345+0000", 2024-09-17T21:31:35.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022191100000000001 2024-09-17T21:31:35.554 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.554 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.554 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.555 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57814 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:35.555 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:31:35.555 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.721132506000004, 2024-09-17T21:31:35.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019604947000000001, 2024-09-17T21:31:35.555 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.555 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.555 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.555 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.555 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.555 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.556 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57814 2024-09-17T21:31:35.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.556 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:31:35.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:31:35.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778085+0000", 2024-09-17T21:31:35.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:31:35.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778086+0000", 2024-09-17T21:31:35.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7850000000000001e-06 2024-09-17T21:31:35.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778093+0000", 2024-09-17T21:31:35.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6449999999999999e-06 2024-09-17T21:31:35.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778098+0000", 2024-09-17T21:31:35.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:31:35.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797383+0000", 2024-09-17T21:31:35.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019284879000000001 2024-09-17T21:31:35.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797431+0000", 2024-09-17T21:31:35.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8856000000000001e-05 2024-09-17T21:31:35.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797689+0000", 2024-09-17T21:31:35.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025785800000000002 2024-09-17T21:31:35.560 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.560 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.560 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.560 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57815 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2031616~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:35.560 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:35.560 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.721111837999999, 2024-09-17T21:31:35.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019910231, 2024-09-17T21:31:35.560 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.560 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.561 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.561 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.561 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.561 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.561 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57815 2024-09-17T21:31:35.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.561 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:35.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:35.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:35.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:31:35.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778118+0000", 2024-09-17T21:31:35.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2967e-05 2024-09-17T21:31:35.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778125+0000", 2024-09-17T21:31:35.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.601e-06 2024-09-17T21:31:35.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778130+0000", 2024-09-17T21:31:35.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1839999999999998e-06 2024-09-17T21:31:35.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797720+0000", 2024-09-17T21:31:35.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019589766000000002 2024-09-17T21:31:35.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797765+0000", 2024-09-17T21:31:35.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5281999999999998e-05 2024-09-17T21:31:35.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798015+0000", 2024-09-17T21:31:35.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025006200000000001 2024-09-17T21:31:35.565 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.565 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.565 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.565 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57816 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2293760~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:35.566 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:35.566 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.721079243999995, 2024-09-17T21:31:35.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020175322999999998, 2024-09-17T21:31:35.566 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.566 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.566 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.566 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.566 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.566 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.566 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57816 2024-09-17T21:31:35.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.567 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:35.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:35.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:35.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:31:35.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778140+0000", 2024-09-17T21:31:35.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T21:31:35.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778145+0000", 2024-09-17T21:31:35.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7320000000000003e-06 2024-09-17T21:31:35.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778154+0000", 2024-09-17T21:31:35.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6969999999999999e-06 2024-09-17T21:31:35.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798049+0000", 2024-09-17T21:31:35.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019894603 2024-09-17T21:31:35.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798106+0000", 2024-09-17T21:31:35.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7207000000000003e-05 2024-09-17T21:31:35.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798313+0000", 2024-09-17T21:31:35.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206861 2024-09-17T21:31:35.571 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.571 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.571 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.571 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57817 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2555904~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:35.571 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:35.571 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.721055997999997, 2024-09-17T21:31:35.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020466314999999999, 2024-09-17T21:31:35.572 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.572 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.572 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.572 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.572 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.572 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.572 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57817 2024-09-17T21:31:35.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.572 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:35.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:35.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:35.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2399999999999999e-07 2024-09-17T21:31:35.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778163+0000", 2024-09-17T21:31:35.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.204e-06 2024-09-17T21:31:35.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778169+0000", 2024-09-17T21:31:35.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:31:35.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778173+0000", 2024-09-17T21:31:35.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:31:35.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798342+0000", 2024-09-17T21:31:35.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020168416000000002 2024-09-17T21:31:35.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798388+0000", 2024-09-17T21:31:35.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6171000000000002e-05 2024-09-17T21:31:35.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798627+0000", 2024-09-17T21:31:35.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000239058 2024-09-17T21:31:35.577 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.577 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.577 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.577 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57818 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2818048~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:35.577 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:31:35.577 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.721037300999996, 2024-09-17T21:31:35.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020767240999999999, 2024-09-17T21:31:35.577 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.577 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.577 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.578 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.578 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.578 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.578 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57818 2024-09-17T21:31:35.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.578 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:31:35.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:31:35.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778180+0000", 2024-09-17T21:31:35.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:31:35.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778182+0000", 2024-09-17T21:31:35.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7269999999999999e-06 2024-09-17T21:31:35.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778188+0000", 2024-09-17T21:31:35.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6080000000000001e-06 2024-09-17T21:31:35.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778202+0000", 2024-09-17T21:31:35.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3369e-05 2024-09-17T21:31:35.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798660+0000", 2024-09-17T21:31:35.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020458171000000001 2024-09-17T21:31:35.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798704+0000", 2024-09-17T21:31:35.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4273000000000001e-05 2024-09-17T21:31:35.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798947+0000", 2024-09-17T21:31:35.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242651 2024-09-17T21:31:35.582 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.582 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.582 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.582 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57819 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:35.582 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:31:35.582 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.721006431999996, 2024-09-17T21:31:35.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020892340999999998, 2024-09-17T21:31:35.582 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.583 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.583 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.583 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57819 2024-09-17T21:31:35.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.583 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:31:35.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:31:35.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778211+0000", 2024-09-17T21:31:35.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:31:35.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778217+0000", 2024-09-17T21:31:35.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T21:31:35.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778224+0000", 2024-09-17T21:31:35.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3520000000000003e-06 2024-09-17T21:31:35.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778228+0000", 2024-09-17T21:31:35.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7480000000000001e-06 2024-09-17T21:31:35.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798977+0000", 2024-09-17T21:31:35.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020748571 2024-09-17T21:31:35.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799035+0000", 2024-09-17T21:31:35.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7908999999999997e-05 2024-09-17T21:31:35.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799103+0000", 2024-09-17T21:31:35.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7794999999999997e-05 2024-09-17T21:31:35.587 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.587 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.587 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.587 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57820 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:35.587 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:35.588 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.72098124, 2024-09-17T21:31:35.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021016658000000001, 2024-09-17T21:31:35.588 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.588 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.588 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.588 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.588 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.588 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.588 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57820 2024-09-17T21:31:35.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.588 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:35.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:35.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:35.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:31:35.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778238+0000", 2024-09-17T21:31:35.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.945e-06 2024-09-17T21:31:35.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778244+0000", 2024-09-17T21:31:35.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6520000000000003e-06 2024-09-17T21:31:35.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778248+0000", 2024-09-17T21:31:35.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5920000000000002e-06 2024-09-17T21:31:35.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799131+0000", 2024-09-17T21:31:35.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020883288999999999 2024-09-17T21:31:35.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799185+0000", 2024-09-17T21:31:35.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3223000000000002e-05 2024-09-17T21:31:35.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799252+0000", 2024-09-17T21:31:35.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7576999999999994e-05 2024-09-17T21:31:35.592 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.592 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.592 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.593 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57821 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:35.593 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:35.593 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.720962895, 2024-09-17T21:31:35.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021153528000000001, 2024-09-17T21:31:35.593 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.593 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.593 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.593 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.593 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.593 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.594 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57821 2024-09-17T21:31:35.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.594 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:35.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:35.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:35.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-07 2024-09-17T21:31:35.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778256+0000", 2024-09-17T21:31:35.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.604e-06 2024-09-17T21:31:35.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778262+0000", 2024-09-17T21:31:35.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7629999999999997e-06 2024-09-17T21:31:35.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778266+0000", 2024-09-17T21:31:35.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:31:35.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799284+0000", 2024-09-17T21:31:35.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021017312 2024-09-17T21:31:35.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799326+0000", 2024-09-17T21:31:35.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2639000000000003e-05 2024-09-17T21:31:35.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799407+0000", 2024-09-17T21:31:35.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1077999999999999e-05 2024-09-17T21:31:35.597 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.598 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.598 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.598 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57822 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:35.598 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:31:35.598 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.720944584999998, 2024-09-17T21:31:35.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021268966, 2024-09-17T21:31:35.598 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.598 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.599 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.599 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.599 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.599 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.599 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57822 2024-09-17T21:31:35.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.599 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:31:35.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:31:35.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778273+0000", 2024-09-17T21:31:35.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:31:35.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778274+0000", 2024-09-17T21:31:35.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.598e-06 2024-09-17T21:31:35.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778283+0000", 2024-09-17T21:31:35.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8279999999999992e-06 2024-09-17T21:31:35.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778289+0000", 2024-09-17T21:31:35.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5609999999999998e-06 2024-09-17T21:31:35.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799432+0000", 2024-09-17T21:31:35.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021143862999999999 2024-09-17T21:31:35.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799473+0000", 2024-09-17T21:31:35.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1075999999999998e-05 2024-09-17T21:31:35.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799541+0000", 2024-09-17T21:31:35.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7657999999999995e-05 2024-09-17T21:31:35.603 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.603 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.603 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.603 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57823 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:35.603 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:35.604 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.720921623999999, 2024-09-17T21:31:35.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021379770999999999, 2024-09-17T21:31:35.604 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.604 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.604 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.604 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.604 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.604 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.604 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57823 2024-09-17T21:31:35.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.604 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:35.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:35.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:35.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:31:35.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778297+0000", 2024-09-17T21:31:35.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:31:35.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778303+0000", 2024-09-17T21:31:35.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4559999999999999e-06 2024-09-17T21:31:35.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778307+0000", 2024-09-17T21:31:35.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6229999999999996e-06 2024-09-17T21:31:35.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799560+0000", 2024-09-17T21:31:35.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021252931999999999 2024-09-17T21:31:35.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799609+0000", 2024-09-17T21:31:35.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8301999999999999e-05 2024-09-17T21:31:35.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799675+0000", 2024-09-17T21:31:35.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6205999999999996e-05 2024-09-17T21:31:35.608 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.608 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.609 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.609 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.609 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.609 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:35.610 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:31:35.611 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:35.608+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:35.711 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:35.709+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:35.723 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:35.723 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:35.723 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:35.723 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:35.732 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:31:35.812 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:35.810+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:35.834 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:35.835 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:35.835 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:35.835 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:35.835 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:35.843 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:31:35.914 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:35.912+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:35.948 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:35.948 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:35.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:35.948 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:35.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.948 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:49361 3.19 e46/25 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:head v 46'135, mlcod=45'133)", 2024-09-17T21:31:35.948 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:35.948 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.191492572000001, 2024-09-17T21:31:35.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043477744999999998, 2024-09-17T21:31:35.948 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.949 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.949 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:35.949 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:35.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:35.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:35.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-07 2024-09-17T21:31:35.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769703+0000", 2024-09-17T21:31:35.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016134381 2024-09-17T21:31:35.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769724+0000", 2024-09-17T21:31:35.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0781999999999999e-05 2024-09-17T21:31:35.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769734+0000", 2024-09-17T21:31:35.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6870000000000006e-06 2024-09-17T21:31:35.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769797+0000", 2024-09-17T21:31:35.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3418999999999997e-05 2024-09-17T21:31:35.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769822+0000", 2024-09-17T21:31:35.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4652999999999999e-05 2024-09-17T21:31:35.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:35.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.796982+0000", 2024-09-17T21:31:35.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027160205 2024-09-17T21:31:35.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.797047+0000", 2024-09-17T21:31:35.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4455999999999994e-05 2024-09-17T21:31:35.952 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.952 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.952 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.952 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50805 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 17825792~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.952 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:31:35.953 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.449909078000005, 2024-09-17T21:31:35.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035591579999999998, 2024-09-17T21:31:35.953 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.953 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.953 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.953 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.953 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.953 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.953 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50805 2024-09-17T21:31:35.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.953 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:31:35.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:31:35.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495153+0000", 2024-09-17T21:31:35.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:31:35.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495155+0000", 2024-09-17T21:31:35.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1849999999999998e-06 2024-09-17T21:31:35.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495162+0000", 2024-09-17T21:31:35.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.64e-06 2024-09-17T21:31:35.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495166+0000", 2024-09-17T21:31:35.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:31:35.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530020+0000", 2024-09-17T21:31:35.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.034853824999999998 2024-09-17T21:31:35.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530076+0000", 2024-09-17T21:31:35.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5235999999999998e-05 2024-09-17T21:31:35.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530744+0000", 2024-09-17T21:31:35.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066835500000000003 2024-09-17T21:31:35.956 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.956 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.956 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.957 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50806 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19398656~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.957 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:35.957 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.449888610000002, 2024-09-17T21:31:35.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036408459999999997, 2024-09-17T21:31:35.957 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.957 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.957 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.957 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.957 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.957 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.957 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50806 2024-09-17T21:31:35.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.957 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:35.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:35.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:35.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.72e-07 2024-09-17T21:31:35.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495175+0000", 2024-09-17T21:31:35.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:31:35.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495182+0000", 2024-09-17T21:31:35.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8079999999999999e-06 2024-09-17T21:31:35.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495186+0000", 2024-09-17T21:31:35.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5549999999999996e-06 2024-09-17T21:31:35.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530791+0000", 2024-09-17T21:31:35.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035604963000000003 2024-09-17T21:31:35.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530855+0000", 2024-09-17T21:31:35.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3286000000000004e-05 2024-09-17T21:31:35.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531581+0000", 2024-09-17T21:31:35.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00072655699999999996 2024-09-17T21:31:35.960 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.961 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.961 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.961 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50807 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23068672~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.961 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:35.961 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.449869724999999, 2024-09-17T21:31:35.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037183314000000002, 2024-09-17T21:31:35.961 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.961 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.961 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.961 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.961 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.961 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.962 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50807 2024-09-17T21:31:35.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.962 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:35.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:35.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:35.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9999999999999999e-07 2024-09-17T21:31:35.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495194+0000", 2024-09-17T21:31:35.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.779e-06 2024-09-17T21:31:35.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495199+0000", 2024-09-17T21:31:35.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.468e-06 2024-09-17T21:31:35.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495203+0000", 2024-09-17T21:31:35.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0010000000000003e-06 2024-09-17T21:31:35.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531636+0000", 2024-09-17T21:31:35.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036432665000000003 2024-09-17T21:31:35.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531727+0000", 2024-09-17T21:31:35.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1607999999999994e-05 2024-09-17T21:31:35.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532375+0000", 2024-09-17T21:31:35.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00064749299999999996 2024-09-17T21:31:35.965 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.965 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.965 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.965 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50808 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24641536~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.965 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:31:35.965 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.449852981999996, 2024-09-17T21:31:35.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037870190999999997, 2024-09-17T21:31:35.966 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.966 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.966 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.966 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.966 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.966 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.966 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50808 2024-09-17T21:31:35.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.966 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:31:35.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:31:35.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495209+0000", 2024-09-17T21:31:35.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:31:35.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495215+0000", 2024-09-17T21:31:35.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1630000000000001e-06 2024-09-17T21:31:35.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495221+0000", 2024-09-17T21:31:35.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-17T21:31:35.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495226+0000", 2024-09-17T21:31:35.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8189999999999999e-06 2024-09-17T21:31:35.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532425+0000", 2024-09-17T21:31:35.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037199135000000001 2024-09-17T21:31:35.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532531+0000", 2024-09-17T21:31:35.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106314 2024-09-17T21:31:35.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533079+0000", 2024-09-17T21:31:35.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00054739700000000003 2024-09-17T21:31:35.969 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.969 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.969 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.970 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50809 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28311552~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.970 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:31:35.970 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.449830016000007, 2024-09-17T21:31:35.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038597605, 2024-09-17T21:31:35.970 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.970 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.970 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.970 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.970 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.970 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.970 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50809 2024-09-17T21:31:35.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.971 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:31:35.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:31:35.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495232+0000", 2024-09-17T21:31:35.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.39e-07 2024-09-17T21:31:35.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495234+0000", 2024-09-17T21:31:35.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T21:31:35.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495239+0000", 2024-09-17T21:31:35.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7969999999999997e-06 2024-09-17T21:31:35.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495244+0000", 2024-09-17T21:31:35.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999996e-06 2024-09-17T21:31:35.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533125+0000", 2024-09-17T21:31:35.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037881523 2024-09-17T21:31:35.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533237+0000", 2024-09-17T21:31:35.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000111879 2024-09-17T21:31:35.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533829+0000", 2024-09-17T21:31:35.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000591933 2024-09-17T21:31:35.974 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.974 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.974 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.974 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50810 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 29884416~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.974 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:31:35.974 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.449812097999995, 2024-09-17T21:31:35.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039149809000000001, 2024-09-17T21:31:35.974 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.975 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.975 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.975 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.975 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.975 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.975 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50810 2024-09-17T21:31:35.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.975 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:31:35.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:31:35.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495250+0000", 2024-09-17T21:31:35.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:31:35.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495251+0000", 2024-09-17T21:31:35.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.81e-06 2024-09-17T21:31:35.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495257+0000", 2024-09-17T21:31:35.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1399999999999999e-06 2024-09-17T21:31:35.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495261+0000", 2024-09-17T21:31:35.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4229999999999998e-06 2024-09-17T21:31:35.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533879+0000", 2024-09-17T21:31:35.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038618292999999998 2024-09-17T21:31:35.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533939+0000", 2024-09-17T21:31:35.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.012e-05 2024-09-17T21:31:35.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534399+0000", 2024-09-17T21:31:35.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00045968099999999998 2024-09-17T21:31:35.979 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.979 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.979 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.979 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50850 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 2359296~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.979 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:35.979 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.449793722999999, 2024-09-17T21:31:35.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039731491000000001, 2024-09-17T21:31:35.979 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.979 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.979 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.979 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.980 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.980 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.980 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50850 2024-09-17T21:31:35.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.980 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:35.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:35.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:35.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3200000000000001e-07 2024-09-17T21:31:35.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495270+0000", 2024-09-17T21:31:35.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.694e-06 2024-09-17T21:31:35.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495275+0000", 2024-09-17T21:31:35.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5890000000000002e-06 2024-09-17T21:31:35.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495279+0000", 2024-09-17T21:31:35.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-17T21:31:35.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534440+0000", 2024-09-17T21:31:35.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039160805999999999 2024-09-17T21:31:35.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534526+0000", 2024-09-17T21:31:35.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5327999999999993e-05 2024-09-17T21:31:35.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534999+0000", 2024-09-17T21:31:35.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00047353400000000001 2024-09-17T21:31:35.983 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.983 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.983 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.984 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50851 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 3932160~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.984 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:31:35.984 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.449776111000006, 2024-09-17T21:31:35.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040211991000000002, 2024-09-17T21:31:35.984 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.984 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.984 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.984 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.984 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.984 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.984 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50851 2024-09-17T21:31:35.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.985 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:31:35.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:31:35.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495286+0000", 2024-09-17T21:31:35.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:31:35.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495287+0000", 2024-09-17T21:31:35.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6640000000000001e-06 2024-09-17T21:31:35.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495293+0000", 2024-09-17T21:31:35.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6710000000000004e-06 2024-09-17T21:31:35.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495298+0000", 2024-09-17T21:31:35.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6700000000000002e-06 2024-09-17T21:31:35.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535017+0000", 2024-09-17T21:31:35.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039719059000000001 2024-09-17T21:31:35.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535067+0000", 2024-09-17T21:31:35.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0046000000000002e-05 2024-09-17T21:31:35.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535497+0000", 2024-09-17T21:31:35.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043050299999999999 2024-09-17T21:31:35.988 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.988 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.988 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.988 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50852 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 7602176~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.988 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:35.988 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.449757194, 2024-09-17T21:31:35.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041166337999999997, 2024-09-17T21:31:35.988 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.988 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.988 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.989 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.989 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.989 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.989 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50852 2024-09-17T21:31:35.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.989 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:35.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:35.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495305+0000", 2024-09-17T21:31:35.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1600000000000002e-07 2024-09-17T21:31:35.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495307+0000", 2024-09-17T21:31:35.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0980000000000001e-06 2024-09-17T21:31:35.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495312+0000", 2024-09-17T21:31:35.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6210000000000001e-06 2024-09-17T21:31:35.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495317+0000", 2024-09-17T21:31:35.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T21:31:35.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535531+0000", 2024-09-17T21:31:35.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040214580999999999 2024-09-17T21:31:35.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535618+0000", 2024-09-17T21:31:35.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6648000000000001e-05 2024-09-17T21:31:35.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536471+0000", 2024-09-17T21:31:35.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00085251599999999999 2024-09-17T21:31:35.992 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.992 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.992 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.992 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50853 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 9175040~204800] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.993 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:35.993 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.449738862999993, 2024-09-17T21:31:35.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041673496999999997, 2024-09-17T21:31:35.993 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.993 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.993 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.993 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.993 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.993 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.993 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50853 2024-09-17T21:31:35.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.993 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:35.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:35.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:35.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:31:35.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495334+0000", 2024-09-17T21:31:35.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.076e-05 2024-09-17T21:31:35.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495340+0000", 2024-09-17T21:31:35.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:31:35.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:35.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495345+0000", 2024-09-17T21:31:35.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:31:35.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:35.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536514+0000", 2024-09-17T21:31:35.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041168534999999999 2024-09-17T21:31:35.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:35.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536572+0000", 2024-09-17T21:31:35.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8742000000000001e-05 2024-09-17T21:31:35.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:35.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536996+0000", 2024-09-17T21:31:35.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042368899999999999 2024-09-17T21:31:35.996 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.997 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:35.997 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:35.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.997 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50854 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 12845056~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:35.997 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:35.997 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.449709803999994, 2024-09-17T21:31:35.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042777397000000002, 2024-09-17T21:31:35.997 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:35.997 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:35.997 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:35.997 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:35.997 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:35.998 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:35.998 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50854 2024-09-17T21:31:35.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.998 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:35.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:35.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:35.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:35.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:35.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:35.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:35.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:35.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.82e-07 2024-09-17T21:31:35.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:35.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495354+0000", 2024-09-17T21:31:35.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.091e-06 2024-09-17T21:31:35.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:35.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:35.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:35.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495360+0000", 2024-09-17T21:31:35.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.716e-06 2024-09-17T21:31:36.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495365+0000", 2024-09-17T21:31:36.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2939999999999996e-06 2024-09-17T21:31:36.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:36.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537020+0000", 2024-09-17T21:31:36.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041654419999999998 2024-09-17T21:31:36.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:36.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537105+0000", 2024-09-17T21:31:36.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5857000000000001e-05 2024-09-17T21:31:36.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:36.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538129+0000", 2024-09-17T21:31:36.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010235369999999999 2024-09-17T21:31:36.001 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.001 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:36.001 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.001 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50855 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 14417920~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:36.001 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:36.001 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.449689839000001, 2024-09-17T21:31:36.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043254325000000003, 2024-09-17T21:31:36.002 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:36.002 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:36.002 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:36.002 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:36.002 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:36.002 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.002 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50855 2024-09-17T21:31:36.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.002 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:36.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:36.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:36.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:36.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:36.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:36.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:36.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:36.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:36.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:31:36.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:36.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495374+0000", 2024-09-17T21:31:36.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.922e-06 2024-09-17T21:31:36.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:36.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495379+0000", 2024-09-17T21:31:36.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-17T21:31:36.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495384+0000", 2024-09-17T21:31:36.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-17T21:31:36.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:36.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538167+0000", 2024-09-17T21:31:36.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042782845 2024-09-17T21:31:36.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:36.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538218+0000", 2024-09-17T21:31:36.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1335000000000002e-05 2024-09-17T21:31:36.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:36.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538626+0000", 2024-09-17T21:31:36.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000407656 2024-09-17T21:31:36.005 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.005 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:36.005 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.006 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50856 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 18087936~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:36.006 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:36.006 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.449671383999998, 2024-09-17T21:31:36.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043911444000000001, 2024-09-17T21:31:36.006 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:36.006 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:36.006 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:36.006 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:36.006 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:36.006 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.006 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50856 2024-09-17T21:31:36.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.007 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:36.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:36.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:36.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:36.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:36.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:36.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:36.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:36.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:36.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0499999999999999e-07 2024-09-17T21:31:36.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:36.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495392+0000", 2024-09-17T21:31:36.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7430000000000001e-06 2024-09-17T21:31:36.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:36.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495398+0000", 2024-09-17T21:31:36.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3040000000000001e-06 2024-09-17T21:31:36.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495403+0000", 2024-09-17T21:31:36.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5600000000000004e-06 2024-09-17T21:31:36.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:36.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538660+0000", 2024-09-17T21:31:36.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043256832000000002 2024-09-17T21:31:36.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:36.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538746+0000", 2024-09-17T21:31:36.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6448999999999999e-05 2024-09-17T21:31:36.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:36.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539301+0000", 2024-09-17T21:31:36.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055525099999999999 2024-09-17T21:31:36.010 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.010 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:36.010 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.010 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50857 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19660800~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:36.010 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:36.010 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.449651978999995, 2024-09-17T21:31:36.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044408156999999997, 2024-09-17T21:31:36.010 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:36.011 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:36.011 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:36.011 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:36.011 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:36.011 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.011 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50857 2024-09-17T21:31:36.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.011 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:36.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:36.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:36.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:36.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:36.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:36.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:36.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:36.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495410+0000", 2024-09-17T21:31:36.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:31:36.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:36.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495412+0000", 2024-09-17T21:31:36.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9580000000000001e-06 2024-09-17T21:31:36.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:36.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495418+0000", 2024-09-17T21:31:36.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1909999999999997e-06 2024-09-17T21:31:36.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495423+0000", 2024-09-17T21:31:36.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2429999999999996e-06 2024-09-17T21:31:36.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:36.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539338+0000", 2024-09-17T21:31:36.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043914478 2024-09-17T21:31:36.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:36.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539393+0000", 2024-09-17T21:31:36.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5226999999999998e-05 2024-09-17T21:31:36.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:36.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539818+0000", 2024-09-17T21:31:36.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042469300000000002 2024-09-17T21:31:36.014 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.014 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:36.015 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.015 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50858 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23330816~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:36.015 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:36.015 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.449630881999994, 2024-09-17T21:31:36.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045010798999999997, 2024-09-17T21:31:36.015 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:36.015 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:36.015 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:36.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:36.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:36.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.015 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50858 2024-09-17T21:31:36.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.016 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:36.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:36.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:36.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:36.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:36.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:36.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:36.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:36.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:36.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:31:36.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:36.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495433+0000", 2024-09-17T21:31:36.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.787e-06 2024-09-17T21:31:36.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:36.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495438+0000", 2024-09-17T21:31:36.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:31:36.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495442+0000", 2024-09-17T21:31:36.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2640000000000002e-06 2024-09-17T21:31:36.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:36.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539852+0000", 2024-09-17T21:31:36.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044409592999999997 2024-09-17T21:31:36.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:36.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539940+0000", 2024-09-17T21:31:36.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7696999999999997e-05 2024-09-17T21:31:36.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:36.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540441+0000", 2024-09-17T21:31:36.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050164299999999997 2024-09-17T21:31:36.019 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.019 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:36.019 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.019 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50859 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24903680~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:36.019 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:36.019 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.449613204000002, 2024-09-17T21:31:36.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045571569999999999, 2024-09-17T21:31:36.020 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:36.020 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:36.020 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:36.020 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:36.020 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:36.020 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.020 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50859 2024-09-17T21:31:36.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.020 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:36.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:36.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:36.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:36.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:36.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:36.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:36.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:36.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495449+0000", 2024-09-17T21:31:36.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:36.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:36.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495450+0000", 2024-09-17T21:31:36.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7889999999999999e-06 2024-09-17T21:31:36.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:36.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495456+0000", 2024-09-17T21:31:36.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5679999999999999e-06 2024-09-17T21:31:36.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495460+0000", 2024-09-17T21:31:36.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4989999999999996e-06 2024-09-17T21:31:36.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:36.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540472+0000", 2024-09-17T21:31:36.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045011585999999999 2024-09-17T21:31:36.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:36.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540554+0000", 2024-09-17T21:31:36.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2443999999999999e-05 2024-09-17T21:31:36.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:36.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541020+0000", 2024-09-17T21:31:36.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00046534300000000001 2024-09-17T21:31:36.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.023 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:36.024 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.024 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50860 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28573696~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:36.024 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:36.024 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.449594851000001, 2024-09-17T21:31:36.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046167899999999998, 2024-09-17T21:31:36.024 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:36.024 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:36.024 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:36.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:36.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:36.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.024 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50860 2024-09-17T21:31:36.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.025 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:36.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:36.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:36.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:36.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:36.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:36.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:36.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:36.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:36.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:31:36.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:36.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495469+0000", 2024-09-17T21:31:36.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.99e-06 2024-09-17T21:31:36.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:36.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495475+0000", 2024-09-17T21:31:36.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:31:36.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495480+0000", 2024-09-17T21:31:36.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:31:36.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:36.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541050+0000", 2024-09-17T21:31:36.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045569853 2024-09-17T21:31:36.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:36.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541138+0000", 2024-09-17T21:31:36.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7997000000000005e-05 2024-09-17T21:31:36.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:36.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541634+0000", 2024-09-17T21:31:36.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00049685500000000004 2024-09-17T21:31:36.032 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.032 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:36.032 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.032 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50861 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 30146560~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:36.032 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:36.032 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 84.449575674000002, 2024-09-17T21:31:36.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046690553000000003, 2024-09-17T21:31:36.032 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:36.032 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:36.032 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:36.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:36.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:36.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.033 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50861 2024-09-17T21:31:36.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.033 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:36.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:36.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:36.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:36.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:36.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:36.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:36.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:36.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:36.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:31:36.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:36.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495488+0000", 2024-09-17T21:31:36.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:31:36.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:36.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495495+0000", 2024-09-17T21:31:36.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:31:36.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495500+0000", 2024-09-17T21:31:36.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8620000000000002e-06 2024-09-17T21:31:36.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:36.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541687+0000", 2024-09-17T21:31:36.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046186434999999998 2024-09-17T21:31:36.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:36.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541741+0000", 2024-09-17T21:31:36.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4228999999999999e-05 2024-09-17T21:31:36.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:36.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.542176+0000", 2024-09-17T21:31:36.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043531100000000002 2024-09-17T21:31:36.036 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.036 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:36.036 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.037 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97804 3.7 3:e725aa05:::rbd_data.1120c318df8c.0000000000000003:24 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:36.037 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:36.037 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.983629573000002, 2024-09-17T21:31:36.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035514416, 2024-09-17T21:31:36.037 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:36.037 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:36.037 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:36.037 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:36.037 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:36.037 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.037 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97804 2024-09-17T21:31:36.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.037 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:36.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:36.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:36.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:36.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:36.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:36.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:36.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:36.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:36.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:31:36.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:36.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961434+0000", 2024-09-17T21:31:36.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8959999999999999e-06 2024-09-17T21:31:36.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:36.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961439+0000", 2024-09-17T21:31:36.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T21:31:36.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961444+0000", 2024-09-17T21:31:36.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4970000000000003e-06 2024-09-17T21:31:36.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:36.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996466+0000", 2024-09-17T21:31:36.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035022438000000003 2024-09-17T21:31:36.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:36.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996556+0000", 2024-09-17T21:31:36.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9561000000000005e-05 2024-09-17T21:31:36.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:36.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:36.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:36.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996946+0000", 2024-09-17T21:31:36.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039028700000000001 2024-09-17T21:31:36.041 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.041 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:36.041 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.041 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:36.041 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:36.041 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:36.043 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:31:36.043 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:36.040+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:36.144 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:36.141+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:36.155 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:36.155 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:36.155 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:36.155 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:36.164 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:31:36.248 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:36.246+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:36.307 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:36.307 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:36.307 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:36.308 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:36.308 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:36.316 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:31:36.345 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:36.343+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:36.442 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:31:36.446 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:36.444+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:36.462 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:36.462 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:36.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:36.462 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:36.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.462 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21342 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2082816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:31:36.462 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:36.462 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.7824651, 2024-09-17T21:31:36.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022083523000000001, 2024-09-17T21:31:36.463 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.463 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.463 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:36.463 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:36.463 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:36.463 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.463 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21342 2024-09-17T21:31:36.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.463 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:36.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:36.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:36.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:31:36.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676659+0000", 2024-09-17T21:31:36.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0950000000000001e-06 2024-09-17T21:31:36.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676662+0000", 2024-09-17T21:31:36.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.971e-06 2024-09-17T21:31:36.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676665+0000", 2024-09-17T21:31:36.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6189999999999998e-06 2024-09-17T21:31:36.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698568+0000", 2024-09-17T21:31:36.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021903549000000001 2024-09-17T21:31:36.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698668+0000", 2024-09-17T21:31:36.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9778999999999995e-05 2024-09-17T21:31:36.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698741+0000", 2024-09-17T21:31:36.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000006e-05 2024-09-17T21:31:36.466 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.466 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.467 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.467 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21343 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2095104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:31:36.467 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:36.467 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.78245516200001, 2024-09-17T21:31:36.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022504024000000001, 2024-09-17T21:31:36.467 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.467 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.467 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:36.467 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:36.467 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:36.467 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.467 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21343 2024-09-17T21:31:36.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.468 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:36.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:36.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:36.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:31:36.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676669+0000", 2024-09-17T21:31:36.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.003e-06 2024-09-17T21:31:36.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676672+0000", 2024-09-17T21:31:36.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9280000000000002e-06 2024-09-17T21:31:36.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676674+0000", 2024-09-17T21:31:36.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5469999999999999e-06 2024-09-17T21:31:36.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699015+0000", 2024-09-17T21:31:36.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022340563000000001 2024-09-17T21:31:36.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699102+0000", 2024-09-17T21:31:36.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7425999999999999e-05 2024-09-17T21:31:36.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699172+0000", 2024-09-17T21:31:36.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9346999999999999e-05 2024-09-17T21:31:36.471 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.471 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.471 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.471 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45576 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:36.471 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:36.471 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 89.754221361999996, 2024-09-17T21:31:36.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022024660000000001, 2024-09-17T21:31:36.472 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.472 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.472 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:36.472 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:36.472 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:36.472 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.472 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45576 2024-09-17T21:31:36.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.472 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:36.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:36.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704902+0000", 2024-09-17T21:31:36.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9100000000000004e-07 2024-09-17T21:31:36.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704905+0000", 2024-09-17T21:31:36.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6139999999999999e-06 2024-09-17T21:31:36.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704911+0000", 2024-09-17T21:31:36.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8619999999999997e-06 2024-09-17T21:31:36.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704917+0000", 2024-09-17T21:31:36.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:31:36.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726681+0000", 2024-09-17T21:31:36.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021764569000000001 2024-09-17T21:31:36.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726726+0000", 2024-09-17T21:31:36.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4218999999999998e-05 2024-09-17T21:31:36.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726926+0000", 2024-09-17T21:31:36.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000200466 2024-09-17T21:31:36.475 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.475 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.475 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.475 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45577 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:36.476 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:36.476 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 89.754197705999999, 2024-09-17T21:31:36.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022269694999999999, 2024-09-17T21:31:36.476 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.476 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.476 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:36.476 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:36.476 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:36.476 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.476 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45577 2024-09-17T21:31:36.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.476 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:36.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:36.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704926+0000", 2024-09-17T21:31:36.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:31:36.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704928+0000", 2024-09-17T21:31:36.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9989999999999998e-06 2024-09-17T21:31:36.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704934+0000", 2024-09-17T21:31:36.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:31:36.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704939+0000", 2024-09-17T21:31:36.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5909999999999998e-06 2024-09-17T21:31:36.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726954+0000", 2024-09-17T21:31:36.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022014885000000001 2024-09-17T21:31:36.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726995+0000", 2024-09-17T21:31:36.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1474000000000003e-05 2024-09-17T21:31:36.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727195+0000", 2024-09-17T21:31:36.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019968300000000001 2024-09-17T21:31:36.480 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.480 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.480 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.480 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45578 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:36.480 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:36.481 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 89.754176462000004, 2024-09-17T21:31:36.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022484006000000001, 2024-09-17T21:31:36.481 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.481 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.481 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:36.481 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:36.481 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:36.481 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.481 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45578 2024-09-17T21:31:36.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.481 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:36.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:36.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704947+0000", 2024-09-17T21:31:36.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1399999999999997e-07 2024-09-17T21:31:36.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704949+0000", 2024-09-17T21:31:36.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0959999999999999e-06 2024-09-17T21:31:36.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704956+0000", 2024-09-17T21:31:36.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9330000000000004e-06 2024-09-17T21:31:36.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704961+0000", 2024-09-17T21:31:36.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6759999999999998e-06 2024-09-17T21:31:36.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727213+0000", 2024-09-17T21:31:36.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022252568 2024-09-17T21:31:36.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727259+0000", 2024-09-17T21:31:36.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5238000000000003e-05 2024-09-17T21:31:36.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727430+0000", 2024-09-17T21:31:36.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000171881 2024-09-17T21:31:36.484 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.485 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.485 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.485 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45579 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:36.485 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:36.485 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 89.754154533999994, 2024-09-17T21:31:36.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022749080000000001, 2024-09-17T21:31:36.485 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.485 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.485 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:36.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:36.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:36.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.486 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45579 2024-09-17T21:31:36.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.486 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:36.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:36.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704969+0000", 2024-09-17T21:31:36.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:31:36.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704971+0000", 2024-09-17T21:31:36.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.097e-06 2024-09-17T21:31:36.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704977+0000", 2024-09-17T21:31:36.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6329999999999999e-06 2024-09-17T21:31:36.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704983+0000", 2024-09-17T21:31:36.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:31:36.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727457+0000", 2024-09-17T21:31:36.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022474521000000001 2024-09-17T21:31:36.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727498+0000", 2024-09-17T21:31:36.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0558e-05 2024-09-17T21:31:36.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727717+0000", 2024-09-17T21:31:36.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219805 2024-09-17T21:31:36.489 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.489 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.489 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.489 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45652 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:36.490 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:36.490 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 89.753008395999998, 2024-09-17T21:31:36.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022100465999999999, 2024-09-17T21:31:36.490 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.490 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.490 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:36.490 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:36.490 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:36.490 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.490 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45652 2024-09-17T21:31:36.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.490 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:36.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:36.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706115+0000", 2024-09-17T21:31:36.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:31:36.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706117+0000", 2024-09-17T21:31:36.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-06 2024-09-17T21:31:36.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706122+0000", 2024-09-17T21:31:36.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.288e-06 2024-09-17T21:31:36.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706136+0000", 2024-09-17T21:31:36.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3409e-05 2024-09-17T21:31:36.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728112+0000", 2024-09-17T21:31:36.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021976724999999999 2024-09-17T21:31:36.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728151+0000", 2024-09-17T21:31:36.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8318999999999997e-05 2024-09-17T21:31:36.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728215+0000", 2024-09-17T21:31:36.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4172999999999997e-05 2024-09-17T21:31:36.494 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.494 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.494 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.494 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45653 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:36.494 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:36.494 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 89.752979186000005, 2024-09-17T21:31:36.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022249517999999999, 2024-09-17T21:31:36.494 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.494 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.494 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:36.495 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:36.495 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:36.495 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.495 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45653 2024-09-17T21:31:36.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.495 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:36.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:36.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:36.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T21:31:36.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706147+0000", 2024-09-17T21:31:36.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4430000000000002e-06 2024-09-17T21:31:36.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706152+0000", 2024-09-17T21:31:36.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5350000000000004e-06 2024-09-17T21:31:36.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706157+0000", 2024-09-17T21:31:36.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8420000000000004e-06 2024-09-17T21:31:36.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728246+0000", 2024-09-17T21:31:36.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022089237000000001 2024-09-17T21:31:36.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728309+0000", 2024-09-17T21:31:36.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2356999999999999e-05 2024-09-17T21:31:36.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:31:36.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4596000000000001e-05 2024-09-17T21:31:36.498 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.498 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.498 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.498 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45657 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 36864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:36.499 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:36.499 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 89.752947922000004, 2024-09-17T21:31:36.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022218252000000001, 2024-09-17T21:31:36.499 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.499 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.499 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:36.499 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:36.499 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:36.499 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.499 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45657 2024-09-17T21:31:36.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.499 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:36.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:36.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:36.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8299999999999997e-07 2024-09-17T21:31:36.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706178+0000", 2024-09-17T21:31:36.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3700000000000002e-06 2024-09-17T21:31:36.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706184+0000", 2024-09-17T21:31:36.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.144e-06 2024-09-17T21:31:36.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706190+0000", 2024-09-17T21:31:36.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8590000000000001e-06 2024-09-17T21:31:36.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728240+0000", 2024-09-17T21:31:36.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022049804999999999 2024-09-17T21:31:36.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728307+0000", 2024-09-17T21:31:36.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7477999999999993e-05 2024-09-17T21:31:36.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:31:36.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6112999999999994e-05 2024-09-17T21:31:36.503 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.503 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.503 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.504 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45658 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:36.504 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:36.504 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 89.752924801999995, 2024-09-17T21:31:36.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022376455, 2024-09-17T21:31:36.504 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.504 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.504 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:36.504 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:36.504 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:36.504 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.504 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45658 2024-09-17T21:31:36.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.505 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:36.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:36.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:36.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:31:36.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706200+0000", 2024-09-17T21:31:36.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:31:36.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706206+0000", 2024-09-17T21:31:36.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7039999999999999e-06 2024-09-17T21:31:36.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706211+0000", 2024-09-17T21:31:36.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9429999999999999e-06 2024-09-17T21:31:36.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728418+0000", 2024-09-17T21:31:36.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022206760999999998 2024-09-17T21:31:36.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728497+0000", 2024-09-17T21:31:36.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9493999999999995e-05 2024-09-17T21:31:36.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728574+0000", 2024-09-17T21:31:36.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7180000000000003e-05 2024-09-17T21:31:36.508 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.508 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.508 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.508 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45707 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 212992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:36.508 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:36.508 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 89.751220735999993, 2024-09-17T21:31:36.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022235550999999999, 2024-09-17T21:31:36.509 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.509 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.509 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:36.509 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:36.509 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:36.509 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.509 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45707 2024-09-17T21:31:36.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.509 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:36.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:36.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:36.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:31:36.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707905+0000", 2024-09-17T21:31:36.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2120000000000002e-06 2024-09-17T21:31:36.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707911+0000", 2024-09-17T21:31:36.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1929999999999998e-06 2024-09-17T21:31:36.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707915+0000", 2024-09-17T21:31:36.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:31:36.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729897+0000", 2024-09-17T21:31:36.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021981186 2024-09-17T21:31:36.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729944+0000", 2024-09-17T21:31:36.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571000000000002e-05 2024-09-17T21:31:36.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730138+0000", 2024-09-17T21:31:36.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000193451 2024-09-17T21:31:36.513 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.513 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.513 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.513 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45708 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:36.513 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:36.513 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 89.751200604000005, 2024-09-17T21:31:36.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022533742999999998, 2024-09-17T21:31:36.513 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.513 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.513 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:36.513 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:36.513 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:36.514 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.514 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45708 2024-09-17T21:31:36.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.514 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:36.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:36.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707923+0000", 2024-09-17T21:31:36.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:31:36.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707932+0000", 2024-09-17T21:31:36.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6430000000000007e-06 2024-09-17T21:31:36.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707939+0000", 2024-09-17T21:31:36.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3539999999999996e-06 2024-09-17T21:31:36.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707944+0000", 2024-09-17T21:31:36.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3249999999999998e-06 2024-09-17T21:31:36.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730176+0000", 2024-09-17T21:31:36.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022232393 2024-09-17T21:31:36.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730263+0000", 2024-09-17T21:31:36.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6638999999999995e-05 2024-09-17T21:31:36.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730456+0000", 2024-09-17T21:31:36.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019304899999999999 2024-09-17T21:31:36.517 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.517 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.517 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.517 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45752 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:36.518 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:36.518 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 89.750945384000005, 2024-09-17T21:31:36.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022426878000000001, 2024-09-17T21:31:36.518 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.518 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.518 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:36.518 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:36.518 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:36.518 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.518 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45752 2024-09-17T21:31:36.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.518 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:36.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:36.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708178+0000", 2024-09-17T21:31:36.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:31:36.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708181+0000", 2024-09-17T21:31:36.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.909e-06 2024-09-17T21:31:36.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708187+0000", 2024-09-17T21:31:36.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4999999999999996e-06 2024-09-17T21:31:36.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708193+0000", 2024-09-17T21:31:36.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:31:36.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730484+0000", 2024-09-17T21:31:36.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022291765000000002 2024-09-17T21:31:36.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730541+0000", 2024-09-17T21:31:36.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6483000000000001e-05 2024-09-17T21:31:36.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730604+0000", 2024-09-17T21:31:36.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3429000000000005e-05 2024-09-17T21:31:36.521 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.522 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.522 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.522 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45761 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:36.522 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:36.522 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 89.750917474000005, 2024-09-17T21:31:36.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022614408999999999, 2024-09-17T21:31:36.522 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.522 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.522 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:36.522 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:36.522 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:36.522 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.523 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45761 2024-09-17T21:31:36.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.523 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:36.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:36.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708206+0000", 2024-09-17T21:31:36.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4700000000000001e-07 2024-09-17T21:31:36.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708208+0000", 2024-09-17T21:31:36.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1509999999999998e-06 2024-09-17T21:31:36.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708214+0000", 2024-09-17T21:31:36.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.49e-06 2024-09-17T21:31:36.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708219+0000", 2024-09-17T21:31:36.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8119999999999998e-06 2024-09-17T21:31:36.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730641+0000", 2024-09-17T21:31:36.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022422765000000001 2024-09-17T21:31:36.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730731+0000", 2024-09-17T21:31:36.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9598999999999994e-05 2024-09-17T21:31:36.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730820+0000", 2024-09-17T21:31:36.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8844999999999994e-05 2024-09-17T21:31:36.526 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.526 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.526 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.526 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45785 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:36.526 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:36.526 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 89.749695686999999, 2024-09-17T21:31:36.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022044113000000001, 2024-09-17T21:31:36.527 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.527 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.527 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:36.527 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:36.527 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:36.527 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.527 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45785 2024-09-17T21:31:36.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.527 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:36.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:36.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709428+0000", 2024-09-17T21:31:36.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:31:36.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709430+0000", 2024-09-17T21:31:36.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:31:36.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709451+0000", 2024-09-17T21:31:36.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1372000000000001e-05 2024-09-17T21:31:36.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709472+0000", 2024-09-17T21:31:36.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1007999999999999e-05 2024-09-17T21:31:36.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731293+0000", 2024-09-17T21:31:36.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021821320000000002 2024-09-17T21:31:36.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731378+0000", 2024-09-17T21:31:36.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5059e-05 2024-09-17T21:31:36.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731471+0000", 2024-09-17T21:31:36.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2867000000000004e-05 2024-09-17T21:31:36.530 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.530 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.530 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.531 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45786 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:36.531 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:36.531 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 89.749636890000005, 2024-09-17T21:31:36.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022181821000000001, 2024-09-17T21:31:36.531 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.531 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.531 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:36.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:36.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:36.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.531 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45786 2024-09-17T21:31:36.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.532 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:36.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:36.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709487+0000", 2024-09-17T21:31:36.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1299999999999999e-07 2024-09-17T21:31:36.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709491+0000", 2024-09-17T21:31:36.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2089999999999999e-06 2024-09-17T21:31:36.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709498+0000", 2024-09-17T21:31:36.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3980000000000002e-06 2024-09-17T21:31:36.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709504+0000", 2024-09-17T21:31:36.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7219999999999996e-06 2024-09-17T21:31:36.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731499+0000", 2024-09-17T21:31:36.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021995297 2024-09-17T21:31:36.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731591+0000", 2024-09-17T21:31:36.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1373999999999996e-05 2024-09-17T21:31:36.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731668+0000", 2024-09-17T21:31:36.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7107999999999994e-05 2024-09-17T21:31:36.535 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.535 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.535 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.535 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50202 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'139, mlcod=45'137)", 2024-09-17T21:31:36.535 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:36.535 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 85.334901020999993, 2024-09-17T21:31:36.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059659441000000001, 2024-09-17T21:31:36.536 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.536 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.536 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:36.536 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:36.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:36.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124224+0000", 2024-09-17T21:31:36.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5000000000000002e-06 2024-09-17T21:31:36.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147385+0000", 2024-09-17T21:31:36.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023160544000000002 2024-09-17T21:31:36.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147411+0000", 2024-09-17T21:31:36.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6095999999999999e-05 2024-09-17T21:31:36.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147438+0000", 2024-09-17T21:31:36.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6982999999999999e-05 2024-09-17T21:31:36.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147504+0000", 2024-09-17T21:31:36.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6156999999999997e-05 2024-09-17T21:31:36.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147531+0000", 2024-09-17T21:31:36.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6741999999999999e-05 2024-09-17T21:31:36.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:36.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183820+0000", 2024-09-17T21:31:36.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036289090000000003 2024-09-17T21:31:36.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183881+0000", 2024-09-17T21:31:36.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1328999999999995e-05 2024-09-17T21:31:36.539 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.539 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.539 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.540 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50203 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'141, mlcod=45'137)", 2024-09-17T21:31:36.540 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:36.540 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 85.297638277999994, 2024-09-17T21:31:36.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023138823999999999, 2024-09-17T21:31:36.540 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.540 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.540 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:36.540 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:36.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:36.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161486+0000", 2024-09-17T21:31:36.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5290000000000001e-06 2024-09-17T21:31:36.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161496+0000", 2024-09-17T21:31:36.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0063e-05 2024-09-17T21:31:36.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161514+0000", 2024-09-17T21:31:36.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7728e-05 2024-09-17T21:31:36.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161524+0000", 2024-09-17T21:31:36.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0339e-05 2024-09-17T21:31:36.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170872+0000", 2024-09-17T21:31:36.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093473640000000004 2024-09-17T21:31:36.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170899+0000", 2024-09-17T21:31:36.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7321000000000002e-05 2024-09-17T21:31:36.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:36.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184580+0000", 2024-09-17T21:31:36.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013680988999999999 2024-09-17T21:31:36.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184623+0000", 2024-09-17T21:31:36.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3491000000000001e-05 2024-09-17T21:31:36.544 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.544 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.544 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.544 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:88193 3.9 3:97498da5:::rbd_data.1120ea662e44.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-17T21:31:36.544 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:36.544 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.135457367999997, 2024-09-17T21:31:36.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026278261000000001, 2024-09-17T21:31:36.544 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.544 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.544 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:36.545 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:36.545 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:36.545 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.545 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 88193 2024-09-17T21:31:36.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.545 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:36.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:36.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323666+0000", 2024-09-17T21:31:36.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:31:36.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323670+0000", 2024-09-17T21:31:36.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.771e-06 2024-09-17T21:31:36.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323680+0000", 2024-09-17T21:31:36.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9469999999999998e-06 2024-09-17T21:31:36.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323685+0000", 2024-09-17T21:31:36.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9919999999999998e-06 2024-09-17T21:31:36.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323728+0000", 2024-09-17T21:31:36.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2811000000000001e-05 2024-09-17T21:31:36.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323776+0000", 2024-09-17T21:31:36.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8310000000000003e-05 2024-09-17T21:31:36.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:36.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.328583+0000", 2024-09-17T21:31:36.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048072899999999997 2024-09-17T21:31:36.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:36.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349861+0000", 2024-09-17T21:31:36.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021277695999999999 2024-09-17T21:31:36.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:36.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349877+0000", 2024-09-17T21:31:36.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:31:36.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:36.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349911+0000", 2024-09-17T21:31:36.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3844999999999999e-05 2024-09-17T21:31:36.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349944+0000", 2024-09-17T21:31:36.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2663000000000002e-05 2024-09-17T21:31:36.550 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.550 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.550 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.550 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:101384 3.9 3:90f2a32a:::rbd_data.11206b8c0882.0000000000000178:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T21:31:36.550 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:36.550 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 48.132587768999997, 2024-09-17T21:31:36.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025286194000000001, 2024-09-17T21:31:36.551 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:36.551 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:36.551 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:36.551 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:36.551 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:36.551 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:36.551 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 101384 2024-09-17T21:31:36.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.551 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:36.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:36.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:36.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:36.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:36.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:36.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:36.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326536+0000", 2024-09-17T21:31:36.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4799999999999997e-07 2024-09-17T21:31:36.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:36.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326541+0000", 2024-09-17T21:31:36.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8489999999999997e-06 2024-09-17T21:31:36.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:36.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326552+0000", 2024-09-17T21:31:36.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0912e-05 2024-09-17T21:31:36.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:36.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326557+0000", 2024-09-17T21:31:36.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0359999999999997e-06 2024-09-17T21:31:36.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:36.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326603+0000", 2024-09-17T21:31:36.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6428999999999999e-05 2024-09-17T21:31:36.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:36.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326686+0000", 2024-09-17T21:31:36.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2521999999999994e-05 2024-09-17T21:31:36.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:36.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.332698+0000", 2024-09-17T21:31:36.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060124599999999999 2024-09-17T21:31:36.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:36.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351744+0000", 2024-09-17T21:31:36.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019045838999999998 2024-09-17T21:31:36.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:36.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351754+0000", 2024-09-17T21:31:36.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0380999999999999e-05 2024-09-17T21:31:36.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:36.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351791+0000", 2024-09-17T21:31:36.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6678999999999999e-05 2024-09-17T21:31:36.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:36.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:36.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:36.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351821+0000", 2024-09-17T21:31:36.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0139000000000001e-05 2024-09-17T21:31:36.556 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.556 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.556 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.556 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:36.556 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:36.556 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:36.557 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:31:36.558 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:36.555+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:36.565 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:36.565 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:36.584 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:31:36.658 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:36.656+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:36.709 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:36.709 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:36.709 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:36.709 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:36.718 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:31:36.759 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:36.757+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:36.784 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:36.784 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:36.799 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:36.800 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:36.815 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:36.816 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:36.832 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:36.832 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:36.848 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:36.848 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:36.860 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:36.858+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:36.863 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:36.863 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:36.864 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:36.864 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:36.872 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:31:36.880 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:36.880 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:36.961 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:36.959+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:37.017 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:37.017 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:37.026 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:31:37.062 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:37.060+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:37.130 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:37.130 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:37.139 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:31:37.183 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:37.181+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:37.242 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:37.243 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:37.254 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:31:37.295 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:37.293+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:37.365 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:37.362+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:37.398 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:37.398 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:37.406 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:31:37.411 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is unset 2024-09-17T21:31:37.468 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:37.466+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:37.558 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:37.559 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:37.567 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:31:37.568 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:37.565+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:37.668 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:37.665+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:37.710 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:37.711 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:37.719 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:31:37.769 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:37.767+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:37.823 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:37.823 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:37.831 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:31:37.875 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:37.873+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:37.936 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:37.936 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:37.944 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:31:37.985 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:37.983+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:38.071 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:38.069+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:38.086 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:38.086 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:38.095 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:31:38.172 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:38.169+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:38.197 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:38.197 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:38.207 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:31:38.273 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:38.271+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:38.350 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:38.350 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:38.360 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:31:38.403 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:38.401+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:38.472 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:38.472 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:38.475 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:38.472+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:38.481 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:31:38.573 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-17T21:31:38.573 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:31:38.573 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-17T21:31:38.574 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T21:31:38.574 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T21:31:38.574 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-17T21:31:38.575 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:38.573+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:38.629 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:38.629 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:38.638 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:31:38.680 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:38.680 INFO:teuthology.orchestra.run.smithi171.stdout: "success": "" 2024-09-17T21:31:38.680 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:38.683 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:38.681+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:38.743 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:38.743 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:38.752 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:31:38.777 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:38.775+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:38.878 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:38.876+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:38.892 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:31:38.897 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:38.897 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:38.907 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:31:38.991 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:38.989+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:39.014 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:39.014 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:39.031 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:39.032 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:39.048 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:39.048 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:39.056 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:39.056 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:39.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:39.057 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:39.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.057 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "pg_update_log_missing(3.5 epoch 45/37 rep_tid 844 entries 45'73 (0'0) error 3:a65c40ae:::rbd_data.1120b21c3309.0000000000000126:head by client.4384.0:48900 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 45'70)", 2024-09-17T21:31:39.057 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:31:39.057 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 92.006639699999994, 2024-09-17T21:31:39.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019554365000000001, 2024-09-17T21:31:39.057 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.057 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.057 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T21:31:39.057 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:31:39.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047111+0000", 2024-09-17T21:31:39.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047112+0000", 2024-09-17T21:31:39.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:31:39.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047121+0000", 2024-09-17T21:31:39.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4379999999999995e-06 2024-09-17T21:31:39.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047139+0000", 2024-09-17T21:31:39.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8235000000000001e-05 2024-09-17T21:31:39.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047303+0000", 2024-09-17T21:31:39.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016389400000000001 2024-09-17T21:31:39.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.047396+0000", 2024-09-17T21:31:39.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3355000000000003e-05 2024-09-17T21:31:39.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066666+0000", 2024-09-17T21:31:39.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019269649999999999 2024-09-17T21:31:39.060 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.060 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.060 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.061 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:48906 3.4 e45/25 3:2503fe27:::rbd_data.1120b21c3309.0000000000000127:head v 45'148, mlcod=45'146)", 2024-09-17T21:31:39.061 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:39.061 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 92.004513521000007, 2024-09-17T21:31:39.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018169739000000001, 2024-09-17T21:31:39.061 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.061 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.061 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:39.061 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:39.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:39.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049238+0000", 2024-09-17T21:31:39.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6300000000000004e-07 2024-09-17T21:31:39.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.049244+0000", 2024-09-17T21:31:39.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8390000000000003e-06 2024-09-17T21:31:39.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066411+0000", 2024-09-17T21:31:39.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017166818 2024-09-17T21:31:39.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066426+0000", 2024-09-17T21:31:39.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4888999999999999e-05 2024-09-17T21:31:39.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066611+0000", 2024-09-17T21:31:39.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018464499999999999 2024-09-17T21:31:39.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.066630+0000", 2024-09-17T21:31:39.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9201000000000001e-05 2024-09-17T21:31:39.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:39.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067373+0000", 2024-09-17T21:31:39.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00074322600000000004 2024-09-17T21:31:39.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.067407+0000", 2024-09-17T21:31:39.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4357999999999999e-05 2024-09-17T21:31:39.065 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.065 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.065 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.065 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50206 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'301, mlcod=45'299)", 2024-09-17T21:31:39.065 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:31:39.066 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 87.875036676999997, 2024-09-17T21:31:39.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068285810000000002, 2024-09-17T21:31:39.066 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.066 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.066 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:39.066 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:31:39.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178715+0000", 2024-09-17T21:31:39.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.178716+0000", 2024-09-17T21:31:39.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.438e-06 2024-09-17T21:31:39.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192044+0000", 2024-09-17T21:31:39.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013328370000000001 2024-09-17T21:31:39.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192069+0000", 2024-09-17T21:31:39.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4496000000000001e-05 2024-09-17T21:31:39.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192085+0000", 2024-09-17T21:31:39.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5758999999999999e-05 2024-09-17T21:31:39.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192167+0000", 2024-09-17T21:31:39.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2842000000000004e-05 2024-09-17T21:31:39.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.192201+0000", 2024-09-17T21:31:39.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3127999999999999e-05 2024-09-17T21:31:39.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:39.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.246938+0000", 2024-09-17T21:31:39.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054736982000000003 2024-09-17T21:31:39.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247000+0000", 2024-09-17T21:31:39.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2794999999999998e-05 2024-09-17T21:31:39.069 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.069 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.069 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.070 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:50207 3.1f e49/37 3:fe111236:::rbd_data.1120c2fd7e1b.0000000000000000:head v 49'303, mlcod=45'299)", 2024-09-17T21:31:39.070 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:31:39.070 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 87.832089628000006, 2024-09-17T21:31:39.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026003433999999999, 2024-09-17T21:31:39.070 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.070 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.070 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:39.070 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:31:39.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221662+0000", 2024-09-17T21:31:39.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221663+0000", 2024-09-17T21:31:39.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.271e-06 2024-09-17T21:31:39.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221672+0000", 2024-09-17T21:31:39.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6810000000000004e-06 2024-09-17T21:31:39.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221684+0000", 2024-09-17T21:31:39.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:31:39.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221691+0000", 2024-09-17T21:31:39.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7429999999999996e-06 2024-09-17T21:31:39.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221743+0000", 2024-09-17T21:31:39.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2868999999999998e-05 2024-09-17T21:31:39.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.221765+0000", 2024-09-17T21:31:39.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1316999999999999e-05 2024-09-17T21:31:39.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:39.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247616+0000", 2024-09-17T21:31:39.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025851504000000001 2024-09-17T21:31:39.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.247665+0000", 2024-09-17T21:31:39.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8739000000000002e-05 2024-09-17T21:31:39.074 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.074 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.074 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.074 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57808 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 196608~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:39.074 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:39.074 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 77.275790588000007, 2024-09-17T21:31:39.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017751475999999999, 2024-09-17T21:31:39.074 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.074 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.074 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.074 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.075 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.075 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.075 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57808 2024-09-17T21:31:39.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.075 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:39.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:39.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777961+0000", 2024-09-17T21:31:39.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:31:39.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777968+0000", 2024-09-17T21:31:39.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8530000000000003e-06 2024-09-17T21:31:39.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777975+0000", 2024-09-17T21:31:39.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4410000000000004e-06 2024-09-17T21:31:39.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777980+0000", 2024-09-17T21:31:39.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7210000000000002e-06 2024-09-17T21:31:39.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795323+0000", 2024-09-17T21:31:39.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017343319999999999 2024-09-17T21:31:39.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795410+0000", 2024-09-17T21:31:39.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6567e-05 2024-09-17T21:31:39.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795712+0000", 2024-09-17T21:31:39.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030219200000000001 2024-09-17T21:31:39.078 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.078 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.078 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.078 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57809 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 458752~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:39.078 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:31:39.078 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 77.275764735999999, 2024-09-17T21:31:39.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018092869000000001, 2024-09-17T21:31:39.079 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.079 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.079 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.079 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.079 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.079 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.079 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57809 2024-09-17T21:31:39.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.079 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:31:39.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777986+0000", 2024-09-17T21:31:39.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777987+0000", 2024-09-17T21:31:39.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T21:31:39.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777989+0000", 2024-09-17T21:31:39.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8649999999999999e-06 2024-09-17T21:31:39.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.777994+0000", 2024-09-17T21:31:39.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4199999999999998e-06 2024-09-17T21:31:39.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778004+0000", 2024-09-17T21:31:39.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6900000000000004e-06 2024-09-17T21:31:39.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795745+0000", 2024-09-17T21:31:39.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017740859000000001 2024-09-17T21:31:39.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.795815+0000", 2024-09-17T21:31:39.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0219e-05 2024-09-17T21:31:39.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796079+0000", 2024-09-17T21:31:39.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000264446 2024-09-17T21:31:39.083 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.083 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.083 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.083 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57810 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 720896~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:39.083 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:31:39.083 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 77.275741009000001, 2024-09-17T21:31:39.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018361253000000001, 2024-09-17T21:31:39.083 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.083 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.083 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.083 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.084 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.084 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.084 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57810 2024-09-17T21:31:39.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.084 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:31:39.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778010+0000", 2024-09-17T21:31:39.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778011+0000", 2024-09-17T21:31:39.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:31:39.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778012+0000", 2024-09-17T21:31:39.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.872e-06 2024-09-17T21:31:39.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778018+0000", 2024-09-17T21:31:39.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8150000000000002e-06 2024-09-17T21:31:39.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778023+0000", 2024-09-17T21:31:39.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3939999999999998e-06 2024-09-17T21:31:39.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796106+0000", 2024-09-17T21:31:39.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018083446999999999 2024-09-17T21:31:39.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796159+0000", 2024-09-17T21:31:39.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3029999999999999e-05 2024-09-17T21:31:39.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796371+0000", 2024-09-17T21:31:39.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021231700000000001 2024-09-17T21:31:39.087 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.087 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.087 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.087 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57811 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:39.088 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:39.088 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 77.275722588999997, 2024-09-17T21:31:39.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018666035000000001, 2024-09-17T21:31:39.088 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.088 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.088 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.088 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.088 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.088 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.088 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57811 2024-09-17T21:31:39.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.088 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:39.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:39.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778029+0000", 2024-09-17T21:31:39.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:39.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778031+0000", 2024-09-17T21:31:39.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.796e-06 2024-09-17T21:31:39.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778036+0000", 2024-09-17T21:31:39.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3290000000000001e-06 2024-09-17T21:31:39.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778041+0000", 2024-09-17T21:31:39.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5839999999999996e-06 2024-09-17T21:31:39.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796409+0000", 2024-09-17T21:31:39.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018368703 2024-09-17T21:31:39.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796483+0000", 2024-09-17T21:31:39.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3751999999999995e-05 2024-09-17T21:31:39.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796695+0000", 2024-09-17T21:31:39.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021153 2024-09-17T21:31:39.094 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.094 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.095 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.095 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57812 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1245184~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:39.095 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:39.095 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 77.275703691999993, 2024-09-17T21:31:39.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019000064000000001, 2024-09-17T21:31:39.095 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.095 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.095 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.095 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.095 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.095 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.095 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57812 2024-09-17T21:31:39.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.096 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:39.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:39.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778048+0000", 2024-09-17T21:31:39.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:31:39.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778050+0000", 2024-09-17T21:31:39.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7e-06 2024-09-17T21:31:39.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778055+0000", 2024-09-17T21:31:39.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4049999999999999e-06 2024-09-17T21:31:39.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778059+0000", 2024-09-17T21:31:39.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1479999999999999e-06 2024-09-17T21:31:39.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796734+0000", 2024-09-17T21:31:39.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018674399000000001 2024-09-17T21:31:39.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.796786+0000", 2024-09-17T21:31:39.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2879e-05 2024-09-17T21:31:39.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797048+0000", 2024-09-17T21:31:39.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026118400000000003 2024-09-17T21:31:39.099 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.099 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.099 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.099 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57813 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1507328~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:39.099 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:31:39.099 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 77.275685800000005, 2024-09-17T21:31:39.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019279392999999999, 2024-09-17T21:31:39.099 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.100 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.100 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.100 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.100 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.100 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.100 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57813 2024-09-17T21:31:39.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.100 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:31:39.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778065+0000", 2024-09-17T21:31:39.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778066+0000", 2024-09-17T21:31:39.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6199999999999999e-07 2024-09-17T21:31:39.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778068+0000", 2024-09-17T21:31:39.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7990000000000001e-06 2024-09-17T21:31:39.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778074+0000", 2024-09-17T21:31:39.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:31:39.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778078+0000", 2024-09-17T21:31:39.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2579999999999997e-06 2024-09-17T21:31:39.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797077+0000", 2024-09-17T21:31:39.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018999180000000001 2024-09-17T21:31:39.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797123+0000", 2024-09-17T21:31:39.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5834999999999997e-05 2024-09-17T21:31:39.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797345+0000", 2024-09-17T21:31:39.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022191100000000001 2024-09-17T21:31:39.104 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.104 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.104 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.104 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57814 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:39.104 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:31:39.104 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 77.275666928000007, 2024-09-17T21:31:39.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019604947000000001, 2024-09-17T21:31:39.105 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.105 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.105 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.105 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.105 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.105 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.105 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57814 2024-09-17T21:31:39.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.105 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:31:39.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778084+0000", 2024-09-17T21:31:39.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778085+0000", 2024-09-17T21:31:39.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:31:39.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778086+0000", 2024-09-17T21:31:39.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7850000000000001e-06 2024-09-17T21:31:39.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778093+0000", 2024-09-17T21:31:39.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6449999999999999e-06 2024-09-17T21:31:39.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778098+0000", 2024-09-17T21:31:39.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:31:39.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797383+0000", 2024-09-17T21:31:39.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019284879000000001 2024-09-17T21:31:39.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797431+0000", 2024-09-17T21:31:39.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8856000000000001e-05 2024-09-17T21:31:39.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797689+0000", 2024-09-17T21:31:39.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025785800000000002 2024-09-17T21:31:39.109 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.109 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.109 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.109 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57815 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2031616~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:39.109 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:39.109 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 77.275646260000002, 2024-09-17T21:31:39.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019910231, 2024-09-17T21:31:39.109 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.109 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.110 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.110 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.110 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.110 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.110 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57815 2024-09-17T21:31:39.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.110 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:39.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:39.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778105+0000", 2024-09-17T21:31:39.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:31:39.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778118+0000", 2024-09-17T21:31:39.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2967e-05 2024-09-17T21:31:39.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778125+0000", 2024-09-17T21:31:39.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.601e-06 2024-09-17T21:31:39.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778130+0000", 2024-09-17T21:31:39.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1839999999999998e-06 2024-09-17T21:31:39.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797720+0000", 2024-09-17T21:31:39.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019589766000000002 2024-09-17T21:31:39.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.797765+0000", 2024-09-17T21:31:39.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5281999999999998e-05 2024-09-17T21:31:39.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798015+0000", 2024-09-17T21:31:39.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025006200000000001 2024-09-17T21:31:39.114 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.114 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.114 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.114 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57816 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2293760~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:39.115 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:39.115 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 77.275613665999998, 2024-09-17T21:31:39.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020175322999999998, 2024-09-17T21:31:39.115 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.115 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.115 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.115 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.115 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.115 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.115 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57816 2024-09-17T21:31:39.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.115 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:39.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:39.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778138+0000", 2024-09-17T21:31:39.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:31:39.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778140+0000", 2024-09-17T21:31:39.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T21:31:39.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778145+0000", 2024-09-17T21:31:39.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7320000000000003e-06 2024-09-17T21:31:39.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778154+0000", 2024-09-17T21:31:39.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6969999999999999e-06 2024-09-17T21:31:39.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798049+0000", 2024-09-17T21:31:39.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.019894603 2024-09-17T21:31:39.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798106+0000", 2024-09-17T21:31:39.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7207000000000003e-05 2024-09-17T21:31:39.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798313+0000", 2024-09-17T21:31:39.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206861 2024-09-17T21:31:39.121 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.121 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.121 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.121 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57817 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2555904~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:39.121 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:39.121 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 77.27559042, 2024-09-17T21:31:39.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020466314999999999, 2024-09-17T21:31:39.122 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.122 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.122 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.122 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.122 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.122 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.122 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57817 2024-09-17T21:31:39.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.122 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:39.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:39.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778161+0000", 2024-09-17T21:31:39.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2399999999999999e-07 2024-09-17T21:31:39.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778163+0000", 2024-09-17T21:31:39.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.204e-06 2024-09-17T21:31:39.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778169+0000", 2024-09-17T21:31:39.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4960000000000004e-06 2024-09-17T21:31:39.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778173+0000", 2024-09-17T21:31:39.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5460000000000002e-06 2024-09-17T21:31:39.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798342+0000", 2024-09-17T21:31:39.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020168416000000002 2024-09-17T21:31:39.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798388+0000", 2024-09-17T21:31:39.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6171000000000002e-05 2024-09-17T21:31:39.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798627+0000", 2024-09-17T21:31:39.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000239058 2024-09-17T21:31:39.126 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.126 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.126 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.127 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57818 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 2818048~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:39.127 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:31:39.127 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 77.275571722999999, 2024-09-17T21:31:39.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020767240999999999, 2024-09-17T21:31:39.127 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.127 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.127 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.127 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.127 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.127 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.127 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57818 2024-09-17T21:31:39.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.127 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:31:39.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778179+0000", 2024-09-17T21:31:39.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778180+0000", 2024-09-17T21:31:39.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:31:39.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778182+0000", 2024-09-17T21:31:39.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7269999999999999e-06 2024-09-17T21:31:39.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778188+0000", 2024-09-17T21:31:39.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6080000000000001e-06 2024-09-17T21:31:39.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778202+0000", 2024-09-17T21:31:39.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3369e-05 2024-09-17T21:31:39.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798660+0000", 2024-09-17T21:31:39.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020458171000000001 2024-09-17T21:31:39.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798704+0000", 2024-09-17T21:31:39.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4273000000000001e-05 2024-09-17T21:31:39.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798947+0000", 2024-09-17T21:31:39.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242651 2024-09-17T21:31:39.132 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.132 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.132 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.132 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57819 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:39.132 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:31:39.132 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 77.275540853999999, 2024-09-17T21:31:39.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020892340999999998, 2024-09-17T21:31:39.133 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.133 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.133 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.133 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.133 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.133 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.133 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57819 2024-09-17T21:31:39.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.133 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:31:39.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778210+0000", 2024-09-17T21:31:39.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778211+0000", 2024-09-17T21:31:39.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:31:39.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778217+0000", 2024-09-17T21:31:39.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T21:31:39.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778224+0000", 2024-09-17T21:31:39.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3520000000000003e-06 2024-09-17T21:31:39.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778228+0000", 2024-09-17T21:31:39.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7480000000000001e-06 2024-09-17T21:31:39.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.798977+0000", 2024-09-17T21:31:39.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020748571 2024-09-17T21:31:39.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799035+0000", 2024-09-17T21:31:39.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7908999999999997e-05 2024-09-17T21:31:39.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799103+0000", 2024-09-17T21:31:39.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7794999999999997e-05 2024-09-17T21:31:39.137 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.137 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.137 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.137 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57820 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:39.137 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:39.137 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 77.275515662000004, 2024-09-17T21:31:39.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021016658000000001, 2024-09-17T21:31:39.138 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.138 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.138 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.138 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.138 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.138 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.138 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57820 2024-09-17T21:31:39.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.138 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:39.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:39.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778236+0000", 2024-09-17T21:31:39.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:31:39.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778238+0000", 2024-09-17T21:31:39.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.945e-06 2024-09-17T21:31:39.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778244+0000", 2024-09-17T21:31:39.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6520000000000003e-06 2024-09-17T21:31:39.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778248+0000", 2024-09-17T21:31:39.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5920000000000002e-06 2024-09-17T21:31:39.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799131+0000", 2024-09-17T21:31:39.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020883288999999999 2024-09-17T21:31:39.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799185+0000", 2024-09-17T21:31:39.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3223000000000002e-05 2024-09-17T21:31:39.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799252+0000", 2024-09-17T21:31:39.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7576999999999994e-05 2024-09-17T21:31:39.142 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.142 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.142 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.142 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57821 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:39.142 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:39.142 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 77.275497317000003, 2024-09-17T21:31:39.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021153528000000001, 2024-09-17T21:31:39.142 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.142 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.142 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.142 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.142 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.143 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.143 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57821 2024-09-17T21:31:39.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.143 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:39.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:39.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778254+0000", 2024-09-17T21:31:39.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-07 2024-09-17T21:31:39.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778256+0000", 2024-09-17T21:31:39.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.604e-06 2024-09-17T21:31:39.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778262+0000", 2024-09-17T21:31:39.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7629999999999997e-06 2024-09-17T21:31:39.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778266+0000", 2024-09-17T21:31:39.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:31:39.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799284+0000", 2024-09-17T21:31:39.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021017312 2024-09-17T21:31:39.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799326+0000", 2024-09-17T21:31:39.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2639000000000003e-05 2024-09-17T21:31:39.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799407+0000", 2024-09-17T21:31:39.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1077999999999999e-05 2024-09-17T21:31:39.146 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.146 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.146 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.146 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57822 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:39.147 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:31:39.147 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 77.275479007000001, 2024-09-17T21:31:39.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021268966, 2024-09-17T21:31:39.147 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.147 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.147 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.147 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57822 2024-09-17T21:31:39.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.147 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:31:39.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778272+0000", 2024-09-17T21:31:39.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778273+0000", 2024-09-17T21:31:39.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:31:39.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778274+0000", 2024-09-17T21:31:39.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.598e-06 2024-09-17T21:31:39.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778283+0000", 2024-09-17T21:31:39.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8279999999999992e-06 2024-09-17T21:31:39.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778289+0000", 2024-09-17T21:31:39.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5609999999999998e-06 2024-09-17T21:31:39.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799432+0000", 2024-09-17T21:31:39.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021143862999999999 2024-09-17T21:31:39.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799473+0000", 2024-09-17T21:31:39.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1075999999999998e-05 2024-09-17T21:31:39.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799541+0000", 2024-09-17T21:31:39.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7657999999999995e-05 2024-09-17T21:31:39.151 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.151 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.151 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.151 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:57823 3.4 3:24994175:::rbd_data.11201c5740fc.0000000000000000:18 [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T21:31:39.151 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:39.151 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 77.275456046000002, 2024-09-17T21:31:39.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021379770999999999, 2024-09-17T21:31:39.151 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.151 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.151 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.152 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.152 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.152 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.152 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 57823 2024-09-17T21:31:39.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.152 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:39.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:39.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778295+0000", 2024-09-17T21:31:39.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:31:39.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778297+0000", 2024-09-17T21:31:39.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:31:39.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778303+0000", 2024-09-17T21:31:39.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4559999999999999e-06 2024-09-17T21:31:39.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.778307+0000", 2024-09-17T21:31:39.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6229999999999996e-06 2024-09-17T21:31:39.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799560+0000", 2024-09-17T21:31:39.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.021252931999999999 2024-09-17T21:31:39.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799609+0000", 2024-09-17T21:31:39.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8301999999999999e-05 2024-09-17T21:31:39.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:21.799675+0000", 2024-09-17T21:31:39.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6205999999999996e-05 2024-09-17T21:31:39.156 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.156 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.156 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.156 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.156 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.156 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:39.157 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:39.158 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:39.159 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:31:39.159 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:39.159 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:39.160 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:39.160 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:39.161 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:39.161 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:39.162 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:39.162 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:39.156+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:39.163 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:39.258 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:39.256+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:39.302 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:39.302 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:39.302 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:39.302 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:39.311 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:31:39.359 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:39.356+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:39.413 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:39.414 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:39.426 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:31:39.428 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:31:39.459 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:39.457+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:39.561 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:39.559+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:39.584 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4384.0:49361 3.19 e46/25 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:head v 46'135, mlcod=45'133)", 2024-09-17T21:31:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 91.827820805000002, 2024-09-17T21:31:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043477744999999998, 2024-09-17T21:31:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.753569+0000", 2024-09-17T21:31:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-07 2024-09-17T21:31:39.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769703+0000", 2024-09-17T21:31:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016134381 2024-09-17T21:31:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769724+0000", 2024-09-17T21:31:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0781999999999999e-05 2024-09-17T21:31:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769734+0000", 2024-09-17T21:31:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6870000000000006e-06 2024-09-17T21:31:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769797+0000", 2024-09-17T21:31:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3418999999999997e-05 2024-09-17T21:31:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.769822+0000", 2024-09-17T21:31:39.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4652999999999999e-05 2024-09-17T21:31:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T21:31:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.796982+0000", 2024-09-17T21:31:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027160205 2024-09-17T21:31:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:07.797047+0000", 2024-09-17T21:31:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4455999999999994e-05 2024-09-17T21:31:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.589 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50805 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 17825792~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:31:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.086237311000005, 2024-09-17T21:31:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035591579999999998, 2024-09-17T21:31:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50805 2024-09-17T21:31:39.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:31:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495152+0000", 2024-09-17T21:31:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495153+0000", 2024-09-17T21:31:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:31:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495155+0000", 2024-09-17T21:31:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1849999999999998e-06 2024-09-17T21:31:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495162+0000", 2024-09-17T21:31:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.64e-06 2024-09-17T21:31:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495166+0000", 2024-09-17T21:31:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9060000000000001e-06 2024-09-17T21:31:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530020+0000", 2024-09-17T21:31:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.034853824999999998 2024-09-17T21:31:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530076+0000", 2024-09-17T21:31:39.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5235999999999998e-05 2024-09-17T21:31:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530744+0000", 2024-09-17T21:31:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066835500000000003 2024-09-17T21:31:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50806 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19398656~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:39.594 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.086216843000003, 2024-09-17T21:31:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036408459999999997, 2024-09-17T21:31:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50806 2024-09-17T21:31:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495173+0000", 2024-09-17T21:31:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.72e-07 2024-09-17T21:31:39.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495175+0000", 2024-09-17T21:31:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:31:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495182+0000", 2024-09-17T21:31:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8079999999999999e-06 2024-09-17T21:31:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495186+0000", 2024-09-17T21:31:39.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5549999999999996e-06 2024-09-17T21:31:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530791+0000", 2024-09-17T21:31:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035604963000000003 2024-09-17T21:31:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.530855+0000", 2024-09-17T21:31:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3286000000000004e-05 2024-09-17T21:31:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531581+0000", 2024-09-17T21:31:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00072655699999999996 2024-09-17T21:31:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50807 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23068672~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.086197958, 2024-09-17T21:31:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037183314000000002, 2024-09-17T21:31:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.599 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50807 2024-09-17T21:31:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495192+0000", 2024-09-17T21:31:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9999999999999999e-07 2024-09-17T21:31:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495194+0000", 2024-09-17T21:31:39.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.779e-06 2024-09-17T21:31:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495199+0000", 2024-09-17T21:31:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.468e-06 2024-09-17T21:31:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495203+0000", 2024-09-17T21:31:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0010000000000003e-06 2024-09-17T21:31:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531636+0000", 2024-09-17T21:31:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036432665000000003 2024-09-17T21:31:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.531727+0000", 2024-09-17T21:31:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1607999999999994e-05 2024-09-17T21:31:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532375+0000", 2024-09-17T21:31:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00064749299999999996 2024-09-17T21:31:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50808 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24641536~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:31:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.086181214999996, 2024-09-17T21:31:39.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037870190999999997, 2024-09-17T21:31:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50808 2024-09-17T21:31:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:31:39.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495208+0000", 2024-09-17T21:31:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495209+0000", 2024-09-17T21:31:39.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:31:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495215+0000", 2024-09-17T21:31:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1630000000000001e-06 2024-09-17T21:31:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495221+0000", 2024-09-17T21:31:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-17T21:31:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495226+0000", 2024-09-17T21:31:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8189999999999999e-06 2024-09-17T21:31:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532425+0000", 2024-09-17T21:31:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037199135000000001 2024-09-17T21:31:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.532531+0000", 2024-09-17T21:31:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106314 2024-09-17T21:31:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533079+0000", 2024-09-17T21:31:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00054739700000000003 2024-09-17T21:31:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.609 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50809 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28311552~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:31:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.086158248999993, 2024-09-17T21:31:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038597605, 2024-09-17T21:31:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50809 2024-09-17T21:31:39.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:31:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495231+0000", 2024-09-17T21:31:39.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495232+0000", 2024-09-17T21:31:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.39e-07 2024-09-17T21:31:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495234+0000", 2024-09-17T21:31:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T21:31:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495239+0000", 2024-09-17T21:31:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7969999999999997e-06 2024-09-17T21:31:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495244+0000", 2024-09-17T21:31:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999996e-06 2024-09-17T21:31:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533125+0000", 2024-09-17T21:31:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037881523 2024-09-17T21:31:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533237+0000", 2024-09-17T21:31:39.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000111879 2024-09-17T21:31:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533829+0000", 2024-09-17T21:31:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000591933 2024-09-17T21:31:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50810 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 29884416~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:31:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.086140330999996, 2024-09-17T21:31:39.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039149809000000001, 2024-09-17T21:31:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50810 2024-09-17T21:31:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:31:39.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495249+0000", 2024-09-17T21:31:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495250+0000", 2024-09-17T21:31:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:31:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495251+0000", 2024-09-17T21:31:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.81e-06 2024-09-17T21:31:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495257+0000", 2024-09-17T21:31:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1399999999999999e-06 2024-09-17T21:31:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495261+0000", 2024-09-17T21:31:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4229999999999998e-06 2024-09-17T21:31:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533879+0000", 2024-09-17T21:31:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038618292999999998 2024-09-17T21:31:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.533939+0000", 2024-09-17T21:31:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.012e-05 2024-09-17T21:31:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534399+0000", 2024-09-17T21:31:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00045968099999999998 2024-09-17T21:31:39.619 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.620 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.620 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.620 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50850 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 2359296~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:39.620 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:39.620 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.086121955999999, 2024-09-17T21:31:39.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039731491000000001, 2024-09-17T21:31:39.620 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.620 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.620 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.620 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.620 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.621 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.621 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50850 2024-09-17T21:31:39.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.621 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:39.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:39.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495268+0000", 2024-09-17T21:31:39.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3200000000000001e-07 2024-09-17T21:31:39.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495270+0000", 2024-09-17T21:31:39.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.694e-06 2024-09-17T21:31:39.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495275+0000", 2024-09-17T21:31:39.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5890000000000002e-06 2024-09-17T21:31:39.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495279+0000", 2024-09-17T21:31:39.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-17T21:31:39.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534440+0000", 2024-09-17T21:31:39.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039160805999999999 2024-09-17T21:31:39.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534526+0000", 2024-09-17T21:31:39.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5327999999999993e-05 2024-09-17T21:31:39.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.534999+0000", 2024-09-17T21:31:39.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00047353400000000001 2024-09-17T21:31:39.624 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.624 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.624 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.625 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50851 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 3932160~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:39.625 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:31:39.625 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.086104344000006, 2024-09-17T21:31:39.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040211991000000002, 2024-09-17T21:31:39.625 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.625 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.625 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.625 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.625 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.625 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.625 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50851 2024-09-17T21:31:39.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.626 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:31:39.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495285+0000", 2024-09-17T21:31:39.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495286+0000", 2024-09-17T21:31:39.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:31:39.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495287+0000", 2024-09-17T21:31:39.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6640000000000001e-06 2024-09-17T21:31:39.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495293+0000", 2024-09-17T21:31:39.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6710000000000004e-06 2024-09-17T21:31:39.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495298+0000", 2024-09-17T21:31:39.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6700000000000002e-06 2024-09-17T21:31:39.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535017+0000", 2024-09-17T21:31:39.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039719059000000001 2024-09-17T21:31:39.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535067+0000", 2024-09-17T21:31:39.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0046000000000002e-05 2024-09-17T21:31:39.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535497+0000", 2024-09-17T21:31:39.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043050299999999999 2024-09-17T21:31:39.629 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.629 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.629 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.629 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50852 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 7602176~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:39.630 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:39.630 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.086085427, 2024-09-17T21:31:39.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041166337999999997, 2024-09-17T21:31:39.630 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.630 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.630 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.630 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.630 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.631 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.631 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50852 2024-09-17T21:31:39.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.631 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:39.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495304+0000", 2024-09-17T21:31:39.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495305+0000", 2024-09-17T21:31:39.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1600000000000002e-07 2024-09-17T21:31:39.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495307+0000", 2024-09-17T21:31:39.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0980000000000001e-06 2024-09-17T21:31:39.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495312+0000", 2024-09-17T21:31:39.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6210000000000001e-06 2024-09-17T21:31:39.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495317+0000", 2024-09-17T21:31:39.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T21:31:39.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535531+0000", 2024-09-17T21:31:39.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040214580999999999 2024-09-17T21:31:39.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.535618+0000", 2024-09-17T21:31:39.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6648000000000001e-05 2024-09-17T21:31:39.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536471+0000", 2024-09-17T21:31:39.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00085251599999999999 2024-09-17T21:31:39.634 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.634 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.634 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.635 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50853 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 9175040~204800] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:39.635 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:39.635 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.086067095999994, 2024-09-17T21:31:39.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041673496999999997, 2024-09-17T21:31:39.635 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.635 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.635 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.635 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.635 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.636 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.636 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50853 2024-09-17T21:31:39.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.636 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:39.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:39.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495323+0000", 2024-09-17T21:31:39.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3299999999999998e-07 2024-09-17T21:31:39.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495334+0000", 2024-09-17T21:31:39.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.076e-05 2024-09-17T21:31:39.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495340+0000", 2024-09-17T21:31:39.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:31:39.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495345+0000", 2024-09-17T21:31:39.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8130000000000003e-06 2024-09-17T21:31:39.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536514+0000", 2024-09-17T21:31:39.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041168534999999999 2024-09-17T21:31:39.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536572+0000", 2024-09-17T21:31:39.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8742000000000001e-05 2024-09-17T21:31:39.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.536996+0000", 2024-09-17T21:31:39.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042368899999999999 2024-09-17T21:31:39.640 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.640 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.640 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.640 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50854 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 12845056~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:39.640 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:39.640 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.086038036999994, 2024-09-17T21:31:39.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042777397000000002, 2024-09-17T21:31:39.640 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.641 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.641 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.641 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.641 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.641 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.641 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50854 2024-09-17T21:31:39.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.641 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:39.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:39.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495352+0000", 2024-09-17T21:31:39.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.82e-07 2024-09-17T21:31:39.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495354+0000", 2024-09-17T21:31:39.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.091e-06 2024-09-17T21:31:39.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495360+0000", 2024-09-17T21:31:39.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.716e-06 2024-09-17T21:31:39.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495365+0000", 2024-09-17T21:31:39.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2939999999999996e-06 2024-09-17T21:31:39.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537020+0000", 2024-09-17T21:31:39.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041654419999999998 2024-09-17T21:31:39.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.537105+0000", 2024-09-17T21:31:39.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5857000000000001e-05 2024-09-17T21:31:39.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538129+0000", 2024-09-17T21:31:39.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010235369999999999 2024-09-17T21:31:39.645 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.645 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.645 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.645 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50855 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 14417920~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:39.645 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:39.645 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.086018072000002, 2024-09-17T21:31:39.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043254325000000003, 2024-09-17T21:31:39.645 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.645 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.646 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.646 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.646 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.646 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.646 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50855 2024-09-17T21:31:39.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.646 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:39.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:39.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495372+0000", 2024-09-17T21:31:39.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:31:39.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495374+0000", 2024-09-17T21:31:39.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.922e-06 2024-09-17T21:31:39.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495379+0000", 2024-09-17T21:31:39.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-17T21:31:39.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495384+0000", 2024-09-17T21:31:39.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-17T21:31:39.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538167+0000", 2024-09-17T21:31:39.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042782845 2024-09-17T21:31:39.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538218+0000", 2024-09-17T21:31:39.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1335000000000002e-05 2024-09-17T21:31:39.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538626+0000", 2024-09-17T21:31:39.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000407656 2024-09-17T21:31:39.651 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.651 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.651 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.651 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50856 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 18087936~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:39.651 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:39.651 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.085999616999999, 2024-09-17T21:31:39.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043911444000000001, 2024-09-17T21:31:39.652 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.652 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.652 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.652 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.652 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.652 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.652 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50856 2024-09-17T21:31:39.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.652 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:39.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:39.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495390+0000", 2024-09-17T21:31:39.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0499999999999999e-07 2024-09-17T21:31:39.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495392+0000", 2024-09-17T21:31:39.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7430000000000001e-06 2024-09-17T21:31:39.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495398+0000", 2024-09-17T21:31:39.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3040000000000001e-06 2024-09-17T21:31:39.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495403+0000", 2024-09-17T21:31:39.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5600000000000004e-06 2024-09-17T21:31:39.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538660+0000", 2024-09-17T21:31:39.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043256832000000002 2024-09-17T21:31:39.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.538746+0000", 2024-09-17T21:31:39.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6448999999999999e-05 2024-09-17T21:31:39.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539301+0000", 2024-09-17T21:31:39.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055525099999999999 2024-09-17T21:31:39.655 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.656 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.656 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.656 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50857 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 19660800~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:39.656 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:39.656 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.085980211999996, 2024-09-17T21:31:39.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044408156999999997, 2024-09-17T21:31:39.656 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.656 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.656 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.656 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.656 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.656 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.657 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50857 2024-09-17T21:31:39.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.657 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:39.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495409+0000", 2024-09-17T21:31:39.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495410+0000", 2024-09-17T21:31:39.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:31:39.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495412+0000", 2024-09-17T21:31:39.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9580000000000001e-06 2024-09-17T21:31:39.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495418+0000", 2024-09-17T21:31:39.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1909999999999997e-06 2024-09-17T21:31:39.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495423+0000", 2024-09-17T21:31:39.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2429999999999996e-06 2024-09-17T21:31:39.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539338+0000", 2024-09-17T21:31:39.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043914478 2024-09-17T21:31:39.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539393+0000", 2024-09-17T21:31:39.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5226999999999998e-05 2024-09-17T21:31:39.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539818+0000", 2024-09-17T21:31:39.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00042469300000000002 2024-09-17T21:31:39.660 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.660 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.660 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.660 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50858 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 23330816~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:39.660 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:39.660 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.085959114999994, 2024-09-17T21:31:39.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045010798999999997, 2024-09-17T21:31:39.661 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.661 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.661 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.661 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50858 2024-09-17T21:31:39.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.661 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:39.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:39.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495431+0000", 2024-09-17T21:31:39.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:31:39.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495433+0000", 2024-09-17T21:31:39.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.787e-06 2024-09-17T21:31:39.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495438+0000", 2024-09-17T21:31:39.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:31:39.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495442+0000", 2024-09-17T21:31:39.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2640000000000002e-06 2024-09-17T21:31:39.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539852+0000", 2024-09-17T21:31:39.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.044409592999999997 2024-09-17T21:31:39.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.539940+0000", 2024-09-17T21:31:39.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7696999999999997e-05 2024-09-17T21:31:39.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540441+0000", 2024-09-17T21:31:39.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050164299999999997 2024-09-17T21:31:39.664 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.665 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.665 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.665 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50859 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 24903680~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:39.665 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:39.665 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.085941437000002, 2024-09-17T21:31:39.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045571569999999999, 2024-09-17T21:31:39.665 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.665 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.665 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.665 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50859 2024-09-17T21:31:39.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.666 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:39.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495448+0000", 2024-09-17T21:31:39.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495449+0000", 2024-09-17T21:31:39.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.41e-07 2024-09-17T21:31:39.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495450+0000", 2024-09-17T21:31:39.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7889999999999999e-06 2024-09-17T21:31:39.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495456+0000", 2024-09-17T21:31:39.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5679999999999999e-06 2024-09-17T21:31:39.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495460+0000", 2024-09-17T21:31:39.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4989999999999996e-06 2024-09-17T21:31:39.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540472+0000", 2024-09-17T21:31:39.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045011585999999999 2024-09-17T21:31:39.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.540554+0000", 2024-09-17T21:31:39.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2443999999999999e-05 2024-09-17T21:31:39.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541020+0000", 2024-09-17T21:31:39.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00046534300000000001 2024-09-17T21:31:39.669 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.669 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.669 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.669 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50860 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 28573696~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:39.669 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:39.669 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.085923084000001, 2024-09-17T21:31:39.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046167899999999998, 2024-09-17T21:31:39.670 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.670 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.670 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.670 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.670 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.670 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.670 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50860 2024-09-17T21:31:39.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.670 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:39.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:39.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495467+0000", 2024-09-17T21:31:39.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:31:39.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495469+0000", 2024-09-17T21:31:39.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.99e-06 2024-09-17T21:31:39.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495475+0000", 2024-09-17T21:31:39.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:31:39.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495480+0000", 2024-09-17T21:31:39.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:31:39.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541050+0000", 2024-09-17T21:31:39.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045569853 2024-09-17T21:31:39.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541138+0000", 2024-09-17T21:31:39.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7997000000000005e-05 2024-09-17T21:31:39.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541634+0000", 2024-09-17T21:31:39.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00049685500000000004 2024-09-17T21:31:39.675 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.675 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.675 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.675 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:50861 3.19 3:993d3632:::rbd_data.1120c2fd7e1b.0000000000000003:10 [sparse-read 30146560~262144] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T21:31:39.675 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:39.675 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 88.085903907000002, 2024-09-17T21:31:39.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046690553000000003, 2024-09-17T21:31:39.676 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.676 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.676 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.676 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.676 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.676 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.677 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 50861 2024-09-17T21:31:39.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.677 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:39.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:39.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495486+0000", 2024-09-17T21:31:39.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:31:39.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495488+0000", 2024-09-17T21:31:39.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:31:39.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495495+0000", 2024-09-17T21:31:39.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:31:39.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.495500+0000", 2024-09-17T21:31:39.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8620000000000002e-06 2024-09-17T21:31:39.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541687+0000", 2024-09-17T21:31:39.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046186434999999998 2024-09-17T21:31:39.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.541741+0000", 2024-09-17T21:31:39.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4228999999999999e-05 2024-09-17T21:31:39.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:11.542176+0000", 2024-09-17T21:31:39.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043531100000000002 2024-09-17T21:31:39.681 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.681 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.682 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.682 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97804 3.7 3:e725aa05:::rbd_data.1120c318df8c.0000000000000003:24 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:39.682 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:39.682 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.619957806000002, 2024-09-17T21:31:39.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035514416, 2024-09-17T21:31:39.682 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:39.682 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:39.683 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:39.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:39.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:39.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:39.683 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97804 2024-09-17T21:31:39.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.683 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:39.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:39.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:39.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:39.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:39.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:39.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:39.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961432+0000", 2024-09-17T21:31:39.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T21:31:39.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:39.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961434+0000", 2024-09-17T21:31:39.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8959999999999999e-06 2024-09-17T21:31:39.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:39.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961439+0000", 2024-09-17T21:31:39.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T21:31:39.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:39.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.961444+0000", 2024-09-17T21:31:39.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4970000000000003e-06 2024-09-17T21:31:39.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:39.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996466+0000", 2024-09-17T21:31:39.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.035022438000000003 2024-09-17T21:31:39.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:39.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996556+0000", 2024-09-17T21:31:39.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9561000000000005e-05 2024-09-17T21:31:39.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:39.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:39.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:39.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.996946+0000", 2024-09-17T21:31:39.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039028700000000001 2024-09-17T21:31:39.687 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.687 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.687 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.687 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:39.687 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:39.687 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:39.688 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:31:39.689 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:39.686+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:39.789 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:39.787+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:39.794 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:39.794 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:39.794 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:39.794 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:39.803 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:31:39.890 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:39.888+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:39.946 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:39.946 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:39.947 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:39.947 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:39.947 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:39.955 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:31:39.990 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:39.988+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:40.092 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:40.090+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:40.099 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:40.099 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:40.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:40.099 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:40.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.099 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21342 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2082816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:31:40.099 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:40.099 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 110.419457421, 2024-09-17T21:31:40.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022083523000000001, 2024-09-17T21:31:40.099 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.099 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.100 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.100 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.100 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.100 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.100 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21342 2024-09-17T21:31:40.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.100 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:40.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:40.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:40.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:31:40.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676659+0000", 2024-09-17T21:31:40.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0950000000000001e-06 2024-09-17T21:31:40.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676662+0000", 2024-09-17T21:31:40.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.971e-06 2024-09-17T21:31:40.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676665+0000", 2024-09-17T21:31:40.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6189999999999998e-06 2024-09-17T21:31:40.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698568+0000", 2024-09-17T21:31:40.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021903549000000001 2024-09-17T21:31:40.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698668+0000", 2024-09-17T21:31:40.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9778999999999995e-05 2024-09-17T21:31:40.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698741+0000", 2024-09-17T21:31:40.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000006e-05 2024-09-17T21:31:40.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.103 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.104 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21343 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2095104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:31:40.104 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:40.104 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 110.419447483, 2024-09-17T21:31:40.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022504024000000001, 2024-09-17T21:31:40.104 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.104 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.104 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.104 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21343 2024-09-17T21:31:40.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.104 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:40.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:40.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:40.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:31:40.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676669+0000", 2024-09-17T21:31:40.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.003e-06 2024-09-17T21:31:40.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676672+0000", 2024-09-17T21:31:40.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9280000000000002e-06 2024-09-17T21:31:40.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676674+0000", 2024-09-17T21:31:40.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5469999999999999e-06 2024-09-17T21:31:40.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699015+0000", 2024-09-17T21:31:40.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022340563000000001 2024-09-17T21:31:40.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699102+0000", 2024-09-17T21:31:40.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7425999999999999e-05 2024-09-17T21:31:40.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699172+0000", 2024-09-17T21:31:40.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9346999999999999e-05 2024-09-17T21:31:40.111 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.111 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.111 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.111 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45576 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:40.111 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:40.111 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.391213683000004, 2024-09-17T21:31:40.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022024660000000001, 2024-09-17T21:31:40.111 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.111 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.111 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.111 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.111 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.112 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.112 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45576 2024-09-17T21:31:40.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.112 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:40.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:40.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704902+0000", 2024-09-17T21:31:40.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9100000000000004e-07 2024-09-17T21:31:40.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704905+0000", 2024-09-17T21:31:40.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6139999999999999e-06 2024-09-17T21:31:40.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704911+0000", 2024-09-17T21:31:40.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8619999999999997e-06 2024-09-17T21:31:40.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704917+0000", 2024-09-17T21:31:40.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:31:40.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726681+0000", 2024-09-17T21:31:40.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021764569000000001 2024-09-17T21:31:40.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726726+0000", 2024-09-17T21:31:40.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4218999999999998e-05 2024-09-17T21:31:40.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726926+0000", 2024-09-17T21:31:40.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000200466 2024-09-17T21:31:40.115 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.115 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.115 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.115 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45577 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:40.115 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:40.115 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.391190026999993, 2024-09-17T21:31:40.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022269694999999999, 2024-09-17T21:31:40.116 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.116 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.116 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.116 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.116 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.116 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.116 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45577 2024-09-17T21:31:40.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.116 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:40.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:40.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704926+0000", 2024-09-17T21:31:40.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:31:40.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704928+0000", 2024-09-17T21:31:40.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9989999999999998e-06 2024-09-17T21:31:40.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704934+0000", 2024-09-17T21:31:40.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:31:40.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704939+0000", 2024-09-17T21:31:40.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5909999999999998e-06 2024-09-17T21:31:40.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726954+0000", 2024-09-17T21:31:40.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022014885000000001 2024-09-17T21:31:40.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726995+0000", 2024-09-17T21:31:40.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1474000000000003e-05 2024-09-17T21:31:40.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727195+0000", 2024-09-17T21:31:40.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019968300000000001 2024-09-17T21:31:40.119 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.119 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.119 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.120 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45578 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:40.120 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:40.120 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.391168782999998, 2024-09-17T21:31:40.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022484006000000001, 2024-09-17T21:31:40.120 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.120 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.120 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.120 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.120 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.120 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.120 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45578 2024-09-17T21:31:40.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.120 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:40.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:40.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704947+0000", 2024-09-17T21:31:40.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1399999999999997e-07 2024-09-17T21:31:40.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704949+0000", 2024-09-17T21:31:40.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0959999999999999e-06 2024-09-17T21:31:40.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704956+0000", 2024-09-17T21:31:40.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9330000000000004e-06 2024-09-17T21:31:40.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704961+0000", 2024-09-17T21:31:40.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6759999999999998e-06 2024-09-17T21:31:40.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727213+0000", 2024-09-17T21:31:40.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022252568 2024-09-17T21:31:40.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727259+0000", 2024-09-17T21:31:40.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5238000000000003e-05 2024-09-17T21:31:40.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727430+0000", 2024-09-17T21:31:40.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000171881 2024-09-17T21:31:40.124 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.124 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.124 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.124 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45579 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:40.124 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:40.124 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.391146855000002, 2024-09-17T21:31:40.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022749080000000001, 2024-09-17T21:31:40.124 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.124 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.124 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.124 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.124 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.125 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.125 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45579 2024-09-17T21:31:40.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.125 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:40.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:40.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704969+0000", 2024-09-17T21:31:40.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:31:40.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704971+0000", 2024-09-17T21:31:40.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.097e-06 2024-09-17T21:31:40.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704977+0000", 2024-09-17T21:31:40.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6329999999999999e-06 2024-09-17T21:31:40.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704983+0000", 2024-09-17T21:31:40.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:31:40.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727457+0000", 2024-09-17T21:31:40.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022474521000000001 2024-09-17T21:31:40.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727498+0000", 2024-09-17T21:31:40.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0558e-05 2024-09-17T21:31:40.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727717+0000", 2024-09-17T21:31:40.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219805 2024-09-17T21:31:40.128 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.128 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.128 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.128 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45652 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:40.128 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:40.128 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.390000717000007, 2024-09-17T21:31:40.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022100465999999999, 2024-09-17T21:31:40.128 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.128 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.128 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.128 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.129 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.129 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.129 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45652 2024-09-17T21:31:40.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.129 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:40.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:40.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706115+0000", 2024-09-17T21:31:40.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:31:40.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706117+0000", 2024-09-17T21:31:40.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-06 2024-09-17T21:31:40.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706122+0000", 2024-09-17T21:31:40.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.288e-06 2024-09-17T21:31:40.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706136+0000", 2024-09-17T21:31:40.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3409e-05 2024-09-17T21:31:40.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728112+0000", 2024-09-17T21:31:40.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021976724999999999 2024-09-17T21:31:40.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728151+0000", 2024-09-17T21:31:40.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8318999999999997e-05 2024-09-17T21:31:40.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728215+0000", 2024-09-17T21:31:40.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4172999999999997e-05 2024-09-17T21:31:40.132 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.132 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.132 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.133 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45653 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:40.133 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:40.133 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.389971506999998, 2024-09-17T21:31:40.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022249517999999999, 2024-09-17T21:31:40.133 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.133 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.133 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.133 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.133 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.133 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.133 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45653 2024-09-17T21:31:40.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.134 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:40.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:40.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:40.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T21:31:40.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706147+0000", 2024-09-17T21:31:40.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4430000000000002e-06 2024-09-17T21:31:40.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706152+0000", 2024-09-17T21:31:40.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5350000000000004e-06 2024-09-17T21:31:40.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706157+0000", 2024-09-17T21:31:40.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8420000000000004e-06 2024-09-17T21:31:40.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728246+0000", 2024-09-17T21:31:40.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022089237000000001 2024-09-17T21:31:40.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728309+0000", 2024-09-17T21:31:40.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2356999999999999e-05 2024-09-17T21:31:40.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:31:40.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4596000000000001e-05 2024-09-17T21:31:40.137 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.137 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.137 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.137 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45657 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 36864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:40.137 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:40.137 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.389940242999998, 2024-09-17T21:31:40.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022218252000000001, 2024-09-17T21:31:40.137 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.137 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.137 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.138 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.138 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.138 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.138 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45657 2024-09-17T21:31:40.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.138 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:40.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:40.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:40.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8299999999999997e-07 2024-09-17T21:31:40.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706178+0000", 2024-09-17T21:31:40.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3700000000000002e-06 2024-09-17T21:31:40.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706184+0000", 2024-09-17T21:31:40.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.144e-06 2024-09-17T21:31:40.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706190+0000", 2024-09-17T21:31:40.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8590000000000001e-06 2024-09-17T21:31:40.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728240+0000", 2024-09-17T21:31:40.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022049804999999999 2024-09-17T21:31:40.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728307+0000", 2024-09-17T21:31:40.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7477999999999993e-05 2024-09-17T21:31:40.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:31:40.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6112999999999994e-05 2024-09-17T21:31:40.141 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.141 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.141 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.141 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45658 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:40.141 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:40.141 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.389917123000004, 2024-09-17T21:31:40.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022376455, 2024-09-17T21:31:40.142 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.142 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.142 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.142 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.142 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.142 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.142 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45658 2024-09-17T21:31:40.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.142 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:40.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:40.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:40.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:31:40.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706200+0000", 2024-09-17T21:31:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:31:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706206+0000", 2024-09-17T21:31:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7039999999999999e-06 2024-09-17T21:31:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706211+0000", 2024-09-17T21:31:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9429999999999999e-06 2024-09-17T21:31:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728418+0000", 2024-09-17T21:31:40.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022206760999999998 2024-09-17T21:31:40.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728497+0000", 2024-09-17T21:31:40.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9493999999999995e-05 2024-09-17T21:31:40.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728574+0000", 2024-09-17T21:31:40.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7180000000000003e-05 2024-09-17T21:31:40.145 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.145 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.146 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.146 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45707 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 212992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:40.146 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:40.146 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.388213057000002, 2024-09-17T21:31:40.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022235550999999999, 2024-09-17T21:31:40.146 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.146 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.146 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.146 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.146 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.146 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.146 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45707 2024-09-17T21:31:40.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.146 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:40.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:40.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:40.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:31:40.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707905+0000", 2024-09-17T21:31:40.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2120000000000002e-06 2024-09-17T21:31:40.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707911+0000", 2024-09-17T21:31:40.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1929999999999998e-06 2024-09-17T21:31:40.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707915+0000", 2024-09-17T21:31:40.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:31:40.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729897+0000", 2024-09-17T21:31:40.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021981186 2024-09-17T21:31:40.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729944+0000", 2024-09-17T21:31:40.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571000000000002e-05 2024-09-17T21:31:40.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730138+0000", 2024-09-17T21:31:40.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000193451 2024-09-17T21:31:40.150 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.150 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.150 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.150 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45708 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:40.150 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:40.150 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.388192924999998, 2024-09-17T21:31:40.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022533742999999998, 2024-09-17T21:31:40.150 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.150 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.151 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.151 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.151 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.151 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.151 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45708 2024-09-17T21:31:40.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.151 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:40.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:40.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707923+0000", 2024-09-17T21:31:40.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:31:40.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707932+0000", 2024-09-17T21:31:40.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6430000000000007e-06 2024-09-17T21:31:40.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707939+0000", 2024-09-17T21:31:40.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3539999999999996e-06 2024-09-17T21:31:40.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707944+0000", 2024-09-17T21:31:40.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3249999999999998e-06 2024-09-17T21:31:40.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730176+0000", 2024-09-17T21:31:40.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022232393 2024-09-17T21:31:40.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730263+0000", 2024-09-17T21:31:40.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6638999999999995e-05 2024-09-17T21:31:40.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730456+0000", 2024-09-17T21:31:40.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019304899999999999 2024-09-17T21:31:40.154 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.154 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.154 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.154 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45752 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:40.155 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:40.155 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.387937704999999, 2024-09-17T21:31:40.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022426878000000001, 2024-09-17T21:31:40.155 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.155 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.155 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.155 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.155 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.155 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.155 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45752 2024-09-17T21:31:40.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.155 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:40.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:40.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708178+0000", 2024-09-17T21:31:40.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:31:40.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708181+0000", 2024-09-17T21:31:40.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.909e-06 2024-09-17T21:31:40.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708187+0000", 2024-09-17T21:31:40.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4999999999999996e-06 2024-09-17T21:31:40.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708193+0000", 2024-09-17T21:31:40.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:31:40.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730484+0000", 2024-09-17T21:31:40.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022291765000000002 2024-09-17T21:31:40.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730541+0000", 2024-09-17T21:31:40.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6483000000000001e-05 2024-09-17T21:31:40.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730604+0000", 2024-09-17T21:31:40.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3429000000000005e-05 2024-09-17T21:31:40.159 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.159 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.159 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.159 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45761 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:40.159 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:40.159 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.387909794999999, 2024-09-17T21:31:40.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022614408999999999, 2024-09-17T21:31:40.159 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.159 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.159 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.159 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.159 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.160 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.160 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45761 2024-09-17T21:31:40.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.160 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:40.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:40.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708206+0000", 2024-09-17T21:31:40.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4700000000000001e-07 2024-09-17T21:31:40.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708208+0000", 2024-09-17T21:31:40.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1509999999999998e-06 2024-09-17T21:31:40.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708214+0000", 2024-09-17T21:31:40.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.49e-06 2024-09-17T21:31:40.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708219+0000", 2024-09-17T21:31:40.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8119999999999998e-06 2024-09-17T21:31:40.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730641+0000", 2024-09-17T21:31:40.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022422765000000001 2024-09-17T21:31:40.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730731+0000", 2024-09-17T21:31:40.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9598999999999994e-05 2024-09-17T21:31:40.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730820+0000", 2024-09-17T21:31:40.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8844999999999994e-05 2024-09-17T21:31:40.163 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.163 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.163 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.163 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45785 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:40.163 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:40.164 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.386688007999993, 2024-09-17T21:31:40.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022044113000000001, 2024-09-17T21:31:40.164 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.164 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.164 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.164 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.164 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.164 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.164 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45785 2024-09-17T21:31:40.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.164 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:40.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:40.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709428+0000", 2024-09-17T21:31:40.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:31:40.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709430+0000", 2024-09-17T21:31:40.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:31:40.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709451+0000", 2024-09-17T21:31:40.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1372000000000001e-05 2024-09-17T21:31:40.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709472+0000", 2024-09-17T21:31:40.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1007999999999999e-05 2024-09-17T21:31:40.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731293+0000", 2024-09-17T21:31:40.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021821320000000002 2024-09-17T21:31:40.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731378+0000", 2024-09-17T21:31:40.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5059e-05 2024-09-17T21:31:40.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731471+0000", 2024-09-17T21:31:40.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2867000000000004e-05 2024-09-17T21:31:40.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.168 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.168 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.168 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45786 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:40.168 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:40.168 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 93.386629210999999, 2024-09-17T21:31:40.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022181821000000001, 2024-09-17T21:31:40.168 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.168 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.168 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.169 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45786 2024-09-17T21:31:40.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.169 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:40.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:40.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709487+0000", 2024-09-17T21:31:40.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1299999999999999e-07 2024-09-17T21:31:40.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709491+0000", 2024-09-17T21:31:40.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2089999999999999e-06 2024-09-17T21:31:40.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709498+0000", 2024-09-17T21:31:40.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3980000000000002e-06 2024-09-17T21:31:40.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709504+0000", 2024-09-17T21:31:40.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7219999999999996e-06 2024-09-17T21:31:40.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731499+0000", 2024-09-17T21:31:40.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021995297 2024-09-17T21:31:40.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731591+0000", 2024-09-17T21:31:40.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1373999999999996e-05 2024-09-17T21:31:40.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731668+0000", 2024-09-17T21:31:40.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7107999999999994e-05 2024-09-17T21:31:40.172 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.172 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.172 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.172 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50202 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'139, mlcod=45'137)", 2024-09-17T21:31:40.173 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:40.173 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 88.971893342000001, 2024-09-17T21:31:40.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059659441000000001, 2024-09-17T21:31:40.173 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.173 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.173 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:40.173 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:40.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:40.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124224+0000", 2024-09-17T21:31:40.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5000000000000002e-06 2024-09-17T21:31:40.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147385+0000", 2024-09-17T21:31:40.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023160544000000002 2024-09-17T21:31:40.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147411+0000", 2024-09-17T21:31:40.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6095999999999999e-05 2024-09-17T21:31:40.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147438+0000", 2024-09-17T21:31:40.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6982999999999999e-05 2024-09-17T21:31:40.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147504+0000", 2024-09-17T21:31:40.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6156999999999997e-05 2024-09-17T21:31:40.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147531+0000", 2024-09-17T21:31:40.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6741999999999999e-05 2024-09-17T21:31:40.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:40.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183820+0000", 2024-09-17T21:31:40.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036289090000000003 2024-09-17T21:31:40.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183881+0000", 2024-09-17T21:31:40.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1328999999999995e-05 2024-09-17T21:31:40.177 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.177 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.177 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.177 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50203 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'141, mlcod=45'137)", 2024-09-17T21:31:40.177 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:40.177 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 88.934630599000002, 2024-09-17T21:31:40.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023138823999999999, 2024-09-17T21:31:40.177 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.178 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.178 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:40.178 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:40.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:40.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161486+0000", 2024-09-17T21:31:40.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5290000000000001e-06 2024-09-17T21:31:40.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161496+0000", 2024-09-17T21:31:40.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0063e-05 2024-09-17T21:31:40.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161514+0000", 2024-09-17T21:31:40.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7728e-05 2024-09-17T21:31:40.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161524+0000", 2024-09-17T21:31:40.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0339e-05 2024-09-17T21:31:40.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170872+0000", 2024-09-17T21:31:40.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093473640000000004 2024-09-17T21:31:40.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170899+0000", 2024-09-17T21:31:40.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7321000000000002e-05 2024-09-17T21:31:40.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:40.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184580+0000", 2024-09-17T21:31:40.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013680988999999999 2024-09-17T21:31:40.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184623+0000", 2024-09-17T21:31:40.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3491000000000001e-05 2024-09-17T21:31:40.181 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.181 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.181 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.182 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:88193 3.9 3:97498da5:::rbd_data.1120ea662e44.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-17T21:31:40.182 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:40.182 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.772449688999998, 2024-09-17T21:31:40.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026278261000000001, 2024-09-17T21:31:40.182 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.182 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.182 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:40.182 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.182 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.182 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.182 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 88193 2024-09-17T21:31:40.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.183 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:40.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:40.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323666+0000", 2024-09-17T21:31:40.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:31:40.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323670+0000", 2024-09-17T21:31:40.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.771e-06 2024-09-17T21:31:40.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323680+0000", 2024-09-17T21:31:40.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9469999999999998e-06 2024-09-17T21:31:40.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323685+0000", 2024-09-17T21:31:40.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9919999999999998e-06 2024-09-17T21:31:40.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323728+0000", 2024-09-17T21:31:40.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2811000000000001e-05 2024-09-17T21:31:40.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323776+0000", 2024-09-17T21:31:40.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8310000000000003e-05 2024-09-17T21:31:40.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:40.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.328583+0000", 2024-09-17T21:31:40.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048072899999999997 2024-09-17T21:31:40.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:40.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349861+0000", 2024-09-17T21:31:40.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021277695999999999 2024-09-17T21:31:40.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:40.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349877+0000", 2024-09-17T21:31:40.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:31:40.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349911+0000", 2024-09-17T21:31:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3844999999999999e-05 2024-09-17T21:31:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349944+0000", 2024-09-17T21:31:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2663000000000002e-05 2024-09-17T21:31:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:101384 3.9 3:90f2a32a:::rbd_data.11206b8c0882.0000000000000178:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T21:31:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.769580089999998, 2024-09-17T21:31:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025286194000000001, 2024-09-17T21:31:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 101384 2024-09-17T21:31:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326536+0000", 2024-09-17T21:31:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4799999999999997e-07 2024-09-17T21:31:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326541+0000", 2024-09-17T21:31:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8489999999999997e-06 2024-09-17T21:31:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326552+0000", 2024-09-17T21:31:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0912e-05 2024-09-17T21:31:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326557+0000", 2024-09-17T21:31:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0359999999999997e-06 2024-09-17T21:31:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326603+0000", 2024-09-17T21:31:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6428999999999999e-05 2024-09-17T21:31:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326686+0000", 2024-09-17T21:31:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2521999999999994e-05 2024-09-17T21:31:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.332698+0000", 2024-09-17T21:31:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060124599999999999 2024-09-17T21:31:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351744+0000", 2024-09-17T21:31:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019045838999999998 2024-09-17T21:31:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351754+0000", 2024-09-17T21:31:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0380999999999999e-05 2024-09-17T21:31:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351791+0000", 2024-09-17T21:31:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6678999999999999e-05 2024-09-17T21:31:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351821+0000", 2024-09-17T21:31:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0139000000000001e-05 2024-09-17T21:31:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.193 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:40.195 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:31:40.195 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:40.192+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:40.295 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:40.292+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:40.338 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:40.338 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:40.339 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:40.339 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:40.347 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:31:40.415 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is set 2024-09-17T21:31:40.432 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:40.430+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:40.496 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:40.494+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:40.497 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:40.498 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:40.498 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:40.498 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:40.498 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:40.507 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:31:40.597 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:40.595+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:40.653 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:40.653 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:40.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:40.653 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:40.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.653 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10799 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.653 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:40.653 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.584875309, 2024-09-17T21:31:40.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039122745, 2024-09-17T21:31:40.653 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.653 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.654 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:40.654 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.654 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.654 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.654 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10799 2024-09-17T21:31:40.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.654 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:40.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:40.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065469+0000", 2024-09-17T21:31:40.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:31:40.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065477+0000", 2024-09-17T21:31:40.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1319999999999994e-06 2024-09-17T21:31:40.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065483+0000", 2024-09-17T21:31:40.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2040000000000002e-06 2024-09-17T21:31:40.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065487+0000", 2024-09-17T21:31:40.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.42e-06 2024-09-17T21:31:40.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099984+0000", 2024-09-17T21:31:40.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034496484000000001 2024-09-17T21:31:40.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100065+0000", 2024-09-17T21:31:40.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1200999999999997e-05 2024-09-17T21:31:40.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:40.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100422+0000", 2024-09-17T21:31:40.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035657700000000001 2024-09-17T21:31:40.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:40.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104214+0000", 2024-09-17T21:31:40.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003792045 2024-09-17T21:31:40.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:40.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104505+0000", 2024-09-17T21:31:40.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029141399999999999 2024-09-17T21:31:40.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:40.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104556+0000", 2024-09-17T21:31:40.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1388999999999999e-05 2024-09-17T21:31:40.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104591+0000", 2024-09-17T21:31:40.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4455999999999997e-05 2024-09-17T21:31:40.658 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.658 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.659 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.659 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12187 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.659 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:40.659 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.510782578, 2024-09-17T21:31:40.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038825750999999999, 2024-09-17T21:31:40.659 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.659 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.659 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.659 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.659 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.659 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.659 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12187 2024-09-17T21:31:40.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.659 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:40.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:40.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:40.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:31:40.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139563+0000", 2024-09-17T21:31:40.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.624e-06 2024-09-17T21:31:40.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139569+0000", 2024-09-17T21:31:40.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1090000000000003e-06 2024-09-17T21:31:40.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139573+0000", 2024-09-17T21:31:40.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4429999999999996e-06 2024-09-17T21:31:40.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178206+0000", 2024-09-17T21:31:40.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038632376000000003 2024-09-17T21:31:40.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178282+0000", 2024-09-17T21:31:40.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6076999999999997e-05 2024-09-17T21:31:40.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178387+0000", 2024-09-17T21:31:40.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104682 2024-09-17T21:31:40.663 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.663 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.663 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.663 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12188 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.663 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:40.663 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.510764397, 2024-09-17T21:31:40.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039053826999999999, 2024-09-17T21:31:40.663 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.663 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.663 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.663 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.663 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.663 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.664 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12188 2024-09-17T21:31:40.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.664 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:40.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:40.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139746+0000", 2024-09-17T21:31:40.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000167281 2024-09-17T21:31:40.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139749+0000", 2024-09-17T21:31:40.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9660000000000001e-06 2024-09-17T21:31:40.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139757+0000", 2024-09-17T21:31:40.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8029999999999997e-06 2024-09-17T21:31:40.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139771+0000", 2024-09-17T21:31:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3541e-05 2024-09-17T21:31:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178415+0000", 2024-09-17T21:31:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038644803999999998 2024-09-17T21:31:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178517+0000", 2024-09-17T21:31:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101298 2024-09-17T21:31:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178633+0000", 2024-09-17T21:31:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116134 2024-09-17T21:31:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12189 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.51056392, 2024-09-17T21:31:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039135991000000002, 2024-09-17T21:31:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12189 2024-09-17T21:31:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139780+0000", 2024-09-17T21:31:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:31:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139782+0000", 2024-09-17T21:31:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1950000000000002e-06 2024-09-17T21:31:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139788+0000", 2024-09-17T21:31:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0639999999999999e-06 2024-09-17T21:31:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139792+0000", 2024-09-17T21:31:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1960000000000001e-06 2024-09-17T21:31:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178671+0000", 2024-09-17T21:31:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038878176 2024-09-17T21:31:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178778+0000", 2024-09-17T21:31:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107278 2024-09-17T21:31:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178915+0000", 2024-09-17T21:31:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013757900000000001 2024-09-17T21:31:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12190 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.510544737, 2024-09-17T21:31:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039343781000000001, 2024-09-17T21:31:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12190 2024-09-17T21:31:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139801+0000", 2024-09-17T21:31:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.916e-06 2024-09-17T21:31:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139807+0000", 2024-09-17T21:31:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4540000000000004e-06 2024-09-17T21:31:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139812+0000", 2024-09-17T21:31:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9729999999999997e-06 2024-09-17T21:31:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178960+0000", 2024-09-17T21:31:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039147567000000001 2024-09-17T21:31:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179051+0000", 2024-09-17T21:31:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0754000000000006e-05 2024-09-17T21:31:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179142+0000", 2024-09-17T21:31:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1712999999999994e-05 2024-09-17T21:31:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12191 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.510524879, 2024-09-17T21:31:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039464336000000003, 2024-09-17T21:31:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12191 2024-09-17T21:31:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:31:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139821+0000", 2024-09-17T21:31:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1260000000000001e-06 2024-09-17T21:31:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139827+0000", 2024-09-17T21:31:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T21:31:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139840+0000", 2024-09-17T21:31:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2607e-05 2024-09-17T21:31:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179162+0000", 2024-09-17T21:31:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039322863999999999 2024-09-17T21:31:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179204+0000", 2024-09-17T21:31:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1938999999999999e-05 2024-09-17T21:31:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179283+0000", 2024-09-17T21:31:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8547000000000005e-05 2024-09-17T21:31:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12192 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.510497167, 2024-09-17T21:31:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039652318999999998, 2024-09-17T21:31:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12192 2024-09-17T21:31:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141458+0000", 2024-09-17T21:31:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001612098 2024-09-17T21:31:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141469+0000", 2024-09-17T21:31:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-17T21:31:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141478+0000", 2024-09-17T21:31:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T21:31:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141485+0000", 2024-09-17T21:31:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4760000000000003e-06 2024-09-17T21:31:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179317+0000", 2024-09-17T21:31:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037831832000000003 2024-09-17T21:31:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179395+0000", 2024-09-17T21:31:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8349000000000005e-05 2024-09-17T21:31:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179499+0000", 2024-09-17T21:31:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103628 2024-09-17T21:31:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12257 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.508308359, 2024-09-17T21:31:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038862751000000001, 2024-09-17T21:31:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12257 2024-09-17T21:31:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T21:31:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142037+0000", 2024-09-17T21:31:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.03e-06 2024-09-17T21:31:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142043+0000", 2024-09-17T21:31:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6319999999999996e-06 2024-09-17T21:31:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142052+0000", 2024-09-17T21:31:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2709999999999998e-06 2024-09-17T21:31:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180794+0000", 2024-09-17T21:31:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038741758000000001 2024-09-17T21:31:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180843+0000", 2024-09-17T21:31:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9021999999999999e-05 2024-09-17T21:31:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180898+0000", 2024-09-17T21:31:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673000000000003e-05 2024-09-17T21:31:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12258 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.50828484199999, 2024-09-17T21:31:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038950717000000003, 2024-09-17T21:31:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12258 2024-09-17T21:31:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199999999999999e-07 2024-09-17T21:31:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142061+0000", 2024-09-17T21:31:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8279999999999999e-06 2024-09-17T21:31:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142067+0000", 2024-09-17T21:31:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5600000000000001e-06 2024-09-17T21:31:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142071+0000", 2024-09-17T21:31:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6240000000000001e-06 2024-09-17T21:31:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180921+0000", 2024-09-17T21:31:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038849366000000003 2024-09-17T21:31:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180961+0000", 2024-09-17T21:31:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0219000000000002e-05 2024-09-17T21:31:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181009+0000", 2024-09-17T21:31:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8437999999999999e-05 2024-09-17T21:31:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12259 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.508257595, 2024-09-17T21:31:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039116832999999997, 2024-09-17T21:31:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12259 2024-09-17T21:31:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2200000000000004e-07 2024-09-17T21:31:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142088+0000", 2024-09-17T21:31:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0329999999999998e-06 2024-09-17T21:31:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142095+0000", 2024-09-17T21:31:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4320000000000004e-06 2024-09-17T21:31:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142100+0000", 2024-09-17T21:31:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:31:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181037+0000", 2024-09-17T21:31:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038937601000000002 2024-09-17T21:31:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181114+0000", 2024-09-17T21:31:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6857e-05 2024-09-17T21:31:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181203+0000", 2024-09-17T21:31:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8437999999999996e-05 2024-09-17T21:31:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12260 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.50823730800001, 2024-09-17T21:31:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039204184000000003, 2024-09-17T21:31:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12260 2024-09-17T21:31:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142107+0000", 2024-09-17T21:31:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:31:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142108+0000", 2024-09-17T21:31:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.753e-06 2024-09-17T21:31:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142114+0000", 2024-09-17T21:31:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7660000000000004e-06 2024-09-17T21:31:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142119+0000", 2024-09-17T21:31:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:31:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181216+0000", 2024-09-17T21:31:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039097191000000003 2024-09-17T21:31:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181260+0000", 2024-09-17T21:31:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4731000000000003e-05 2024-09-17T21:31:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181310+0000", 2024-09-17T21:31:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9818000000000003e-05 2024-09-17T21:31:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12261 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.508218984, 2024-09-17T21:31:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039292200999999999, 2024-09-17T21:31:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12261 2024-09-17T21:31:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142125+0000", 2024-09-17T21:31:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:31:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142127+0000", 2024-09-17T21:31:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.792e-06 2024-09-17T21:31:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142132+0000", 2024-09-17T21:31:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3979999999999998e-06 2024-09-17T21:31:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142136+0000", 2024-09-17T21:31:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.211e-06 2024-09-17T21:31:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181324+0000", 2024-09-17T21:31:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039187560000000003 2024-09-17T21:31:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181363+0000", 2024-09-17T21:31:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9375000000000002e-05 2024-09-17T21:31:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181417+0000", 2024-09-17T21:31:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3372000000000003e-05 2024-09-17T21:31:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12262 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.508201191, 2024-09-17T21:31:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039384875, 2024-09-17T21:31:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12262 2024-09-17T21:31:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142143+0000", 2024-09-17T21:31:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:31:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142148+0000", 2024-09-17T21:31:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4360000000000001e-06 2024-09-17T21:31:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142154+0000", 2024-09-17T21:31:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8719999999999999e-06 2024-09-17T21:31:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142158+0000", 2024-09-17T21:31:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.476e-06 2024-09-17T21:31:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181436+0000", 2024-09-17T21:31:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039277919000000001 2024-09-17T21:31:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181480+0000", 2024-09-17T21:31:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3649999999999997e-05 2024-09-17T21:31:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181527+0000", 2024-09-17T21:31:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7131e-05 2024-09-17T21:31:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12263 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1079296~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.50817875600001, 2024-09-17T21:31:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039473339000000003, 2024-09-17T21:31:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12263 2024-09-17T21:31:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:31:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142167+0000", 2024-09-17T21:31:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.81e-06 2024-09-17T21:31:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142172+0000", 2024-09-17T21:31:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:31:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142177+0000", 2024-09-17T21:31:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T21:31:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181553+0000", 2024-09-17T21:31:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039376130000000002 2024-09-17T21:31:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181592+0000", 2024-09-17T21:31:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9153999999999998e-05 2024-09-17T21:31:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181638+0000", 2024-09-17T21:31:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5864e-05 2024-09-17T21:31:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12278 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1095680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.508160781, 2024-09-17T21:31:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039454222999999997, 2024-09-17T21:31:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12278 2024-09-17T21:31:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:40.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:40.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:31:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142185+0000", 2024-09-17T21:31:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.697e-06 2024-09-17T21:31:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142191+0000", 2024-09-17T21:31:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:31:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142196+0000", 2024-09-17T21:31:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4829999999999999e-06 2024-09-17T21:31:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181548+0000", 2024-09-17T21:31:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039351200000000003 2024-09-17T21:31:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181591+0000", 2024-09-17T21:31:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3492000000000003e-05 2024-09-17T21:31:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181637+0000", 2024-09-17T21:31:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5847000000000003e-05 2024-09-17T21:31:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12279 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.508137492, 2024-09-17T21:31:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039539801999999999, 2024-09-17T21:31:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12279 2024-09-17T21:31:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.89e-07 2024-09-17T21:31:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142208+0000", 2024-09-17T21:31:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7579999999999999e-06 2024-09-17T21:31:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142214+0000", 2024-09-17T21:31:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:31:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142218+0000", 2024-09-17T21:31:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3429999999999998e-06 2024-09-17T21:31:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181653+0000", 2024-09-17T21:31:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039435322000000002 2024-09-17T21:31:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181698+0000", 2024-09-17T21:31:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4978999999999997e-05 2024-09-17T21:31:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181746+0000", 2024-09-17T21:31:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571999999999998e-05 2024-09-17T21:31:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12280 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1124352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.50811997300001, 2024-09-17T21:31:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039704358000000002, 2024-09-17T21:31:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12280 2024-09-17T21:31:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142224+0000", 2024-09-17T21:31:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:31:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142226+0000", 2024-09-17T21:31:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:31:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142231+0000", 2024-09-17T21:31:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:31:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142240+0000", 2024-09-17T21:31:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4590000000000004e-06 2024-09-17T21:31:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181774+0000", 2024-09-17T21:31:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039533833999999997 2024-09-17T21:31:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181859+0000", 2024-09-17T21:31:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4328999999999998e-05 2024-09-17T21:31:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181928+0000", 2024-09-17T21:31:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9139000000000003e-05 2024-09-17T21:31:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12281 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1136640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.50809706699999, 2024-09-17T21:31:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039787799999999998, 2024-09-17T21:31:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12281 2024-09-17T21:31:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142309+0000", 2024-09-17T21:31:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2323999999999995e-05 2024-09-17T21:31:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142317+0000", 2024-09-17T21:31:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1859999999999992e-06 2024-09-17T21:31:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142324+0000", 2024-09-17T21:31:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4699999999999996e-06 2024-09-17T21:31:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142330+0000", 2024-09-17T21:31:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4650000000000002e-06 2024-09-17T21:31:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181948+0000", 2024-09-17T21:31:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039618056999999998 2024-09-17T21:31:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181986+0000", 2024-09-17T21:31:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8442999999999998e-05 2024-09-17T21:31:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182034+0000", 2024-09-17T21:31:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7855000000000002e-05 2024-09-17T21:31:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12282 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1150976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.50800483800001, 2024-09-17T21:31:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039871919999999998, 2024-09-17T21:31:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12282 2024-09-17T21:31:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:31:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142341+0000", 2024-09-17T21:31:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.232e-06 2024-09-17T21:31:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142348+0000", 2024-09-17T21:31:40.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7320000000000001e-06 2024-09-17T21:31:40.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142353+0000", 2024-09-17T21:31:40.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:31:40.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182062+0000", 2024-09-17T21:31:40.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039709739000000001 2024-09-17T21:31:40.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182128+0000", 2024-09-17T21:31:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5739000000000002e-05 2024-09-17T21:31:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182210+0000", 2024-09-17T21:31:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:31:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12283 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 111.507984427, 2024-09-17T21:31:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039896434000000001, 2024-09-17T21:31:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:31:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12283 2024-09-17T21:31:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144852+0000", 2024-09-17T21:31:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024934079999999999 2024-09-17T21:31:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144856+0000", 2024-09-17T21:31:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:31:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144864+0000", 2024-09-17T21:31:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0160000000000008e-06 2024-09-17T21:31:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144869+0000", 2024-09-17T21:31:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:31:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182246+0000", 2024-09-17T21:31:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037377207000000003 2024-09-17T21:31:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182255+0000", 2024-09-17T21:31:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1840000000000005e-06 2024-09-17T21:31:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:40.746 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:40.747 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:31:40.747 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:40.744+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:40.848 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:40.845+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:40.891 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:40.891 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:40.891 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:40.891 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:40.900 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:31:40.948 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:40.946+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:41.004 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:41.004 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:41.004 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:41.004 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:41.004 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:41.012 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:31:41.053 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:41.051+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:41.150 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:41.148+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:41.157 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:41.157 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:41.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:41.157 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:41.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.157 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:36935 3.a 3:54c8759b:::rbd_data.1120e62eea89.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-17T21:31:41.157 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:31:41.157 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 104.195344816, 2024-09-17T21:31:41.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020540151999999999, 2024-09-17T21:31:41.158 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.158 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.158 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:41.158 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.158 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.158 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.158 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 36935 2024-09-17T21:31:41.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.158 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:31:41.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:31:41.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958910+0000", 2024-09-17T21:31:41.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:31:41.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958918+0000", 2024-09-17T21:31:41.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6469999999999998e-06 2024-09-17T21:31:41.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958933+0000", 2024-09-17T21:31:41.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4791e-05 2024-09-17T21:31:41.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958940+0000", 2024-09-17T21:31:41.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-17T21:31:41.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958990+0000", 2024-09-17T21:31:41.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9759000000000003e-05 2024-09-17T21:31:41.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.959039+0000", 2024-09-17T21:31:41.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9487000000000002e-05 2024-09-17T21:31:41.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:41.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.962098+0000", 2024-09-17T21:31:41.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003059258 2024-09-17T21:31:41.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:41.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979325+0000", 2024-09-17T21:31:41.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017226841 2024-09-17T21:31:41.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:41.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979343+0000", 2024-09-17T21:31:41.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7486000000000002e-05 2024-09-17T21:31:41.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:41.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979401+0000", 2024-09-17T21:31:41.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7980999999999999e-05 2024-09-17T21:31:41.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979449+0000", 2024-09-17T21:31:41.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8639e-05 2024-09-17T21:31:41.163 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.163 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.163 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.163 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39447 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 909312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.164 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:41.164 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186406915999996, 2024-09-17T21:31:41.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017419153999999999, 2024-09-17T21:31:41.164 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.164 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.164 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.164 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.164 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.164 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.164 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39447 2024-09-17T21:31:41.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.164 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:41.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:41.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:41.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:31:41.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967849+0000", 2024-09-17T21:31:41.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4339999999999999e-06 2024-09-17T21:31:41.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967853+0000", 2024-09-17T21:31:41.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:31:41.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967857+0000", 2024-09-17T21:31:41.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:31:41.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985193+0000", 2024-09-17T21:31:41.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017335973000000001 2024-09-17T21:31:41.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985228+0000", 2024-09-17T21:31:41.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5154999999999998e-05 2024-09-17T21:31:41.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985266+0000", 2024-09-17T21:31:41.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8448000000000002e-05 2024-09-17T21:31:41.168 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.168 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.168 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.168 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39448 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 974848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.168 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:41.168 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186394100000001, 2024-09-17T21:31:41.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017517066000000001, 2024-09-17T21:31:41.168 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.168 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.168 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.169 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.169 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.169 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39448 2024-09-17T21:31:41.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.169 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:41.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:41.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:41.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.72e-07 2024-09-17T21:31:41.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967861+0000", 2024-09-17T21:31:41.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.096e-06 2024-09-17T21:31:41.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967865+0000", 2024-09-17T21:31:41.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9339999999999999e-06 2024-09-17T21:31:41.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967868+0000", 2024-09-17T21:31:41.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.508e-06 2024-09-17T21:31:41.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985285+0000", 2024-09-17T21:31:41.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017417208 2024-09-17T21:31:41.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985339+0000", 2024-09-17T21:31:41.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3931000000000003e-05 2024-09-17T21:31:41.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985377+0000", 2024-09-17T21:31:41.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8016999999999999e-05 2024-09-17T21:31:41.172 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.172 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.172 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.172 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39449 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.172 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:41.173 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186381990000001, 2024-09-17T21:31:41.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017595137, 2024-09-17T21:31:41.173 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.173 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.173 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.173 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.173 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.173 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.173 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39449 2024-09-17T21:31:41.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.173 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:41.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:41.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:41.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2700000000000001e-07 2024-09-17T21:31:41.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967873+0000", 2024-09-17T21:31:41.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.164e-06 2024-09-17T21:31:41.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967877+0000", 2024-09-17T21:31:41.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.23e-06 2024-09-17T21:31:41.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967880+0000", 2024-09-17T21:31:41.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.286e-06 2024-09-17T21:31:41.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985397+0000", 2024-09-17T21:31:41.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017516982 2024-09-17T21:31:41.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985431+0000", 2024-09-17T21:31:41.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3899999999999997e-05 2024-09-17T21:31:41.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985467+0000", 2024-09-17T21:31:41.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6347999999999998e-05 2024-09-17T21:31:41.176 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.176 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.177 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.177 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39450 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1105920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.177 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:31:41.177 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186361732999998, 2024-09-17T21:31:41.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017677824000000002, 2024-09-17T21:31:41.177 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.177 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.177 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.177 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.177 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.177 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.177 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39450 2024-09-17T21:31:41.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.178 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:31:41.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:31:41.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967893+0000", 2024-09-17T21:31:41.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9100000000000004e-07 2024-09-17T21:31:41.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967895+0000", 2024-09-17T21:31:41.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:31:41.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967901+0000", 2024-09-17T21:31:41.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.083e-06 2024-09-17T21:31:41.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967906+0000", 2024-09-17T21:31:41.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:31:41.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985493+0000", 2024-09-17T21:31:41.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017587565999999999 2024-09-17T21:31:41.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985531+0000", 2024-09-17T21:31:41.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7490000000000002e-05 2024-09-17T21:31:41.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985570+0000", 2024-09-17T21:31:41.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9205000000000002e-05 2024-09-17T21:31:41.181 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.181 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.181 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.181 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39451 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1171456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.181 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:41.181 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186341353000003, 2024-09-17T21:31:41.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017752337999999999, 2024-09-17T21:31:41.181 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.181 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.181 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.182 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.182 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.182 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.182 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39451 2024-09-17T21:31:41.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.182 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:41.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:41.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:41.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:31:41.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967915+0000", 2024-09-17T21:31:41.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780000000000001e-06 2024-09-17T21:31:41.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967922+0000", 2024-09-17T21:31:41.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7020000000000004e-06 2024-09-17T21:31:41.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967926+0000", 2024-09-17T21:31:41.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.809e-06 2024-09-17T21:31:41.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985590+0000", 2024-09-17T21:31:41.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017663913999999999 2024-09-17T21:31:41.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985627+0000", 2024-09-17T21:31:41.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6896e-05 2024-09-17T21:31:41.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985665+0000", 2024-09-17T21:31:41.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7793999999999998e-05 2024-09-17T21:31:41.185 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.185 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.185 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.185 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39452 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1236992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.185 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:41.185 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186321410999994, 2024-09-17T21:31:41.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017829602999999999, 2024-09-17T21:31:41.185 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.186 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.186 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.186 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.186 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.186 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.186 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39452 2024-09-17T21:31:41.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.186 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:41.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:41.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:41.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:31:41.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967947+0000", 2024-09-17T21:31:41.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4280999999999999e-05 2024-09-17T21:31:41.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967954+0000", 2024-09-17T21:31:41.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3249999999999996e-06 2024-09-17T21:31:41.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967959+0000", 2024-09-17T21:31:41.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2299999999999999e-06 2024-09-17T21:31:41.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985687+0000", 2024-09-17T21:31:41.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017727844999999999 2024-09-17T21:31:41.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985725+0000", 2024-09-17T21:31:41.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8243e-05 2024-09-17T21:31:41.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985762+0000", 2024-09-17T21:31:41.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7023000000000001e-05 2024-09-17T21:31:41.189 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.189 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.189 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.190 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39453 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.190 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:31:41.190 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186284923000002, 2024-09-17T21:31:41.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017893793000000002, 2024-09-17T21:31:41.190 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.190 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.190 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.190 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.190 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.190 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.190 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39453 2024-09-17T21:31:41.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.190 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:31:41.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:31:41.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967970+0000", 2024-09-17T21:31:41.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:31:41.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967972+0000", 2024-09-17T21:31:41.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0229999999999999e-06 2024-09-17T21:31:41.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967978+0000", 2024-09-17T21:31:41.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0430000000000004e-06 2024-09-17T21:31:41.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967982+0000", 2024-09-17T21:31:41.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2570000000000001e-06 2024-09-17T21:31:41.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985782+0000", 2024-09-17T21:31:41.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017799683 2024-09-17T21:31:41.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985818+0000", 2024-09-17T21:31:41.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6304999999999999e-05 2024-09-17T21:31:41.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985863+0000", 2024-09-17T21:31:41.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4953999999999997e-05 2024-09-17T21:31:41.193 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.193 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.193 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.194 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39454 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1368064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.194 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:31:41.194 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186265594999995, 2024-09-17T21:31:41.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017968886, 2024-09-17T21:31:41.194 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.194 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.194 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.194 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.194 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.194 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.194 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39454 2024-09-17T21:31:41.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.195 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:31:41.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:31:41.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967989+0000", 2024-09-17T21:31:41.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.15e-07 2024-09-17T21:31:41.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967991+0000", 2024-09-17T21:31:41.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.945e-06 2024-09-17T21:31:41.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967997+0000", 2024-09-17T21:31:41.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0190000000000003e-06 2024-09-17T21:31:41.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968001+0000", 2024-09-17T21:31:41.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6650000000000002e-06 2024-09-17T21:31:41.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985883+0000", 2024-09-17T21:31:41.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017881971999999999 2024-09-17T21:31:41.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985920+0000", 2024-09-17T21:31:41.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-05 2024-09-17T21:31:41.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985957+0000", 2024-09-17T21:31:41.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7370000000000003e-05 2024-09-17T21:31:41.198 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.198 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.198 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.198 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39455 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.198 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:41.198 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186246421000007, 2024-09-17T21:31:41.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018044437999999999, 2024-09-17T21:31:41.198 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.198 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.198 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.199 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.199 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.199 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.199 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39455 2024-09-17T21:31:41.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.199 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:41.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:41.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:41.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:31:41.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968010+0000", 2024-09-17T21:31:41.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.886e-06 2024-09-17T21:31:41.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968017+0000", 2024-09-17T21:31:41.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:31:41.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968021+0000", 2024-09-17T21:31:41.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850000000000004e-06 2024-09-17T21:31:41.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985977+0000", 2024-09-17T21:31:41.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017956368 2024-09-17T21:31:41.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986015+0000", 2024-09-17T21:31:41.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7610000000000001e-05 2024-09-17T21:31:41.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986052+0000", 2024-09-17T21:31:41.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7109e-05 2024-09-17T21:31:41.202 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.202 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.202 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.203 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39456 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1499136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.203 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:41.203 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186226121000004, 2024-09-17T21:31:41.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018114514000000002, 2024-09-17T21:31:41.203 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.203 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.203 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.203 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.203 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.203 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.203 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39456 2024-09-17T21:31:41.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.203 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:41.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:41.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:41.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-17T21:31:41.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968030+0000", 2024-09-17T21:31:41.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0049999999999999e-06 2024-09-17T21:31:41.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968036+0000", 2024-09-17T21:31:41.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0129999999999999e-06 2024-09-17T21:31:41.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968041+0000", 2024-09-17T21:31:41.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8300000000000003e-06 2024-09-17T21:31:41.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986071+0000", 2024-09-17T21:31:41.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018029620999999999 2024-09-17T21:31:41.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986106+0000", 2024-09-17T21:31:41.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4867999999999999e-05 2024-09-17T21:31:41.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986142+0000", 2024-09-17T21:31:41.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6566000000000002e-05 2024-09-17T21:31:41.207 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.207 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.207 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.207 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39457 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1564672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.207 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:41.207 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186206304999999, 2024-09-17T21:31:41.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018188644, 2024-09-17T21:31:41.207 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.207 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.207 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.207 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.207 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.208 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.208 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39457 2024-09-17T21:31:41.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.208 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:41.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:41.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:41.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7199999999999999e-07 2024-09-17T21:31:41.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968050+0000", 2024-09-17T21:31:41.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1110000000000002e-06 2024-09-17T21:31:41.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968057+0000", 2024-09-17T21:31:41.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6170000000000003e-06 2024-09-17T21:31:41.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968062+0000", 2024-09-17T21:31:41.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:31:41.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986163+0000", 2024-09-17T21:31:41.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018101435999999999 2024-09-17T21:31:41.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986199+0000", 2024-09-17T21:31:41.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.591e-05 2024-09-17T21:31:41.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986236+0000", 2024-09-17T21:31:41.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7416999999999998e-05 2024-09-17T21:31:41.211 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.211 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.211 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.211 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39458 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1630208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.211 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:41.211 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186179365000001, 2024-09-17T21:31:41.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018255856000000001, 2024-09-17T21:31:41.211 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.212 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.212 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.212 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.212 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.212 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.212 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39458 2024-09-17T21:31:41.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.212 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:41.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:41.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:41.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:31:41.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968077+0000", 2024-09-17T21:31:41.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.779e-06 2024-09-17T21:31:41.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968081+0000", 2024-09-17T21:31:41.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.532e-06 2024-09-17T21:31:41.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968085+0000", 2024-09-17T21:31:41.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999999e-06 2024-09-17T21:31:41.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986256+0000", 2024-09-17T21:31:41.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018171273000000002 2024-09-17T21:31:41.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986293+0000", 2024-09-17T21:31:41.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6534e-05 2024-09-17T21:31:41.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986330+0000", 2024-09-17T21:31:41.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7536000000000001e-05 2024-09-17T21:31:41.215 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.216 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.216 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.216 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39459 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1695744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.216 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:31:41.216 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186164786999996, 2024-09-17T21:31:41.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334493, 2024-09-17T21:31:41.216 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.216 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.216 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.216 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.216 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.216 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.217 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39459 2024-09-17T21:31:41.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.217 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:31:41.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:31:41.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968090+0000", 2024-09-17T21:31:41.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:31:41.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968091+0000", 2024-09-17T21:31:41.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3319999999999999e-06 2024-09-17T21:31:41.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968094+0000", 2024-09-17T21:31:41.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2590000000000001e-06 2024-09-17T21:31:41.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968097+0000", 2024-09-17T21:31:41.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:31:41.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.219 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986350+0000", 2024-09-17T21:31:41.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018253364000000001 2024-09-17T21:31:41.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.219 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986387+0000", 2024-09-17T21:31:41.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6350999999999998e-05 2024-09-17T21:31:41.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.219 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986424+0000", 2024-09-17T21:31:41.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6897999999999998e-05 2024-09-17T21:31:41.220 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.220 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.220 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.220 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39460 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1761280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.220 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:31:41.220 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186153650999998, 2024-09-17T21:31:41.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018415918999999999, 2024-09-17T21:31:41.220 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.220 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.220 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.220 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.220 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.220 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.221 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39460 2024-09-17T21:31:41.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.221 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:31:41.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:31:41.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968101+0000", 2024-09-17T21:31:41.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T21:31:41.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968102+0000", 2024-09-17T21:31:41.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1689999999999999e-06 2024-09-17T21:31:41.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968105+0000", 2024-09-17T21:31:41.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:31:41.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968108+0000", 2024-09-17T21:31:41.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6630000000000002e-06 2024-09-17T21:31:41.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986442+0000", 2024-09-17T21:31:41.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334487 2024-09-17T21:31:41.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986479+0000", 2024-09-17T21:31:41.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6557000000000003e-05 2024-09-17T21:31:41.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986516+0000", 2024-09-17T21:31:41.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7648999999999998e-05 2024-09-17T21:31:41.224 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.224 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.224 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.224 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39461 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1826816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.224 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:41.224 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186142891000003, 2024-09-17T21:31:41.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018499768, 2024-09-17T21:31:41.224 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.224 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.224 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.224 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.224 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.225 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.225 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39461 2024-09-17T21:31:41.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.225 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:41.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:41.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:41.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4699999999999998e-07 2024-09-17T21:31:41.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968112+0000", 2024-09-17T21:31:41.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-06 2024-09-17T21:31:41.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968116+0000", 2024-09-17T21:31:41.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1080000000000001e-06 2024-09-17T21:31:41.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968122+0000", 2024-09-17T21:31:41.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1070000000000001e-06 2024-09-17T21:31:41.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986537+0000", 2024-09-17T21:31:41.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018414834000000001 2024-09-17T21:31:41.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986572+0000", 2024-09-17T21:31:41.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5726000000000003e-05 2024-09-17T21:31:41.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986611+0000", 2024-09-17T21:31:41.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8584000000000002e-05 2024-09-17T21:31:41.228 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.228 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.228 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.228 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39462 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1892352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.228 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:31:41.228 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186128526999994, 2024-09-17T21:31:41.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018583252000000001, 2024-09-17T21:31:41.228 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.228 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.228 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.228 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.229 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.229 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.229 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39462 2024-09-17T21:31:41.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.229 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:31:41.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:31:41.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968126+0000", 2024-09-17T21:31:41.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:41.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968127+0000", 2024-09-17T21:31:41.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.229e-06 2024-09-17T21:31:41.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968131+0000", 2024-09-17T21:31:41.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8010000000000001e-06 2024-09-17T21:31:41.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968134+0000", 2024-09-17T21:31:41.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8880000000000001e-06 2024-09-17T21:31:41.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986634+0000", 2024-09-17T21:31:41.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018500501999999999 2024-09-17T21:31:41.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986671+0000", 2024-09-17T21:31:41.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6881000000000001e-05 2024-09-17T21:31:41.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986709+0000", 2024-09-17T21:31:41.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7546999999999998e-05 2024-09-17T21:31:41.232 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.232 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.232 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.232 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39463 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1957888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.232 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:41.233 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186116912000003, 2024-09-17T21:31:41.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018667186999999998, 2024-09-17T21:31:41.233 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.233 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.233 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.233 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.233 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.233 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.233 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39463 2024-09-17T21:31:41.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.233 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:41.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:41.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:41.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:31:41.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968138+0000", 2024-09-17T21:31:41.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:31:41.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968142+0000", 2024-09-17T21:31:41.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.145e-06 2024-09-17T21:31:41.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968144+0000", 2024-09-17T21:31:41.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6869999999999999e-06 2024-09-17T21:31:41.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986728+0000", 2024-09-17T21:31:41.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018584163000000001 2024-09-17T21:31:41.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986766+0000", 2024-09-17T21:31:41.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7987000000000001e-05 2024-09-17T21:31:41.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986804+0000", 2024-09-17T21:31:41.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7892000000000002e-05 2024-09-17T21:31:41.236 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.236 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.236 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.237 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39464 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.237 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:31:41.237 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186106602999999, 2024-09-17T21:31:41.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018753723999999999, 2024-09-17T21:31:41.237 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.237 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.237 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.237 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.237 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.237 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.237 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39464 2024-09-17T21:31:41.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:31:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:31:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968148+0000", 2024-09-17T21:31:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:31:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968152+0000", 2024-09-17T21:31:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3749999999999996e-06 2024-09-17T21:31:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968155+0000", 2024-09-17T21:31:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.371e-06 2024-09-17T21:31:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968158+0000", 2024-09-17T21:31:41.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T21:31:41.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986824+0000", 2024-09-17T21:31:41.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018665779 2024-09-17T21:31:41.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986863+0000", 2024-09-17T21:31:41.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9273000000000002e-05 2024-09-17T21:31:41.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986901+0000", 2024-09-17T21:31:41.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7978000000000001e-05 2024-09-17T21:31:41.241 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.241 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.241 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.241 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39465 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:41.241 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:41.241 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.186091203999993, 2024-09-17T21:31:41.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018825341999999998, 2024-09-17T21:31:41.241 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:41.241 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:41.241 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:41.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:41.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:41.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:41.242 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39465 2024-09-17T21:31:41.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.242 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:41.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:41.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:41.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:41.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:41.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:41.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:41.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:41.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-17T21:31:41.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:41.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968164+0000", 2024-09-17T21:31:41.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1289999999999999e-06 2024-09-17T21:31:41.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:41.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968167+0000", 2024-09-17T21:31:41.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1379999999999999e-06 2024-09-17T21:31:41.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:41.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968170+0000", 2024-09-17T21:31:41.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:31:41.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:41.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986914+0000", 2024-09-17T21:31:41.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018743829 2024-09-17T21:31:41.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:41.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986950+0000", 2024-09-17T21:31:41.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5819999999999999e-05 2024-09-17T21:31:41.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:41.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:41.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:41.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986988+0000", 2024-09-17T21:31:41.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8362000000000003e-05 2024-09-17T21:31:41.245 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.245 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.245 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.245 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:41.245 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:41.245 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:41.245 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:31:41.246 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:31:41.290 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:41.288+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:41.350 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:41.348+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:41.351 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:41.351 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:41.351 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:41.351 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:41.360 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:31:41.368 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:41.368 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:41.384 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:41.385 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:41.400 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:41.400 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:41.416 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:41.416 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:41.438 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:41.438 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:41.451 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:41.449+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:41.455 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:41.456 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:41.473 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:41.473 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:41.490 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:41.490 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:41.517 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:41.517 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:41.527 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:31:41.571 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:41.569+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:41.631 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:41.631 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:41.640 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:31:41.681 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:41.679+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:41.743 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:41.744 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:41.752 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:31:41.754 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:41.751+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:41.855 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:41.852+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:41.855 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:41.856 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:41.864 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:31:41.955 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:41.952+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:42.007 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:42.007 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:42.015 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:31:42.058 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:42.055+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:42.157 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:42.155+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:42.160 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:42.160 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:42.170 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:31:42.258 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:42.256+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:42.314 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:42.314 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:42.323 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:31:42.359 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:42.357+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:42.434 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:31:42.460 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:42.458+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:42.479 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:42.479 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:42.489 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:31:42.561 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:42.558+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:42.598 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:42.598 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:42.608 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:31:42.692 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:42.690+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:42.753 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:42.753 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:42.762 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:31:42.763 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:42.761+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:42.864 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:42.861+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:42.906 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:42.906 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:42.915 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:31:42.999 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:42.997+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:43.059 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:43.059 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:43.066 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:43.064+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:43.067 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:31:43.167 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:43.165+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:43.211 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:43.211 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:43.220 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:31:43.304 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:43.302+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:43.364 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:43.364 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:43.369 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:43.366+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:43.372 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:31:43.422 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is set 2024-09-17T21:31:43.470 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T21:31:43.468+0000 7f581b460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33368) UID: 0 2024-09-17T21:31:43.502 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:31:43.506 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:43.506 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:43.516 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:31:43.602 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:43.600+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:43.637 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0: {} 2024-09-17T21:31:43.637 INFO:teuthology.orchestra.run.smithi129.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:31:43.653 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1: {} 2024-09-17T21:31:43.654 INFO:teuthology.orchestra.run.smithi129.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:31:43.661 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T21:31:43.661 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:31:43.669 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2: {} 2024-09-17T21:31:43.669 INFO:teuthology.orchestra.run.smithi129.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:31:43.670 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:31:43.671 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:43.669+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:43.685 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3: {} 2024-09-17T21:31:43.685 INFO:teuthology.orchestra.run.smithi129.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:31:43.688 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-17T21:31:43.689 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:31:43.689 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-17T21:31:43.689 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T21:31:43.689 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T21:31:43.689 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-17T21:31:43.702 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4: {} 2024-09-17T21:31:43.702 INFO:teuthology.orchestra.run.smithi129.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:31:43.718 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5: {} 2024-09-17T21:31:43.718 INFO:teuthology.orchestra.run.smithi129.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:31:43.735 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6: {} 2024-09-17T21:31:43.735 INFO:teuthology.orchestra.run.smithi129.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T21:31:43.751 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7: {} 2024-09-17T21:31:43.752 INFO:teuthology.orchestra.run.smithi129.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T21:31:43.773 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:43.771+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:43.816 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:43.816 INFO:teuthology.orchestra.run.smithi171.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T21:31:43.816 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:43.818 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:43.818 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:43.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:43.818 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:43.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.818 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21342 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2082816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:31:43.818 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:43.818 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 114.13830818, 2024-09-17T21:31:43.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022083523000000001, 2024-09-17T21:31:43.818 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.818 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.818 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:43.819 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:43.819 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:43.819 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:43.819 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21342 2024-09-17T21:31:43.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.819 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:43.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:43.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676658+0000", 2024-09-17T21:31:43.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:31:43.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676659+0000", 2024-09-17T21:31:43.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0950000000000001e-06 2024-09-17T21:31:43.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676662+0000", 2024-09-17T21:31:43.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.971e-06 2024-09-17T21:31:43.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676665+0000", 2024-09-17T21:31:43.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6189999999999998e-06 2024-09-17T21:31:43.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698568+0000", 2024-09-17T21:31:43.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021903549000000001 2024-09-17T21:31:43.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698668+0000", 2024-09-17T21:31:43.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9778999999999995e-05 2024-09-17T21:31:43.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.698741+0000", 2024-09-17T21:31:43.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000006e-05 2024-09-17T21:31:43.822 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.822 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.822 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.823 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:21343 3.1a 3:596c52af:::rbd_data.11202c1b6cb4.000000000000001d:4 [read 2095104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T21:31:43.823 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:43.823 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 114.138298242, 2024-09-17T21:31:43.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022504024000000001, 2024-09-17T21:31:43.823 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.823 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.823 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:43.823 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:43.823 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:43.823 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:43.823 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 21343 2024-09-17T21:31:43.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.823 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:43.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:43.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676668+0000", 2024-09-17T21:31:43.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T21:31:43.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676669+0000", 2024-09-17T21:31:43.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.003e-06 2024-09-17T21:31:43.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676672+0000", 2024-09-17T21:31:43.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9280000000000002e-06 2024-09-17T21:31:43.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.676674+0000", 2024-09-17T21:31:43.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5469999999999999e-06 2024-09-17T21:31:43.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699015+0000", 2024-09-17T21:31:43.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022340563000000001 2024-09-17T21:31:43.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699102+0000", 2024-09-17T21:31:43.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7425999999999999e-05 2024-09-17T21:31:43.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.699172+0000", 2024-09-17T21:31:43.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9346999999999999e-05 2024-09-17T21:31:43.827 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.827 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.827 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.827 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45576 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:43.827 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:43.827 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.110064441999995, 2024-09-17T21:31:43.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022024660000000001, 2024-09-17T21:31:43.827 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.828 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.828 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:43.828 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:43.828 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:43.828 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:43.828 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45576 2024-09-17T21:31:43.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.828 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:43.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704901+0000", 2024-09-17T21:31:43.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704902+0000", 2024-09-17T21:31:43.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9100000000000004e-07 2024-09-17T21:31:43.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704905+0000", 2024-09-17T21:31:43.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6139999999999999e-06 2024-09-17T21:31:43.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704911+0000", 2024-09-17T21:31:43.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8619999999999997e-06 2024-09-17T21:31:43.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704917+0000", 2024-09-17T21:31:43.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:31:43.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726681+0000", 2024-09-17T21:31:43.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021764569000000001 2024-09-17T21:31:43.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726726+0000", 2024-09-17T21:31:43.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4218999999999998e-05 2024-09-17T21:31:43.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726926+0000", 2024-09-17T21:31:43.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000200466 2024-09-17T21:31:43.831 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.831 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.831 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.832 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45577 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:43.832 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:43.832 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.110040785999999, 2024-09-17T21:31:43.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022269694999999999, 2024-09-17T21:31:43.832 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.832 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.832 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:43.832 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:43.832 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:43.832 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:43.832 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45577 2024-09-17T21:31:43.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.833 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:43.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704925+0000", 2024-09-17T21:31:43.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704926+0000", 2024-09-17T21:31:43.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:31:43.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704928+0000", 2024-09-17T21:31:43.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9989999999999998e-06 2024-09-17T21:31:43.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704934+0000", 2024-09-17T21:31:43.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-17T21:31:43.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704939+0000", 2024-09-17T21:31:43.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5909999999999998e-06 2024-09-17T21:31:43.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726954+0000", 2024-09-17T21:31:43.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022014885000000001 2024-09-17T21:31:43.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.726995+0000", 2024-09-17T21:31:43.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1474000000000003e-05 2024-09-17T21:31:43.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727195+0000", 2024-09-17T21:31:43.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019968300000000001 2024-09-17T21:31:43.836 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.836 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.836 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.836 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45578 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:43.836 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:43.836 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.110019542000003, 2024-09-17T21:31:43.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022484006000000001, 2024-09-17T21:31:43.836 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.836 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.837 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:43.837 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:43.837 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:43.837 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:43.837 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45578 2024-09-17T21:31:43.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.837 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:43.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704946+0000", 2024-09-17T21:31:43.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704947+0000", 2024-09-17T21:31:43.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1399999999999997e-07 2024-09-17T21:31:43.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704949+0000", 2024-09-17T21:31:43.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0959999999999999e-06 2024-09-17T21:31:43.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704956+0000", 2024-09-17T21:31:43.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9330000000000004e-06 2024-09-17T21:31:43.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704961+0000", 2024-09-17T21:31:43.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6759999999999998e-06 2024-09-17T21:31:43.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727213+0000", 2024-09-17T21:31:43.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022252568 2024-09-17T21:31:43.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727259+0000", 2024-09-17T21:31:43.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5238000000000003e-05 2024-09-17T21:31:43.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727430+0000", 2024-09-17T21:31:43.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000171881 2024-09-17T21:31:43.840 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.840 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.840 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.841 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45579 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:43.841 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:43.841 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.109997613999994, 2024-09-17T21:31:43.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022749080000000001, 2024-09-17T21:31:43.841 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.841 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.841 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:43.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:43.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:43.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:43.841 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45579 2024-09-17T21:31:43.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.842 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:43.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704968+0000", 2024-09-17T21:31:43.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704969+0000", 2024-09-17T21:31:43.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:31:43.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704971+0000", 2024-09-17T21:31:43.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.097e-06 2024-09-17T21:31:43.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704977+0000", 2024-09-17T21:31:43.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6329999999999999e-06 2024-09-17T21:31:43.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.704983+0000", 2024-09-17T21:31:43.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:31:43.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727457+0000", 2024-09-17T21:31:43.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022474521000000001 2024-09-17T21:31:43.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727498+0000", 2024-09-17T21:31:43.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0558e-05 2024-09-17T21:31:43.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.727717+0000", 2024-09-17T21:31:43.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219805 2024-09-17T21:31:43.845 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.845 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.845 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.845 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45652 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:43.845 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:43.845 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.108851475999998, 2024-09-17T21:31:43.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022100465999999999, 2024-09-17T21:31:43.846 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.846 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.846 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:43.846 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:43.846 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:43.846 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:43.846 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45652 2024-09-17T21:31:43.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.846 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:43.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706114+0000", 2024-09-17T21:31:43.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706115+0000", 2024-09-17T21:31:43.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:31:43.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706117+0000", 2024-09-17T21:31:43.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-06 2024-09-17T21:31:43.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706122+0000", 2024-09-17T21:31:43.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.288e-06 2024-09-17T21:31:43.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706136+0000", 2024-09-17T21:31:43.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3409e-05 2024-09-17T21:31:43.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728112+0000", 2024-09-17T21:31:43.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021976724999999999 2024-09-17T21:31:43.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728151+0000", 2024-09-17T21:31:43.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8318999999999997e-05 2024-09-17T21:31:43.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728215+0000", 2024-09-17T21:31:43.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4172999999999997e-05 2024-09-17T21:31:43.850 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.850 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.850 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.850 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45653 3.14 3:2c9ba1e7:::rbd_data.1120d7424d8b.0000000000000015:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:43.850 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:43.850 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.108822266000004, 2024-09-17T21:31:43.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022249517999999999, 2024-09-17T21:31:43.850 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.850 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.850 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:43.851 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:43.851 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:43.851 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:43.851 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45653 2024-09-17T21:31:43.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.851 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:43.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:43.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706144+0000", 2024-09-17T21:31:43.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T21:31:43.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706147+0000", 2024-09-17T21:31:43.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4430000000000002e-06 2024-09-17T21:31:43.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706152+0000", 2024-09-17T21:31:43.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5350000000000004e-06 2024-09-17T21:31:43.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706157+0000", 2024-09-17T21:31:43.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8420000000000004e-06 2024-09-17T21:31:43.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728246+0000", 2024-09-17T21:31:43.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022089237000000001 2024-09-17T21:31:43.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728309+0000", 2024-09-17T21:31:43.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2356999999999999e-05 2024-09-17T21:31:43.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:31:43.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4596000000000001e-05 2024-09-17T21:31:43.854 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.854 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.854 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.855 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45657 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 36864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:43.855 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:43.855 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.108791002000004, 2024-09-17T21:31:43.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022218252000000001, 2024-09-17T21:31:43.855 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.855 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.855 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:43.855 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:43.855 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:43.855 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:43.855 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45657 2024-09-17T21:31:43.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706175+0000", 2024-09-17T21:31:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8299999999999997e-07 2024-09-17T21:31:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706178+0000", 2024-09-17T21:31:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3700000000000002e-06 2024-09-17T21:31:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706184+0000", 2024-09-17T21:31:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.144e-06 2024-09-17T21:31:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706190+0000", 2024-09-17T21:31:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8590000000000001e-06 2024-09-17T21:31:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728240+0000", 2024-09-17T21:31:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022049804999999999 2024-09-17T21:31:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728307+0000", 2024-09-17T21:31:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7477999999999993e-05 2024-09-17T21:31:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728393+0000", 2024-09-17T21:31:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6112999999999994e-05 2024-09-17T21:31:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45658 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.108767881999995, 2024-09-17T21:31:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022376455, 2024-09-17T21:31:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45658 2024-09-17T21:31:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706198+0000", 2024-09-17T21:31:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:31:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706200+0000", 2024-09-17T21:31:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:31:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706206+0000", 2024-09-17T21:31:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7039999999999999e-06 2024-09-17T21:31:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.706211+0000", 2024-09-17T21:31:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9429999999999999e-06 2024-09-17T21:31:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728418+0000", 2024-09-17T21:31:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022206760999999998 2024-09-17T21:31:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728497+0000", 2024-09-17T21:31:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9493999999999995e-05 2024-09-17T21:31:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.728574+0000", 2024-09-17T21:31:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7180000000000003e-05 2024-09-17T21:31:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45707 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 212992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.107063815999993, 2024-09-17T21:31:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022235550999999999, 2024-09-17T21:31:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45707 2024-09-17T21:31:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707902+0000", 2024-09-17T21:31:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:31:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707905+0000", 2024-09-17T21:31:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2120000000000002e-06 2024-09-17T21:31:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707911+0000", 2024-09-17T21:31:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1929999999999998e-06 2024-09-17T21:31:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707915+0000", 2024-09-17T21:31:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5639999999999998e-06 2024-09-17T21:31:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729897+0000", 2024-09-17T21:31:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021981186 2024-09-17T21:31:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.729944+0000", 2024-09-17T21:31:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571000000000002e-05 2024-09-17T21:31:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730138+0000", 2024-09-17T21:31:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000193451 2024-09-17T21:31:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45708 3.14 3:29723fad:::rbd_data.1120d7424d8b.0000000000000005:e [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.107043684000004, 2024-09-17T21:31:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022533742999999998, 2024-09-17T21:31:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45708 2024-09-17T21:31:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707922+0000", 2024-09-17T21:31:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707923+0000", 2024-09-17T21:31:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:31:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707932+0000", 2024-09-17T21:31:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6430000000000007e-06 2024-09-17T21:31:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707939+0000", 2024-09-17T21:31:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3539999999999996e-06 2024-09-17T21:31:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.707944+0000", 2024-09-17T21:31:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3249999999999998e-06 2024-09-17T21:31:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730176+0000", 2024-09-17T21:31:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022232393 2024-09-17T21:31:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730263+0000", 2024-09-17T21:31:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6638999999999995e-05 2024-09-17T21:31:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730456+0000", 2024-09-17T21:31:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019304899999999999 2024-09-17T21:31:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45752 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.106788464000005, 2024-09-17T21:31:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022426878000000001, 2024-09-17T21:31:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45752 2024-09-17T21:31:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708177+0000", 2024-09-17T21:31:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708178+0000", 2024-09-17T21:31:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:31:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708181+0000", 2024-09-17T21:31:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.909e-06 2024-09-17T21:31:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708187+0000", 2024-09-17T21:31:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4999999999999996e-06 2024-09-17T21:31:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708193+0000", 2024-09-17T21:31:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:31:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730484+0000", 2024-09-17T21:31:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022291765000000002 2024-09-17T21:31:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730541+0000", 2024-09-17T21:31:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6483000000000001e-05 2024-09-17T21:31:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730604+0000", 2024-09-17T21:31:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3429000000000005e-05 2024-09-17T21:31:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45761 3.14 3:2e91d223:::rbd_data.1120ab27bfd0.0000000000000005:c [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.106760554000005, 2024-09-17T21:31:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022614408999999999, 2024-09-17T21:31:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45761 2024-09-17T21:31:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708205+0000", 2024-09-17T21:31:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708206+0000", 2024-09-17T21:31:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4700000000000001e-07 2024-09-17T21:31:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708208+0000", 2024-09-17T21:31:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1509999999999998e-06 2024-09-17T21:31:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708214+0000", 2024-09-17T21:31:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.49e-06 2024-09-17T21:31:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.708219+0000", 2024-09-17T21:31:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8119999999999998e-06 2024-09-17T21:31:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730641+0000", 2024-09-17T21:31:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022422765000000001 2024-09-17T21:31:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730731+0000", 2024-09-17T21:31:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9598999999999994e-05 2024-09-17T21:31:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.730820+0000", 2024-09-17T21:31:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8844999999999994e-05 2024-09-17T21:31:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45785 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.105538766999999, 2024-09-17T21:31:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022044113000000001, 2024-09-17T21:31:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45785 2024-09-17T21:31:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709427+0000", 2024-09-17T21:31:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709428+0000", 2024-09-17T21:31:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000002e-07 2024-09-17T21:31:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709430+0000", 2024-09-17T21:31:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:31:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709451+0000", 2024-09-17T21:31:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1372000000000001e-05 2024-09-17T21:31:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709472+0000", 2024-09-17T21:31:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1007999999999999e-05 2024-09-17T21:31:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731293+0000", 2024-09-17T21:31:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021821320000000002 2024-09-17T21:31:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731378+0000", 2024-09-17T21:31:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5059e-05 2024-09-17T21:31:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731471+0000", 2024-09-17T21:31:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2867000000000004e-05 2024-09-17T21:31:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:45786 3.1c 3:3ef3c765:::rbd_data.1120d7424d8b.0000000000000017:e [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-17T21:31:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.105479970000005, 2024-09-17T21:31:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022181821000000001, 2024-09-17T21:31:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 45786 2024-09-17T21:31:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709486+0000", 2024-09-17T21:31:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709487+0000", 2024-09-17T21:31:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1299999999999999e-07 2024-09-17T21:31:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709491+0000", 2024-09-17T21:31:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2089999999999999e-06 2024-09-17T21:31:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709498+0000", 2024-09-17T21:31:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3980000000000002e-06 2024-09-17T21:31:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.709504+0000", 2024-09-17T21:31:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7219999999999996e-06 2024-09-17T21:31:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731499+0000", 2024-09-17T21:31:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021995297 2024-09-17T21:31:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731591+0000", 2024-09-17T21:31:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1373999999999996e-05 2024-09-17T21:31:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:06.731668+0000", 2024-09-17T21:31:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7107999999999994e-05 2024-09-17T21:31:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50202 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'139, mlcod=45'137)", 2024-09-17T21:31:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.690744101000007, 2024-09-17T21:31:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059659441000000001, 2024-09-17T21:31:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124222+0000", 2024-09-17T21:31:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.124224+0000", 2024-09-17T21:31:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5000000000000002e-06 2024-09-17T21:31:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147385+0000", 2024-09-17T21:31:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023160544000000002 2024-09-17T21:31:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147411+0000", 2024-09-17T21:31:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6095999999999999e-05 2024-09-17T21:31:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147438+0000", 2024-09-17T21:31:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6982999999999999e-05 2024-09-17T21:31:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147504+0000", 2024-09-17T21:31:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6156999999999997e-05 2024-09-17T21:31:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.147531+0000", 2024-09-17T21:31:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6741999999999999e-05 2024-09-17T21:31:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183820+0000", 2024-09-17T21:31:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036289090000000003 2024-09-17T21:31:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.183881+0000", 2024-09-17T21:31:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1328999999999995e-05 2024-09-17T21:31:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4384.0:50203 3.1 e49/25 3:864ed53c:::rbd_data.1120c2fd7e1b.0000000000000001:head v 49'141, mlcod=45'137)", 2024-09-17T21:31:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.653481357999993, 2024-09-17T21:31:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023138823999999999, 2024-09-17T21:31:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161485+0000", 2024-09-17T21:31:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161486+0000", 2024-09-17T21:31:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5290000000000001e-06 2024-09-17T21:31:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161496+0000", 2024-09-17T21:31:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0063e-05 2024-09-17T21:31:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161514+0000", 2024-09-17T21:31:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7728e-05 2024-09-17T21:31:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.161524+0000", 2024-09-17T21:31:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0339e-05 2024-09-17T21:31:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170872+0000", 2024-09-17T21:31:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093473640000000004 2024-09-17T21:31:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.170899+0000", 2024-09-17T21:31:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7321000000000002e-05 2024-09-17T21:31:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184580+0000", 2024-09-17T21:31:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013680988999999999 2024-09-17T21:31:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:11.184623+0000", 2024-09-17T21:31:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3491000000000001e-05 2024-09-17T21:31:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:88193 3.9 3:97498da5:::rbd_data.1120ea662e44.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-17T21:31:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.491300447999997, 2024-09-17T21:31:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026278261000000001, 2024-09-17T21:31:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 88193 2024-09-17T21:31:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323665+0000", 2024-09-17T21:31:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323666+0000", 2024-09-17T21:31:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7599999999999996e-07 2024-09-17T21:31:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323670+0000", 2024-09-17T21:31:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.771e-06 2024-09-17T21:31:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323680+0000", 2024-09-17T21:31:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9469999999999998e-06 2024-09-17T21:31:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323685+0000", 2024-09-17T21:31:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9919999999999998e-06 2024-09-17T21:31:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323728+0000", 2024-09-17T21:31:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2811000000000001e-05 2024-09-17T21:31:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.323776+0000", 2024-09-17T21:31:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8310000000000003e-05 2024-09-17T21:31:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.328583+0000", 2024-09-17T21:31:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048072899999999997 2024-09-17T21:31:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349861+0000", 2024-09-17T21:31:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021277695999999999 2024-09-17T21:31:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349877+0000", 2024-09-17T21:31:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6160000000000001e-05 2024-09-17T21:31:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349911+0000", 2024-09-17T21:31:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3844999999999999e-05 2024-09-17T21:31:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:40.349944+0000", 2024-09-17T21:31:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2663000000000002e-05 2024-09-17T21:31:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:101384 3.9 3:90f2a32a:::rbd_data.11206b8c0882.0000000000000178:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T21:31:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.488430848999997, 2024-09-17T21:31:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025286194000000001, 2024-09-17T21:31:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 101384 2024-09-17T21:31:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326535+0000", 2024-09-17T21:31:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326536+0000", 2024-09-17T21:31:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4799999999999997e-07 2024-09-17T21:31:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326541+0000", 2024-09-17T21:31:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8489999999999997e-06 2024-09-17T21:31:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326552+0000", 2024-09-17T21:31:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0912e-05 2024-09-17T21:31:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326557+0000", 2024-09-17T21:31:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0359999999999997e-06 2024-09-17T21:31:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326603+0000", 2024-09-17T21:31:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6428999999999999e-05 2024-09-17T21:31:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.326686+0000", 2024-09-17T21:31:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2521999999999994e-05 2024-09-17T21:31:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T21:31:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.332698+0000", 2024-09-17T21:31:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060124599999999999 2024-09-17T21:31:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351744+0000", 2024-09-17T21:31:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019045838999999998 2024-09-17T21:31:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351754+0000", 2024-09-17T21:31:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0380999999999999e-05 2024-09-17T21:31:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351791+0000", 2024-09-17T21:31:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6678999999999999e-05 2024-09-17T21:31:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:48.351821+0000", 2024-09-17T21:31:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0139000000000001e-05 2024-09-17T21:31:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:43.911 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:43.913 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:31:43.913 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:43.910+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:44.013 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:44.010+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:44.057 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:44.057 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:44.057 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:44.057 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:44.065 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:31:44.149 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:44.147+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:44.209 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:44.209 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:44.209 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:44.209 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:44.209 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:44.215 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:44.213+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:44.218 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:31:44.316 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T21:31:44.314+0000 7fbc0b62b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33374) UID: 0 2024-09-17T21:31:44.363 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:44.364 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:44.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:44.364 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:44.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.364 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:10799 3.1d 3:bb636d8b:::rbd_object_map.11207744cc2c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.364 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:44.364 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.29534151, 2024-09-17T21:31:44.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039122745, 2024-09-17T21:31:44.364 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.364 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.364 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:44.364 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.365 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.365 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.365 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 10799 2024-09-17T21:31:44.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.365 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:44.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065468+0000", 2024-09-17T21:31:44.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065469+0000", 2024-09-17T21:31:44.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-07 2024-09-17T21:31:44.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065477+0000", 2024-09-17T21:31:44.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1319999999999994e-06 2024-09-17T21:31:44.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065483+0000", 2024-09-17T21:31:44.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2040000000000002e-06 2024-09-17T21:31:44.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.065487+0000", 2024-09-17T21:31:44.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.42e-06 2024-09-17T21:31:44.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.099984+0000", 2024-09-17T21:31:44.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034496484000000001 2024-09-17T21:31:44.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100065+0000", 2024-09-17T21:31:44.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1200999999999997e-05 2024-09-17T21:31:44.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:44.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.100422+0000", 2024-09-17T21:31:44.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035657700000000001 2024-09-17T21:31:44.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:44.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104214+0000", 2024-09-17T21:31:44.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003792045 2024-09-17T21:31:44.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:44.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104505+0000", 2024-09-17T21:31:44.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029141399999999999 2024-09-17T21:31:44.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:44.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104556+0000", 2024-09-17T21:31:44.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1388999999999999e-05 2024-09-17T21:31:44.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.104591+0000", 2024-09-17T21:31:44.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4455999999999997e-05 2024-09-17T21:31:44.370 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.370 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.370 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.370 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12187 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.370 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:44.370 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.22124877900001, 2024-09-17T21:31:44.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038825750999999999, 2024-09-17T21:31:44.371 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.371 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.371 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.371 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12187 2024-09-17T21:31:44.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.371 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:44.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:44.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139561+0000", 2024-09-17T21:31:44.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:31:44.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139563+0000", 2024-09-17T21:31:44.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.624e-06 2024-09-17T21:31:44.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139569+0000", 2024-09-17T21:31:44.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1090000000000003e-06 2024-09-17T21:31:44.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139573+0000", 2024-09-17T21:31:44.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4429999999999996e-06 2024-09-17T21:31:44.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178206+0000", 2024-09-17T21:31:44.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038632376000000003 2024-09-17T21:31:44.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178282+0000", 2024-09-17T21:31:44.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6076999999999997e-05 2024-09-17T21:31:44.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178387+0000", 2024-09-17T21:31:44.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104682 2024-09-17T21:31:44.375 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.375 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.375 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.375 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12188 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.375 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:44.375 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.22123059800001, 2024-09-17T21:31:44.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039053826999999999, 2024-09-17T21:31:44.375 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.375 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.375 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.376 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12188 2024-09-17T21:31:44.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.376 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:44.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139579+0000", 2024-09-17T21:31:44.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139746+0000", 2024-09-17T21:31:44.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000167281 2024-09-17T21:31:44.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139749+0000", 2024-09-17T21:31:44.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9660000000000001e-06 2024-09-17T21:31:44.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139757+0000", 2024-09-17T21:31:44.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8029999999999997e-06 2024-09-17T21:31:44.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139771+0000", 2024-09-17T21:31:44.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3541e-05 2024-09-17T21:31:44.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178415+0000", 2024-09-17T21:31:44.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038644803999999998 2024-09-17T21:31:44.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178517+0000", 2024-09-17T21:31:44.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101298 2024-09-17T21:31:44.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178633+0000", 2024-09-17T21:31:44.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116134 2024-09-17T21:31:44.379 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.379 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.379 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.379 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12189 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.379 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:44.379 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.221030121, 2024-09-17T21:31:44.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039135991000000002, 2024-09-17T21:31:44.380 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.380 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.380 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.380 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12189 2024-09-17T21:31:44.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.380 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:44.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139779+0000", 2024-09-17T21:31:44.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139780+0000", 2024-09-17T21:31:44.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:31:44.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139782+0000", 2024-09-17T21:31:44.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1950000000000002e-06 2024-09-17T21:31:44.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139788+0000", 2024-09-17T21:31:44.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0639999999999999e-06 2024-09-17T21:31:44.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139792+0000", 2024-09-17T21:31:44.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1960000000000001e-06 2024-09-17T21:31:44.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178671+0000", 2024-09-17T21:31:44.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038878176 2024-09-17T21:31:44.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178778+0000", 2024-09-17T21:31:44.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107278 2024-09-17T21:31:44.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178915+0000", 2024-09-17T21:31:44.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013757900000000001 2024-09-17T21:31:44.383 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.383 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.383 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.384 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12190 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.384 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:44.384 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.22101093800001, 2024-09-17T21:31:44.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039343781000000001, 2024-09-17T21:31:44.384 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.384 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.384 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.384 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.384 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.384 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.384 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12190 2024-09-17T21:31:44.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.385 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:44.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:44.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139799+0000", 2024-09-17T21:31:44.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:44.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139801+0000", 2024-09-17T21:31:44.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.916e-06 2024-09-17T21:31:44.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139807+0000", 2024-09-17T21:31:44.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4540000000000004e-06 2024-09-17T21:31:44.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139812+0000", 2024-09-17T21:31:44.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9729999999999997e-06 2024-09-17T21:31:44.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.178960+0000", 2024-09-17T21:31:44.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039147567000000001 2024-09-17T21:31:44.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179051+0000", 2024-09-17T21:31:44.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0754000000000006e-05 2024-09-17T21:31:44.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179142+0000", 2024-09-17T21:31:44.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1712999999999994e-05 2024-09-17T21:31:44.388 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.388 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.388 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.388 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12191 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.388 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:44.388 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.22099108, 2024-09-17T21:31:44.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039464336000000003, 2024-09-17T21:31:44.388 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.389 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.389 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.389 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12191 2024-09-17T21:31:44.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.389 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:44.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:44.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139819+0000", 2024-09-17T21:31:44.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:31:44.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139821+0000", 2024-09-17T21:31:44.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1260000000000001e-06 2024-09-17T21:31:44.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139827+0000", 2024-09-17T21:31:44.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T21:31:44.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139840+0000", 2024-09-17T21:31:44.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2607e-05 2024-09-17T21:31:44.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179162+0000", 2024-09-17T21:31:44.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039322863999999999 2024-09-17T21:31:44.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179204+0000", 2024-09-17T21:31:44.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1938999999999999e-05 2024-09-17T21:31:44.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179283+0000", 2024-09-17T21:31:44.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8547000000000005e-05 2024-09-17T21:31:44.393 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.393 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.393 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.393 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12192 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.393 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:44.393 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.220963368, 2024-09-17T21:31:44.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039652318999999998, 2024-09-17T21:31:44.393 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.393 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.394 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.394 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12192 2024-09-17T21:31:44.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.394 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:44.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.139846+0000", 2024-09-17T21:31:44.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141458+0000", 2024-09-17T21:31:44.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001612098 2024-09-17T21:31:44.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141469+0000", 2024-09-17T21:31:44.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-17T21:31:44.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141478+0000", 2024-09-17T21:31:44.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T21:31:44.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.141485+0000", 2024-09-17T21:31:44.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4760000000000003e-06 2024-09-17T21:31:44.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179317+0000", 2024-09-17T21:31:44.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037831832000000003 2024-09-17T21:31:44.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179395+0000", 2024-09-17T21:31:44.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8349000000000005e-05 2024-09-17T21:31:44.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.179499+0000", 2024-09-17T21:31:44.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103628 2024-09-17T21:31:44.397 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.397 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.397 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.398 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12257 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 997376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.398 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:44.398 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.21877456, 2024-09-17T21:31:44.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038862751000000001, 2024-09-17T21:31:44.398 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.398 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.398 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.398 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.398 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.398 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.398 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12257 2024-09-17T21:31:44.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.398 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:44.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:44.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142035+0000", 2024-09-17T21:31:44.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T21:31:44.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142037+0000", 2024-09-17T21:31:44.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.03e-06 2024-09-17T21:31:44.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142043+0000", 2024-09-17T21:31:44.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6319999999999996e-06 2024-09-17T21:31:44.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142052+0000", 2024-09-17T21:31:44.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2709999999999998e-06 2024-09-17T21:31:44.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180794+0000", 2024-09-17T21:31:44.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038741758000000001 2024-09-17T21:31:44.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180843+0000", 2024-09-17T21:31:44.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9021999999999999e-05 2024-09-17T21:31:44.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180898+0000", 2024-09-17T21:31:44.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673000000000003e-05 2024-09-17T21:31:44.402 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.402 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.402 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.402 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12258 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1011712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.402 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:44.402 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.218751043, 2024-09-17T21:31:44.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038950717000000003, 2024-09-17T21:31:44.402 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.402 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.402 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.402 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.403 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.403 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.403 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12258 2024-09-17T21:31:44.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.403 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:44.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:44.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142059+0000", 2024-09-17T21:31:44.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199999999999999e-07 2024-09-17T21:31:44.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142061+0000", 2024-09-17T21:31:44.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8279999999999999e-06 2024-09-17T21:31:44.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142067+0000", 2024-09-17T21:31:44.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5600000000000001e-06 2024-09-17T21:31:44.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142071+0000", 2024-09-17T21:31:44.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6240000000000001e-06 2024-09-17T21:31:44.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180921+0000", 2024-09-17T21:31:44.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038849366000000003 2024-09-17T21:31:44.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.180961+0000", 2024-09-17T21:31:44.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0219000000000002e-05 2024-09-17T21:31:44.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181009+0000", 2024-09-17T21:31:44.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8437999999999999e-05 2024-09-17T21:31:44.406 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.406 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.406 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.407 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12259 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1026048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.407 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:44.407 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.21872379600001, 2024-09-17T21:31:44.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039116832999999997, 2024-09-17T21:31:44.407 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.407 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.407 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.407 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12259 2024-09-17T21:31:44.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.407 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:44.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:44.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142086+0000", 2024-09-17T21:31:44.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2200000000000004e-07 2024-09-17T21:31:44.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142088+0000", 2024-09-17T21:31:44.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0329999999999998e-06 2024-09-17T21:31:44.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142095+0000", 2024-09-17T21:31:44.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4320000000000004e-06 2024-09-17T21:31:44.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142100+0000", 2024-09-17T21:31:44.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:31:44.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181037+0000", 2024-09-17T21:31:44.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038937601000000002 2024-09-17T21:31:44.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181114+0000", 2024-09-17T21:31:44.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6857e-05 2024-09-17T21:31:44.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181203+0000", 2024-09-17T21:31:44.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8437999999999996e-05 2024-09-17T21:31:44.411 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.411 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.411 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.411 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12260 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1038336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.411 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:44.411 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.21870350899999, 2024-09-17T21:31:44.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039204184000000003, 2024-09-17T21:31:44.411 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.412 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.412 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.412 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12260 2024-09-17T21:31:44.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.412 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:44.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142106+0000", 2024-09-17T21:31:44.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142107+0000", 2024-09-17T21:31:44.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:31:44.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142108+0000", 2024-09-17T21:31:44.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.753e-06 2024-09-17T21:31:44.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142114+0000", 2024-09-17T21:31:44.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7660000000000004e-06 2024-09-17T21:31:44.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142119+0000", 2024-09-17T21:31:44.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:31:44.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181216+0000", 2024-09-17T21:31:44.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039097191000000003 2024-09-17T21:31:44.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181260+0000", 2024-09-17T21:31:44.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4731000000000003e-05 2024-09-17T21:31:44.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181310+0000", 2024-09-17T21:31:44.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9818000000000003e-05 2024-09-17T21:31:44.415 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.415 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.415 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.416 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12261 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1052672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.416 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:44.416 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.218685185, 2024-09-17T21:31:44.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039292200999999999, 2024-09-17T21:31:44.416 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.416 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.416 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.416 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.416 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.416 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.416 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12261 2024-09-17T21:31:44.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.416 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:44.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142124+0000", 2024-09-17T21:31:44.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142125+0000", 2024-09-17T21:31:44.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:31:44.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142127+0000", 2024-09-17T21:31:44.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.792e-06 2024-09-17T21:31:44.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142132+0000", 2024-09-17T21:31:44.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3979999999999998e-06 2024-09-17T21:31:44.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142136+0000", 2024-09-17T21:31:44.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.211e-06 2024-09-17T21:31:44.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181324+0000", 2024-09-17T21:31:44.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039187560000000003 2024-09-17T21:31:44.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181363+0000", 2024-09-17T21:31:44.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9375000000000002e-05 2024-09-17T21:31:44.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181417+0000", 2024-09-17T21:31:44.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3372000000000003e-05 2024-09-17T21:31:44.420 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.420 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.420 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.420 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12262 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1067008~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.420 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:44.420 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.218667392, 2024-09-17T21:31:44.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039384875, 2024-09-17T21:31:44.420 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.420 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.420 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.420 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.421 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12262 2024-09-17T21:31:44.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.421 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:44.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142142+0000", 2024-09-17T21:31:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142143+0000", 2024-09-17T21:31:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:31:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142148+0000", 2024-09-17T21:31:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4360000000000001e-06 2024-09-17T21:31:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142154+0000", 2024-09-17T21:31:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8719999999999999e-06 2024-09-17T21:31:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142158+0000", 2024-09-17T21:31:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.476e-06 2024-09-17T21:31:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181436+0000", 2024-09-17T21:31:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039277919000000001 2024-09-17T21:31:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181480+0000", 2024-09-17T21:31:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3649999999999997e-05 2024-09-17T21:31:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181527+0000", 2024-09-17T21:31:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7131e-05 2024-09-17T21:31:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12263 3.1d 3:be60a59f:::rbd_data.11202c1b6cb4.000000000000000c:4 [read 1079296~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.218644957, 2024-09-17T21:31:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039473339000000003, 2024-09-17T21:31:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12263 2024-09-17T21:31:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142165+0000", 2024-09-17T21:31:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T21:31:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142167+0000", 2024-09-17T21:31:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.81e-06 2024-09-17T21:31:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142172+0000", 2024-09-17T21:31:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5929999999999996e-06 2024-09-17T21:31:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142177+0000", 2024-09-17T21:31:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T21:31:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181553+0000", 2024-09-17T21:31:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039376130000000002 2024-09-17T21:31:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181592+0000", 2024-09-17T21:31:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9153999999999998e-05 2024-09-17T21:31:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181638+0000", 2024-09-17T21:31:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5864e-05 2024-09-17T21:31:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12278 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1095680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.218626982, 2024-09-17T21:31:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039454222999999997, 2024-09-17T21:31:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12278 2024-09-17T21:31:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142183+0000", 2024-09-17T21:31:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-17T21:31:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142185+0000", 2024-09-17T21:31:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.697e-06 2024-09-17T21:31:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142191+0000", 2024-09-17T21:31:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0939999999999996e-06 2024-09-17T21:31:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142196+0000", 2024-09-17T21:31:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4829999999999999e-06 2024-09-17T21:31:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181548+0000", 2024-09-17T21:31:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039351200000000003 2024-09-17T21:31:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181591+0000", 2024-09-17T21:31:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3492000000000003e-05 2024-09-17T21:31:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181637+0000", 2024-09-17T21:31:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5847000000000003e-05 2024-09-17T21:31:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12279 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.21860369300001, 2024-09-17T21:31:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039539801999999999, 2024-09-17T21:31:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12279 2024-09-17T21:31:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142206+0000", 2024-09-17T21:31:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.89e-07 2024-09-17T21:31:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142208+0000", 2024-09-17T21:31:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7579999999999999e-06 2024-09-17T21:31:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142214+0000", 2024-09-17T21:31:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:31:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142218+0000", 2024-09-17T21:31:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3429999999999998e-06 2024-09-17T21:31:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181653+0000", 2024-09-17T21:31:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039435322000000002 2024-09-17T21:31:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181698+0000", 2024-09-17T21:31:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4978999999999997e-05 2024-09-17T21:31:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181746+0000", 2024-09-17T21:31:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7571999999999998e-05 2024-09-17T21:31:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12280 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1124352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.218586174, 2024-09-17T21:31:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039704358000000002, 2024-09-17T21:31:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12280 2024-09-17T21:31:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142223+0000", 2024-09-17T21:31:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142224+0000", 2024-09-17T21:31:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:31:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142226+0000", 2024-09-17T21:31:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:31:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142231+0000", 2024-09-17T21:31:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:31:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142240+0000", 2024-09-17T21:31:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4590000000000004e-06 2024-09-17T21:31:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181774+0000", 2024-09-17T21:31:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039533833999999997 2024-09-17T21:31:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181859+0000", 2024-09-17T21:31:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4328999999999998e-05 2024-09-17T21:31:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181928+0000", 2024-09-17T21:31:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9139000000000003e-05 2024-09-17T21:31:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12281 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1136640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.218563268, 2024-09-17T21:31:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039787799999999998, 2024-09-17T21:31:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12281 2024-09-17T21:31:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142246+0000", 2024-09-17T21:31:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142309+0000", 2024-09-17T21:31:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2323999999999995e-05 2024-09-17T21:31:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142317+0000", 2024-09-17T21:31:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1859999999999992e-06 2024-09-17T21:31:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142324+0000", 2024-09-17T21:31:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4699999999999996e-06 2024-09-17T21:31:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142330+0000", 2024-09-17T21:31:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4650000000000002e-06 2024-09-17T21:31:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181948+0000", 2024-09-17T21:31:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039618056999999998 2024-09-17T21:31:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.181986+0000", 2024-09-17T21:31:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8442999999999998e-05 2024-09-17T21:31:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182034+0000", 2024-09-17T21:31:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7855000000000002e-05 2024-09-17T21:31:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12282 3.5 3:a214ef1f:::rbd_data.11202c1b6cb4.0000000000000007:4 [read 1150976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.21847103899999, 2024-09-17T21:31:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039871919999999998, 2024-09-17T21:31:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12282 2024-09-17T21:31:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142339+0000", 2024-09-17T21:31:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:31:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142341+0000", 2024-09-17T21:31:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.232e-06 2024-09-17T21:31:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142348+0000", 2024-09-17T21:31:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7320000000000001e-06 2024-09-17T21:31:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142353+0000", 2024-09-17T21:31:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6779999999999999e-06 2024-09-17T21:31:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182062+0000", 2024-09-17T21:31:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039709739000000001 2024-09-17T21:31:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182128+0000", 2024-09-17T21:31:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5739000000000002e-05 2024-09-17T21:31:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182210+0000", 2024-09-17T21:31:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:31:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:12283 3.5 3.f8f72845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.218450628, 2024-09-17T21:31:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039896434000000001, 2024-09-17T21:31:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T21:31:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 12283 2024-09-17T21:31:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.142359+0000", 2024-09-17T21:31:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144852+0000", 2024-09-17T21:31:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024934079999999999 2024-09-17T21:31:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144856+0000", 2024-09-17T21:31:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:31:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144864+0000", 2024-09-17T21:31:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0160000000000008e-06 2024-09-17T21:31:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.144869+0000", 2024-09-17T21:31:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6999999999999999e-06 2024-09-17T21:31:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182246+0000", 2024-09-17T21:31:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037377207000000003 2024-09-17T21:31:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.182255+0000", 2024-09-17T21:31:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1840000000000005e-06 2024-09-17T21:31:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.456 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.456 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.456 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.456 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:44.457 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T21:31:44.458 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:44.454+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:44.558 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-17T21:31:44.556+0000 7f062d87c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33602) UID: 0 2024-09-17T21:31:44.604 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:44.605 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:44.605 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:44.605 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:44.613 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T21:31:44.658 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:44.655+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:44.716 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:44.717 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:44.717 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:44.717 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:44.717 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:44.725 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T21:31:44.769 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T21:31:44.767+0000 7f6000639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33366) UID: 0 2024-09-17T21:31:44.829 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:36935 3.a 3:54c8759b:::rbd_data.1120e62eea89.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-17T21:31:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:31:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 107.867851198, 2024-09-17T21:31:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020540151999999999, 2024-09-17T21:31:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 36935 2024-09-17T21:31:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:31:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958909+0000", 2024-09-17T21:31:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958910+0000", 2024-09-17T21:31:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:31:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958918+0000", 2024-09-17T21:31:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6469999999999998e-06 2024-09-17T21:31:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958933+0000", 2024-09-17T21:31:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4791e-05 2024-09-17T21:31:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958940+0000", 2024-09-17T21:31:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-17T21:31:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.958990+0000", 2024-09-17T21:31:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9759000000000003e-05 2024-09-17T21:31:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.959039+0000", 2024-09-17T21:31:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9487000000000002e-05 2024-09-17T21:31:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T21:31:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.962098+0000", 2024-09-17T21:31:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003059258 2024-09-17T21:31:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979325+0000", 2024-09-17T21:31:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017226841 2024-09-17T21:31:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979343+0000", 2024-09-17T21:31:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7486000000000002e-05 2024-09-17T21:31:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979401+0000", 2024-09-17T21:31:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7980999999999999e-05 2024-09-17T21:31:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:56.979449+0000", 2024-09-17T21:31:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8639e-05 2024-09-17T21:31:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39447 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 909312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.858913298, 2024-09-17T21:31:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017419153999999999, 2024-09-17T21:31:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39447 2024-09-17T21:31:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967847+0000", 2024-09-17T21:31:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:31:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967849+0000", 2024-09-17T21:31:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4339999999999999e-06 2024-09-17T21:31:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967853+0000", 2024-09-17T21:31:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:31:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967857+0000", 2024-09-17T21:31:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:31:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985193+0000", 2024-09-17T21:31:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017335973000000001 2024-09-17T21:31:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985228+0000", 2024-09-17T21:31:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5154999999999998e-05 2024-09-17T21:31:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985266+0000", 2024-09-17T21:31:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8448000000000002e-05 2024-09-17T21:31:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39448 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 974848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.858900482, 2024-09-17T21:31:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017517066000000001, 2024-09-17T21:31:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39448 2024-09-17T21:31:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967860+0000", 2024-09-17T21:31:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.72e-07 2024-09-17T21:31:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967861+0000", 2024-09-17T21:31:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.096e-06 2024-09-17T21:31:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967865+0000", 2024-09-17T21:31:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9339999999999999e-06 2024-09-17T21:31:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967868+0000", 2024-09-17T21:31:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.508e-06 2024-09-17T21:31:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985285+0000", 2024-09-17T21:31:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017417208 2024-09-17T21:31:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985339+0000", 2024-09-17T21:31:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3931000000000003e-05 2024-09-17T21:31:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985377+0000", 2024-09-17T21:31:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8016999999999999e-05 2024-09-17T21:31:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39449 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.858888372, 2024-09-17T21:31:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017595137, 2024-09-17T21:31:44.845 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.845 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.845 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.845 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39449 2024-09-17T21:31:44.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.845 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:44.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:44.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967872+0000", 2024-09-17T21:31:44.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2700000000000001e-07 2024-09-17T21:31:44.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967873+0000", 2024-09-17T21:31:44.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.164e-06 2024-09-17T21:31:44.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967877+0000", 2024-09-17T21:31:44.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.23e-06 2024-09-17T21:31:44.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967880+0000", 2024-09-17T21:31:44.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.286e-06 2024-09-17T21:31:44.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985397+0000", 2024-09-17T21:31:44.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017516982 2024-09-17T21:31:44.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985431+0000", 2024-09-17T21:31:44.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3899999999999997e-05 2024-09-17T21:31:44.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985467+0000", 2024-09-17T21:31:44.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6347999999999998e-05 2024-09-17T21:31:44.848 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.848 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.848 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.849 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39450 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1105920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.849 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:31:44.849 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.85886811500001, 2024-09-17T21:31:44.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017677824000000002, 2024-09-17T21:31:44.849 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.849 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.849 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.849 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.849 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.849 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.849 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39450 2024-09-17T21:31:44.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.849 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:31:44.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967892+0000", 2024-09-17T21:31:44.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967893+0000", 2024-09-17T21:31:44.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9100000000000004e-07 2024-09-17T21:31:44.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967895+0000", 2024-09-17T21:31:44.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:31:44.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967901+0000", 2024-09-17T21:31:44.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.083e-06 2024-09-17T21:31:44.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967906+0000", 2024-09-17T21:31:44.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T21:31:44.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985493+0000", 2024-09-17T21:31:44.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017587565999999999 2024-09-17T21:31:44.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985531+0000", 2024-09-17T21:31:44.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7490000000000002e-05 2024-09-17T21:31:44.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985570+0000", 2024-09-17T21:31:44.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9205000000000002e-05 2024-09-17T21:31:44.853 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.853 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.853 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.853 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39451 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1171456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.853 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:44.853 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.858847735, 2024-09-17T21:31:44.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017752337999999999, 2024-09-17T21:31:44.853 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.853 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.853 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.853 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.854 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39451 2024-09-17T21:31:44.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.854 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:44.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:44.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967913+0000", 2024-09-17T21:31:44.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:31:44.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967915+0000", 2024-09-17T21:31:44.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780000000000001e-06 2024-09-17T21:31:44.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967922+0000", 2024-09-17T21:31:44.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7020000000000004e-06 2024-09-17T21:31:44.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967926+0000", 2024-09-17T21:31:44.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.809e-06 2024-09-17T21:31:44.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985590+0000", 2024-09-17T21:31:44.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017663913999999999 2024-09-17T21:31:44.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985627+0000", 2024-09-17T21:31:44.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6896e-05 2024-09-17T21:31:44.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985665+0000", 2024-09-17T21:31:44.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7793999999999998e-05 2024-09-17T21:31:44.857 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.857 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.857 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.857 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39452 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1236992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.857 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:44.858 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.858827793, 2024-09-17T21:31:44.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017829602999999999, 2024-09-17T21:31:44.858 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.858 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.858 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.858 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.858 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.858 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.858 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39452 2024-09-17T21:31:44.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.858 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:44.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:44.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967933+0000", 2024-09-17T21:31:44.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:31:44.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967947+0000", 2024-09-17T21:31:44.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4280999999999999e-05 2024-09-17T21:31:44.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967954+0000", 2024-09-17T21:31:44.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3249999999999996e-06 2024-09-17T21:31:44.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967959+0000", 2024-09-17T21:31:44.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2299999999999999e-06 2024-09-17T21:31:44.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985687+0000", 2024-09-17T21:31:44.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017727844999999999 2024-09-17T21:31:44.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985725+0000", 2024-09-17T21:31:44.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8243e-05 2024-09-17T21:31:44.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985762+0000", 2024-09-17T21:31:44.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7023000000000001e-05 2024-09-17T21:31:44.862 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.862 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.862 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.862 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39453 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.862 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:31:44.862 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.858791305, 2024-09-17T21:31:44.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017893793000000002, 2024-09-17T21:31:44.862 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.862 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.862 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.862 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.862 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.863 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.863 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39453 2024-09-17T21:31:44.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.863 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:31:44.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967969+0000", 2024-09-17T21:31:44.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967970+0000", 2024-09-17T21:31:44.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:31:44.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967972+0000", 2024-09-17T21:31:44.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0229999999999999e-06 2024-09-17T21:31:44.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967978+0000", 2024-09-17T21:31:44.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0430000000000004e-06 2024-09-17T21:31:44.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967982+0000", 2024-09-17T21:31:44.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2570000000000001e-06 2024-09-17T21:31:44.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985782+0000", 2024-09-17T21:31:44.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017799683 2024-09-17T21:31:44.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985818+0000", 2024-09-17T21:31:44.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6304999999999999e-05 2024-09-17T21:31:44.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985863+0000", 2024-09-17T21:31:44.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4953999999999997e-05 2024-09-17T21:31:44.866 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.866 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.866 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.867 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39454 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1368064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.867 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:31:44.867 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.858771977, 2024-09-17T21:31:44.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017968886, 2024-09-17T21:31:44.867 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.867 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.867 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.867 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.867 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.867 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.867 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39454 2024-09-17T21:31:44.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.867 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:31:44.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967988+0000", 2024-09-17T21:31:44.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967989+0000", 2024-09-17T21:31:44.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.15e-07 2024-09-17T21:31:44.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967991+0000", 2024-09-17T21:31:44.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.945e-06 2024-09-17T21:31:44.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.967997+0000", 2024-09-17T21:31:44.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0190000000000003e-06 2024-09-17T21:31:44.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968001+0000", 2024-09-17T21:31:44.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6650000000000002e-06 2024-09-17T21:31:44.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985883+0000", 2024-09-17T21:31:44.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017881971999999999 2024-09-17T21:31:44.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985920+0000", 2024-09-17T21:31:44.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-05 2024-09-17T21:31:44.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985957+0000", 2024-09-17T21:31:44.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7370000000000003e-05 2024-09-17T21:31:44.871 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.871 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.871 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.871 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39455 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.871 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:44.871 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.858752803, 2024-09-17T21:31:44.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018044437999999999, 2024-09-17T21:31:44.871 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.871 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.871 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.872 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.872 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.872 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.872 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39455 2024-09-17T21:31:44.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.872 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:44.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:44.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968008+0000", 2024-09-17T21:31:44.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:31:44.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968010+0000", 2024-09-17T21:31:44.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.886e-06 2024-09-17T21:31:44.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968017+0000", 2024-09-17T21:31:44.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:31:44.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968021+0000", 2024-09-17T21:31:44.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850000000000004e-06 2024-09-17T21:31:44.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.985977+0000", 2024-09-17T21:31:44.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017956368 2024-09-17T21:31:44.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986015+0000", 2024-09-17T21:31:44.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7610000000000001e-05 2024-09-17T21:31:44.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986052+0000", 2024-09-17T21:31:44.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7109e-05 2024-09-17T21:31:44.875 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.875 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.875 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.875 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39456 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1499136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.875 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:44.875 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.858732503, 2024-09-17T21:31:44.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018114514000000002, 2024-09-17T21:31:44.876 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.876 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.876 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.876 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.876 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.876 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.876 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39456 2024-09-17T21:31:44.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.876 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:44.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:44.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968028+0000", 2024-09-17T21:31:44.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-17T21:31:44.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968030+0000", 2024-09-17T21:31:44.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0049999999999999e-06 2024-09-17T21:31:44.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968036+0000", 2024-09-17T21:31:44.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0129999999999999e-06 2024-09-17T21:31:44.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968041+0000", 2024-09-17T21:31:44.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8300000000000003e-06 2024-09-17T21:31:44.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986071+0000", 2024-09-17T21:31:44.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018029620999999999 2024-09-17T21:31:44.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986106+0000", 2024-09-17T21:31:44.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4867999999999999e-05 2024-09-17T21:31:44.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986142+0000", 2024-09-17T21:31:44.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6566000000000002e-05 2024-09-17T21:31:44.879 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.879 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.879 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.880 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39457 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1564672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.880 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:44.880 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.85871268699999, 2024-09-17T21:31:44.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018188644, 2024-09-17T21:31:44.880 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.880 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.880 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.880 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.880 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.880 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.880 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39457 2024-09-17T21:31:44.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.880 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:44.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:44.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968048+0000", 2024-09-17T21:31:44.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7199999999999999e-07 2024-09-17T21:31:44.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968050+0000", 2024-09-17T21:31:44.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1110000000000002e-06 2024-09-17T21:31:44.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968057+0000", 2024-09-17T21:31:44.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6170000000000003e-06 2024-09-17T21:31:44.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968062+0000", 2024-09-17T21:31:44.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:31:44.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986163+0000", 2024-09-17T21:31:44.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018101435999999999 2024-09-17T21:31:44.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986199+0000", 2024-09-17T21:31:44.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.591e-05 2024-09-17T21:31:44.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986236+0000", 2024-09-17T21:31:44.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7416999999999998e-05 2024-09-17T21:31:44.884 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.884 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.884 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.884 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39458 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1630208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.884 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:44.884 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.858685747, 2024-09-17T21:31:44.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018255856000000001, 2024-09-17T21:31:44.884 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.885 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.885 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.885 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.885 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.885 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.885 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39458 2024-09-17T21:31:44.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.885 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:44.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:44.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968075+0000", 2024-09-17T21:31:44.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:31:44.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968077+0000", 2024-09-17T21:31:44.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.779e-06 2024-09-17T21:31:44.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968081+0000", 2024-09-17T21:31:44.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.532e-06 2024-09-17T21:31:44.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968085+0000", 2024-09-17T21:31:44.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999999e-06 2024-09-17T21:31:44.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986256+0000", 2024-09-17T21:31:44.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018171273000000002 2024-09-17T21:31:44.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986293+0000", 2024-09-17T21:31:44.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6534e-05 2024-09-17T21:31:44.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986330+0000", 2024-09-17T21:31:44.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7536000000000001e-05 2024-09-17T21:31:44.888 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.888 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.888 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.889 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39459 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1695744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.889 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:31:44.889 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.858671169, 2024-09-17T21:31:44.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334493, 2024-09-17T21:31:44.889 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.889 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.889 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.889 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.889 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.889 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.889 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39459 2024-09-17T21:31:44.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.889 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:31:44.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968089+0000", 2024-09-17T21:31:44.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968090+0000", 2024-09-17T21:31:44.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T21:31:44.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968091+0000", 2024-09-17T21:31:44.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3319999999999999e-06 2024-09-17T21:31:44.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968094+0000", 2024-09-17T21:31:44.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2590000000000001e-06 2024-09-17T21:31:44.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968097+0000", 2024-09-17T21:31:44.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:31:44.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986350+0000", 2024-09-17T21:31:44.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018253364000000001 2024-09-17T21:31:44.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986387+0000", 2024-09-17T21:31:44.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6350999999999998e-05 2024-09-17T21:31:44.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986424+0000", 2024-09-17T21:31:44.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6897999999999998e-05 2024-09-17T21:31:44.893 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.893 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.893 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.893 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39460 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1761280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.893 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:31:44.893 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.85866003300001, 2024-09-17T21:31:44.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018415918999999999, 2024-09-17T21:31:44.893 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.893 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.893 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.893 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.894 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.894 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.894 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39460 2024-09-17T21:31:44.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.894 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:31:44.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968100+0000", 2024-09-17T21:31:44.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968101+0000", 2024-09-17T21:31:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T21:31:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968102+0000", 2024-09-17T21:31:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1689999999999999e-06 2024-09-17T21:31:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968105+0000", 2024-09-17T21:31:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:31:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968108+0000", 2024-09-17T21:31:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6630000000000002e-06 2024-09-17T21:31:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986442+0000", 2024-09-17T21:31:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018334487 2024-09-17T21:31:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986479+0000", 2024-09-17T21:31:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6557000000000003e-05 2024-09-17T21:31:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986516+0000", 2024-09-17T21:31:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7648999999999998e-05 2024-09-17T21:31:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39461 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1826816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.858649273, 2024-09-17T21:31:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018499768, 2024-09-17T21:31:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39461 2024-09-17T21:31:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968111+0000", 2024-09-17T21:31:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4699999999999998e-07 2024-09-17T21:31:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968112+0000", 2024-09-17T21:31:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-06 2024-09-17T21:31:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968116+0000", 2024-09-17T21:31:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1080000000000001e-06 2024-09-17T21:31:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968122+0000", 2024-09-17T21:31:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1070000000000001e-06 2024-09-17T21:31:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986537+0000", 2024-09-17T21:31:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018414834000000001 2024-09-17T21:31:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986572+0000", 2024-09-17T21:31:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5726000000000003e-05 2024-09-17T21:31:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986611+0000", 2024-09-17T21:31:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8584000000000002e-05 2024-09-17T21:31:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39462 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1892352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:31:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.858634909, 2024-09-17T21:31:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018583252000000001, 2024-09-17T21:31:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39462 2024-09-17T21:31:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:31:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968125+0000", 2024-09-17T21:31:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968126+0000", 2024-09-17T21:31:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:31:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968127+0000", 2024-09-17T21:31:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.229e-06 2024-09-17T21:31:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968131+0000", 2024-09-17T21:31:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8010000000000001e-06 2024-09-17T21:31:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968134+0000", 2024-09-17T21:31:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8880000000000001e-06 2024-09-17T21:31:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986634+0000", 2024-09-17T21:31:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018500501999999999 2024-09-17T21:31:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986671+0000", 2024-09-17T21:31:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6881000000000001e-05 2024-09-17T21:31:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986709+0000", 2024-09-17T21:31:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7546999999999998e-05 2024-09-17T21:31:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39463 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 1957888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.858623294, 2024-09-17T21:31:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018667186999999998, 2024-09-17T21:31:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39463 2024-09-17T21:31:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968137+0000", 2024-09-17T21:31:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:31:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968138+0000", 2024-09-17T21:31:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:31:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968142+0000", 2024-09-17T21:31:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.145e-06 2024-09-17T21:31:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968144+0000", 2024-09-17T21:31:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6869999999999999e-06 2024-09-17T21:31:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986728+0000", 2024-09-17T21:31:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018584163000000001 2024-09-17T21:31:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986766+0000", 2024-09-17T21:31:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7987000000000001e-05 2024-09-17T21:31:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986804+0000", 2024-09-17T21:31:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7892000000000002e-05 2024-09-17T21:31:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39464 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:31:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.85861298499999, 2024-09-17T21:31:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018753723999999999, 2024-09-17T21:31:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39464 2024-09-17T21:31:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:31:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968147+0000", 2024-09-17T21:31:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968148+0000", 2024-09-17T21:31:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:31:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968152+0000", 2024-09-17T21:31:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3749999999999996e-06 2024-09-17T21:31:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968155+0000", 2024-09-17T21:31:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.371e-06 2024-09-17T21:31:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968158+0000", 2024-09-17T21:31:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T21:31:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986824+0000", 2024-09-17T21:31:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018665779 2024-09-17T21:31:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986863+0000", 2024-09-17T21:31:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9273000000000002e-05 2024-09-17T21:31:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986901+0000", 2024-09-17T21:31:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7978000000000001e-05 2024-09-17T21:31:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:39465 3.10 3:0fadd8e3:::rbd_data.1120c6307ec.000000000000000b:b [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T21:31:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.858597586, 2024-09-17T21:31:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018825341999999998, 2024-09-17T21:31:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 39465 2024-09-17T21:31:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968163+0000", 2024-09-17T21:31:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-17T21:31:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968164+0000", 2024-09-17T21:31:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1289999999999999e-06 2024-09-17T21:31:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968167+0000", 2024-09-17T21:31:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1379999999999999e-06 2024-09-17T21:31:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.968170+0000", 2024-09-17T21:31:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:31:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986914+0000", 2024-09-17T21:31:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018743829 2024-09-17T21:31:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986950+0000", 2024-09-17T21:31:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5819999999999999e-05 2024-09-17T21:31:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:02.986988+0000", 2024-09-17T21:31:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8362000000000003e-05 2024-09-17T21:31:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:44.919 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:44.920 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T21:31:44.921 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-09-17T21:31:44.917+0000 7f7a06891640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 45351) UID: 0 2024-09-17T21:31:45.022 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:45.019+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:45.065 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:45.065 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:45.065 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T21:31:45.065 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:45.074 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T21:31:45.123 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-09-17T21:31:45.121+0000 7f11c8d26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33596) UID: 0 2024-09-17T21:31:45.217 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T21:31:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T21:31:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T21:31:45.218 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:45.223 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:45.221+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:45.227 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T21:31:45.325 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-09-17T21:31:45.322+0000 7fd451c1d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33583) UID: 0 2024-09-17T21:31:45.332 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T21:31:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T21:31:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T21:31:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T21:31:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4197.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:31:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:31:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 137.389624548, 2024-09-17T21:31:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61572000699999996, 2024-09-17T21:31:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4197", 2024-09-17T21:31:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/3386296386", 2024-09-17T21:31:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T21:31:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:31:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939743+0000", 2024-09-17T21:31:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939745+0000", 2024-09-17T21:31:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6899999999999999e-06 2024-09-17T21:31:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939753+0000", 2024-09-17T21:31:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1950000000000003e-06 2024-09-17T21:31:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939767+0000", 2024-09-17T21:31:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3429e-05 2024-09-17T21:31:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939776+0000", 2024-09-17T21:31:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1600000000000004e-06 2024-09-17T21:31:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939858+0000", 2024-09-17T21:31:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2496000000000004e-05 2024-09-17T21:31:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T21:31:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.939868+0000", 2024-09-17T21:31:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1910000000000006e-06 2024-09-17T21:31:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940612+0000", 2024-09-17T21:31:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074429599999999998 2024-09-17T21:31:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:31:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:27.940621+0000", 2024-09-17T21:31:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9139999999999997e-06 2024-09-17T21:31:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554006+0000", 2024-09-17T21:31:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.61338505099999996 2024-09-17T21:31:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554166+0000", 2024-09-17T21:31:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015982600000000001 2024-09-17T21:31:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T21:31:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554296+0000", 2024-09-17T21:31:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013002399999999999 2024-09-17T21:31:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.554936+0000", 2024-09-17T21:31:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064009400000000001 2024-09-17T21:31:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555389+0000", 2024-09-17T21:31:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045355200000000002 2024-09-17T21:31:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555432+0000", 2024-09-17T21:31:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2477e-05 2024-09-17T21:31:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:28.555463+0000", 2024-09-17T21:31:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1612000000000002e-05 2024-09-17T21:31:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17527 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1005568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:31:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.92439671699999, 2024-09-17T21:31:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046452090000000001, 2024-09-17T21:31:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17527 2024-09-17T21:31:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:31:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.404971+0000", 2024-09-17T21:31:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405496+0000", 2024-09-17T21:31:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052498400000000004 2024-09-17T21:31:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405500+0000", 2024-09-17T21:31:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8410000000000002e-06 2024-09-17T21:31:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405511+0000", 2024-09-17T21:31:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0749e-05 2024-09-17T21:31:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.405517+0000", 2024-09-17T21:31:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:31:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451275+0000", 2024-09-17T21:31:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045758134999999998 2024-09-17T21:31:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451355+0000", 2024-09-17T21:31:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9453e-05 2024-09-17T21:31:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.451423+0000", 2024-09-17T21:31:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8782999999999995e-05 2024-09-17T21:31:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17615 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 860160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.919156229, 2024-09-17T21:31:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046741309000000002, 2024-09-17T21:31:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17615 2024-09-17T21:31:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410212+0000", 2024-09-17T21:31:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3099999999999999e-07 2024-09-17T21:31:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410213+0000", 2024-09-17T21:31:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.274e-06 2024-09-17T21:31:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410217+0000", 2024-09-17T21:31:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8199999999999998e-06 2024-09-17T21:31:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410220+0000", 2024-09-17T21:31:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6900000000000001e-06 2024-09-17T21:31:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455264+0000", 2024-09-17T21:31:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045044002999999999 2024-09-17T21:31:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.455338+0000", 2024-09-17T21:31:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4182000000000002e-05 2024-09-17T21:31:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.456953+0000", 2024-09-17T21:31:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016150089999999999 2024-09-17T21:31:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17616 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 872448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.91914417700001, 2024-09-17T21:31:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946039000000002, 2024-09-17T21:31:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17616 2024-09-17T21:31:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410224+0000", 2024-09-17T21:31:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T21:31:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410225+0000", 2024-09-17T21:31:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3459999999999999e-06 2024-09-17T21:31:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410228+0000", 2024-09-17T21:31:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.083e-06 2024-09-17T21:31:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410232+0000", 2024-09-17T21:31:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4419999999999998e-06 2024-09-17T21:31:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457004+0000", 2024-09-17T21:31:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046771873999999998 2024-09-17T21:31:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457084+0000", 2024-09-17T21:31:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0506000000000005e-05 2024-09-17T21:31:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457170+0000", 2024-09-17T21:31:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5494000000000005e-05 2024-09-17T21:31:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17617 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.91913277499999, 2024-09-17T21:31:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047084168000000003, 2024-09-17T21:31:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17617 2024-09-17T21:31:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410235+0000", 2024-09-17T21:31:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-17T21:31:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410237+0000", 2024-09-17T21:31:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1960000000000001e-06 2024-09-17T21:31:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410246+0000", 2024-09-17T21:31:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7880000000000001e-06 2024-09-17T21:31:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410250+0000", 2024-09-17T21:31:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.518e-06 2024-09-17T21:31:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457209+0000", 2024-09-17T21:31:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046959283999999997 2024-09-17T21:31:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457264+0000", 2024-09-17T21:31:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4673999999999999e-05 2024-09-17T21:31:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457319+0000", 2024-09-17T21:31:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5497000000000001e-05 2024-09-17T21:31:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17618 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 899072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.91911333100001, 2024-09-17T21:31:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047204208999999997, 2024-09-17T21:31:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17618 2024-09-17T21:31:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410255+0000", 2024-09-17T21:31:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8799999999999998e-07 2024-09-17T21:31:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410256+0000", 2024-09-17T21:31:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T21:31:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410260+0000", 2024-09-17T21:31:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6500000000000002e-06 2024-09-17T21:31:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410263+0000", 2024-09-17T21:31:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7089999999999998e-06 2024-09-17T21:31:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457355+0000", 2024-09-17T21:31:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047092756999999999 2024-09-17T21:31:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457407+0000", 2024-09-17T21:31:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1808999999999998e-05 2024-09-17T21:31:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457459+0000", 2024-09-17T21:31:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1684000000000002e-05 2024-09-17T21:31:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17619 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 913408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.91910209300001, 2024-09-17T21:31:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047293443999999997, 2024-09-17T21:31:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17619 2024-09-17T21:31:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410266+0000", 2024-09-17T21:31:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:31:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410267+0000", 2024-09-17T21:31:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2780000000000001e-06 2024-09-17T21:31:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410270+0000", 2024-09-17T21:31:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.01e-06 2024-09-17T21:31:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410273+0000", 2024-09-17T21:31:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6410000000000002e-06 2024-09-17T21:31:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457473+0000", 2024-09-17T21:31:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047199829999999998 2024-09-17T21:31:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457517+0000", 2024-09-17T21:31:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4058999999999999e-05 2024-09-17T21:31:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457559+0000", 2024-09-17T21:31:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2404999999999998e-05 2024-09-17T21:31:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17620 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 927744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.91909092100001, 2024-09-17T21:31:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047402409, 2024-09-17T21:31:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17620 2024-09-17T21:31:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410277+0000", 2024-09-17T21:31:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:31:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410278+0000", 2024-09-17T21:31:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1790000000000001e-06 2024-09-17T21:31:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410281+0000", 2024-09-17T21:31:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.061e-06 2024-09-17T21:31:45.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410284+0000", 2024-09-17T21:31:45.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.65e-06 2024-09-17T21:31:45.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457587+0000", 2024-09-17T21:31:45.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047303206 2024-09-17T21:31:45.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457633+0000", 2024-09-17T21:31:45.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6063999999999998e-05 2024-09-17T21:31:45.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457679+0000", 2024-09-17T21:31:45.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6007000000000002e-05 2024-09-17T21:31:45.372 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.372 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.372 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.372 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17621 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 940032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:45.372 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:31:45.373 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.919080505, 2024-09-17T21:31:45.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047486550000000002, 2024-09-17T21:31:45.373 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.373 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.373 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:45.373 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.373 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.373 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.373 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17621 2024-09-17T21:31:45.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.373 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:31:45.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410287+0000", 2024-09-17T21:31:45.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410288+0000", 2024-09-17T21:31:45.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:31:45.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410289+0000", 2024-09-17T21:31:45.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1179999999999999e-06 2024-09-17T21:31:45.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410292+0000", 2024-09-17T21:31:45.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:31:45.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410294+0000", 2024-09-17T21:31:45.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T21:31:45.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457694+0000", 2024-09-17T21:31:45.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047399416999999999 2024-09-17T21:31:45.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457734+0000", 2024-09-17T21:31:45.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0528999999999998e-05 2024-09-17T21:31:45.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457774+0000", 2024-09-17T21:31:45.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9529e-05 2024-09-17T21:31:45.377 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.377 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.377 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.377 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17622 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:45.377 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:45.377 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.919070323, 2024-09-17T21:31:45.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047596619, 2024-09-17T21:31:45.377 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.377 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.378 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:45.378 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.378 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.378 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.378 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17622 2024-09-17T21:31:45.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.378 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:45.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:45.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410298+0000", 2024-09-17T21:31:45.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:31:45.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410303+0000", 2024-09-17T21:31:45.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2979999999999999e-06 2024-09-17T21:31:45.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410307+0000", 2024-09-17T21:31:45.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:31:45.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410310+0000", 2024-09-17T21:31:45.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.146e-06 2024-09-17T21:31:45.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457801+0000", 2024-09-17T21:31:45.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047491322000000002 2024-09-17T21:31:45.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457850+0000", 2024-09-17T21:31:45.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8585000000000003e-05 2024-09-17T21:31:45.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457894+0000", 2024-09-17T21:31:45.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4455e-05 2024-09-17T21:31:45.381 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.381 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.381 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.382 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17623 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 968704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:45.382 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:45.382 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.91905393899999, 2024-09-17T21:31:45.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047689377999999998, 2024-09-17T21:31:45.382 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.382 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.382 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:45.382 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.382 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.382 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.382 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17623 2024-09-17T21:31:45.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.383 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:45.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:45.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410314+0000", 2024-09-17T21:31:45.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3699999999999999e-07 2024-09-17T21:31:45.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410315+0000", 2024-09-17T21:31:45.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0499999999999999e-06 2024-09-17T21:31:45.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410319+0000", 2024-09-17T21:31:45.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.517e-06 2024-09-17T21:31:45.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410321+0000", 2024-09-17T21:31:45.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:31:45.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457922+0000", 2024-09-17T21:31:45.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047600638000000001 2024-09-17T21:31:45.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.457964+0000", 2024-09-17T21:31:45.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2126000000000003e-05 2024-09-17T21:31:45.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458003+0000", 2024-09-17T21:31:45.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9138999999999999e-05 2024-09-17T21:31:45.386 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.386 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.386 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.386 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17624 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:45.386 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:45.386 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.919043355, 2024-09-17T21:31:45.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047798292999999999, 2024-09-17T21:31:45.386 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.387 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.387 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:45.387 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.387 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.387 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.387 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17624 2024-09-17T21:31:45.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.387 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:45.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:45.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410325+0000", 2024-09-17T21:31:45.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:31:45.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410326+0000", 2024-09-17T21:31:45.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0610000000000001e-06 2024-09-17T21:31:45.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410329+0000", 2024-09-17T21:31:45.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0319999999999999e-06 2024-09-17T21:31:45.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410331+0000", 2024-09-17T21:31:45.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5749999999999999e-06 2024-09-17T21:31:45.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458033+0000", 2024-09-17T21:31:45.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047701371999999999 2024-09-17T21:31:45.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458079+0000", 2024-09-17T21:31:45.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:31:45.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458123+0000", 2024-09-17T21:31:45.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3850999999999997e-05 2024-09-17T21:31:45.390 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.390 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.390 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.391 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17625 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:45.391 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:45.391 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.919033195, 2024-09-17T21:31:45.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047883051000000003, 2024-09-17T21:31:45.391 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.391 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.391 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:45.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.391 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17625 2024-09-17T21:31:45.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.392 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:45.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:45.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410335+0000", 2024-09-17T21:31:45.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:31:45.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410336+0000", 2024-09-17T21:31:45.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1310000000000001e-06 2024-09-17T21:31:45.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410339+0000", 2024-09-17T21:31:45.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000001e-06 2024-09-17T21:31:45.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410342+0000", 2024-09-17T21:31:45.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7460000000000001e-06 2024-09-17T21:31:45.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458137+0000", 2024-09-17T21:31:45.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047794931999999998 2024-09-17T21:31:45.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458178+0000", 2024-09-17T21:31:45.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1013999999999997e-05 2024-09-17T21:31:45.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458218+0000", 2024-09-17T21:31:45.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9675999999999998e-05 2024-09-17T21:31:45.395 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.395 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.395 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.395 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17626 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1009664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:45.395 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:31:45.395 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.919022574, 2024-09-17T21:31:45.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047991601000000002, 2024-09-17T21:31:45.396 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.396 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.396 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:45.396 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.396 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.396 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.396 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17626 2024-09-17T21:31:45.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.396 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:31:45.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410345+0000", 2024-09-17T21:31:45.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410346+0000", 2024-09-17T21:31:45.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:31:45.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410347+0000", 2024-09-17T21:31:45.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.114e-06 2024-09-17T21:31:45.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410350+0000", 2024-09-17T21:31:45.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.101e-06 2024-09-17T21:31:45.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410353+0000", 2024-09-17T21:31:45.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T21:31:45.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458245+0000", 2024-09-17T21:31:45.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0478921 2024-09-17T21:31:45.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458292+0000", 2024-09-17T21:31:45.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397999999999999e-05 2024-09-17T21:31:45.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458337+0000", 2024-09-17T21:31:45.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5221e-05 2024-09-17T21:31:45.399 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.400 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.400 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.400 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17627 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:45.400 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:31:45.400 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.91898547300001, 2024-09-17T21:31:45.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048047383999999999, 2024-09-17T21:31:45.400 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.400 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.400 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:45.400 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.400 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.401 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17627 2024-09-17T21:31:45.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.401 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:31:45.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410382+0000", 2024-09-17T21:31:45.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410383+0000", 2024-09-17T21:31:45.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:31:45.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410385+0000", 2024-09-17T21:31:45.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:31:45.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410391+0000", 2024-09-17T21:31:45.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7389999999999996e-06 2024-09-17T21:31:45.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410395+0000", 2024-09-17T21:31:45.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6539999999999998e-06 2024-09-17T21:31:45.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458350+0000", 2024-09-17T21:31:45.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047954855999999997 2024-09-17T21:31:45.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458392+0000", 2024-09-17T21:31:45.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2094000000000001e-05 2024-09-17T21:31:45.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458430+0000", 2024-09-17T21:31:45.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7474000000000001e-05 2024-09-17T21:31:45.404 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.404 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.404 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.405 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17628 3.d 3:b04c3deb:::rbd_data.11202c1b6cb4.0000000000000010:4 [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:45.405 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:45.405 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.918965912, 2024-09-17T21:31:45.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048142676000000002, 2024-09-17T21:31:45.405 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.405 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.405 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:45.405 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.405 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.405 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.405 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17628 2024-09-17T21:31:45.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.406 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:45.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:45.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410402+0000", 2024-09-17T21:31:45.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:31:45.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410404+0000", 2024-09-17T21:31:45.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-06 2024-09-17T21:31:45.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410411+0000", 2024-09-17T21:31:45.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6850000000000004e-06 2024-09-17T21:31:45.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.410415+0000", 2024-09-17T21:31:45.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6750000000000001e-06 2024-09-17T21:31:45.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458457+0000", 2024-09-17T21:31:45.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048042045999999998 2024-09-17T21:31:45.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458501+0000", 2024-09-17T21:31:45.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3661000000000001e-05 2024-09-17T21:31:45.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458545+0000", 2024-09-17T21:31:45.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3529000000000003e-05 2024-09-17T21:31:45.409 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.409 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.409 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.409 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17687 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:45.409 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:31:45.409 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.916520959, 2024-09-17T21:31:45.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050670448, 2024-09-17T21:31:45.410 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.410 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.410 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:45.410 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.410 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.410 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.410 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17687 2024-09-17T21:31:45.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.410 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:31:45.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412847+0000", 2024-09-17T21:31:45.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.412848+0000", 2024-09-17T21:31:45.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9100000000000003e-07 2024-09-17T21:31:45.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413102+0000", 2024-09-17T21:31:45.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025415600000000001 2024-09-17T21:31:45.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413111+0000", 2024-09-17T21:31:45.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2010000000000005e-06 2024-09-17T21:31:45.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413116+0000", 2024-09-17T21:31:45.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0869999999999997e-06 2024-09-17T21:31:45.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458559+0000", 2024-09-17T21:31:45.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045442914000000001 2024-09-17T21:31:45.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458632+0000", 2024-09-17T21:31:45.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3207000000000006e-05 2024-09-17T21:31:45.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:31:45.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.458785+0000", 2024-09-17T21:31:45.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015226700000000001 2024-09-17T21:31:45.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:45.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.461648+0000", 2024-09-17T21:31:45.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028633069999999998 2024-09-17T21:31:45.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:45.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463416+0000", 2024-09-17T21:31:45.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017680230000000001 2024-09-17T21:31:45.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:45.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463475+0000", 2024-09-17T21:31:45.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9333000000000003e-05 2024-09-17T21:31:45.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463517+0000", 2024-09-17T21:31:45.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2162000000000001e-05 2024-09-17T21:31:45.415 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.415 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.415 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.415 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17688 3.d 3:b6db9af2:::rbd_data.11207744cc2c.0000000000000082:head [write 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:45.415 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:45.416 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.916242091, 2024-09-17T21:31:45.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050625098, 2024-09-17T21:31:45.416 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.416 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.416 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:45.416 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.416 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.416 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.416 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17688 2024-09-17T21:31:45.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.416 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:45.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:45.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413126+0000", 2024-09-17T21:31:45.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:31:45.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413128+0000", 2024-09-17T21:31:45.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.813e-06 2024-09-17T21:31:45.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413132+0000", 2024-09-17T21:31:45.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-17T21:31:45.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.413135+0000", 2024-09-17T21:31:45.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:31:45.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459756+0000", 2024-09-17T21:31:45.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046621293000000001 2024-09-17T21:31:45.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459816+0000", 2024-09-17T21:31:45.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9156000000000001e-05 2024-09-17T21:31:45.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:31:45.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.459912+0000", 2024-09-17T21:31:45.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6302000000000001e-05 2024-09-17T21:31:45.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:45.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463231+0000", 2024-09-17T21:31:45.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033188290000000001 2024-09-17T21:31:45.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:45.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463701+0000", 2024-09-17T21:31:45.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047068199999999999 2024-09-17T21:31:45.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:45.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463728+0000", 2024-09-17T21:31:45.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6726000000000001e-05 2024-09-17T21:31:45.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463751+0000", 2024-09-17T21:31:45.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2824e-05 2024-09-17T21:31:45.421 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.421 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.421 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.422 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:17691 3.d 3:b3a055f3:::rbd_data.11207744cc2c.0000000000000083:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:31:45.422 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:31:45.422 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.912697188, 2024-09-17T21:31:45.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049309370999999998, 2024-09-17T21:31:45.422 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.422 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.422 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:31:45.422 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.422 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.422 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.422 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 17691 2024-09-17T21:31:45.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.422 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:31:45.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416671+0000", 2024-09-17T21:31:45.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416672+0000", 2024-09-17T21:31:45.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.47e-07 2024-09-17T21:31:45.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416915+0000", 2024-09-17T21:31:45.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024344500000000001 2024-09-17T21:31:45.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416924+0000", 2024-09-17T21:31:45.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7739999999999993e-06 2024-09-17T21:31:45.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.416929+0000", 2024-09-17T21:31:45.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6990000000000002e-06 2024-09-17T21:31:45.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460414+0000", 2024-09-17T21:31:45.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043485466 2024-09-17T21:31:45.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460540+0000", 2024-09-17T21:31:45.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012608899999999999 2024-09-17T21:31:45.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T21:31:45.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.460746+0000", 2024-09-17T21:31:45.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000205577 2024-09-17T21:31:45.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:31:45.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.463800+0000", 2024-09-17T21:31:45.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030542030000000001 2024-09-17T21:31:45.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T21:31:45.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465867+0000", 2024-09-17T21:31:45.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020665929999999998 2024-09-17T21:31:45.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T21:31:45.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465914+0000", 2024-09-17T21:31:45.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497000000000003e-05 2024-09-17T21:31:45.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:29:49.465980+0000", 2024-09-17T21:31:45.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6081000000000006e-05 2024-09-17T21:31:45.427 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.427 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.427 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.428 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4384.0:70218 3.16 3:68692d8f:::rbd_data.1120b9193b9f.000000000000001b:head [stat,delete] snapc 1d=[1d] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-17T21:31:45.428 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:45.428 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 74.523730099000005, 2024-09-17T21:31:45.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.951400786, 2024-09-17T21:31:45.428 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T21:31:45.428 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T21:31:45.428 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T21:31:45.428 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T21:31:45.428 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4384", 2024-09-17T21:31:45.428 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.428 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 70218 2024-09-17T21:31:45.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.428 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T21:31:45.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T21:31:45.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:45.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T21:31:45.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:45.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T21:31:45.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T21:31:45.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805638+0000", 2024-09-17T21:31:45.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4799999999999998e-07 2024-09-17T21:31:45.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T21:31:45.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805642+0000", 2024-09-17T21:31:45.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2009999999999999e-06 2024-09-17T21:31:45.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T21:31:45.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805650+0000", 2024-09-17T21:31:45.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1000000000000004e-06 2024-09-17T21:31:45.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805655+0000", 2024-09-17T21:31:45.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:31:45.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805722+0000", 2024-09-17T21:31:45.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6878000000000006e-05 2024-09-17T21:31:45.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T21:31:45.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:30.805733+0000", 2024-09-17T21:31:45.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0895e-05 2024-09-17T21:31:45.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T21:31:45.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755252+0000", 2024-09-17T21:31:45.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.94951843800000002 2024-09-17T21:31:45.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T21:31:45.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.755398+0000", 2024-09-17T21:31:45.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014622600000000001 2024-09-17T21:31:45.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T21:31:45.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T21:31:45.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T21:31:45.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T21:30:31.757039+0000", 2024-09-17T21:31:45.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001640819 2024-09-17T21:31:45.432 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.432 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.432 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.432 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T21:31:45.433 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T21:31:45.433 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T21:31:45.434 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:31:45.434 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T21:31:45.432+0000 7f67341b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33372) UID: 0 2024-09-17T21:31:45.441 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:31:45.516 INFO:teuthology.orchestra.run.smithi129.stdout:All operations completed A-OK! 2024-09-17T21:31:45.533 DEBUG:teuthology.parallel:result is None 2024-09-17T21:31:45.533 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-17T21:31:45.541 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-17T21:31:45.618 INFO:tasks.thrashosds:joining thrashosds 2024-09-17T21:31:45.620 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:45.620 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:45.620 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T21:31:45.621 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:45.621 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:31:45.766 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:45.766 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T21:31:45.766 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T21:31:45.766 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T21:31:45.766 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:45.775 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:31:45.880 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:31:45.881 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T21:31:45.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T21:31:45.881 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T21:31:45.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.881 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97778 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 245760~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.881 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:31:45.881 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.910577146000001, 2024-09-17T21:31:45.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050933728999999997, 2024-09-17T21:31:45.881 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.881 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.881 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.882 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.882 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97778 2024-09-17T21:31:45.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.882 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:31:45.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966898+0000", 2024-09-17T21:31:45.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966899+0000", 2024-09-17T21:31:45.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2399999999999999e-07 2024-09-17T21:31:45.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966900+0000", 2024-09-17T21:31:45.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.384e-06 2024-09-17T21:31:45.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966905+0000", 2024-09-17T21:31:45.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6530000000000002e-06 2024-09-17T21:31:45.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966909+0000", 2024-09-17T21:31:45.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2889999999999999e-06 2024-09-17T21:31:45.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017585+0000", 2024-09-17T21:31:45.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050675691000000002 2024-09-17T21:31:45.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017642+0000", 2024-09-17T21:31:45.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7167999999999999e-05 2024-09-17T21:31:45.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017832+0000", 2024-09-17T21:31:45.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019022 2024-09-17T21:31:45.885 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.885 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.885 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.885 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97779 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.885 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:31:45.886 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.910560078000003, 2024-09-17T21:31:45.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051245027999999998, 2024-09-17T21:31:45.886 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.886 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.886 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.886 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.886 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.886 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.886 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97779 2024-09-17T21:31:45.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.886 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:31:45.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966915+0000", 2024-09-17T21:31:45.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966916+0000", 2024-09-17T21:31:45.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:31:45.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966918+0000", 2024-09-17T21:31:45.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8369999999999999e-06 2024-09-17T21:31:45.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966924+0000", 2024-09-17T21:31:45.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2260000000000002e-06 2024-09-17T21:31:45.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966928+0000", 2024-09-17T21:31:45.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3819999999999997e-06 2024-09-17T21:31:45.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017866+0000", 2024-09-17T21:31:45.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.050938079999999997 2024-09-17T21:31:45.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.017945+0000", 2024-09-17T21:31:45.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8275000000000005e-05 2024-09-17T21:31:45.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018161+0000", 2024-09-17T21:31:45.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021579499999999999 2024-09-17T21:31:45.890 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.890 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.890 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.890 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97780 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 344064~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.890 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:45.890 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.910535799999998, 2024-09-17T21:31:45.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051535301999999998, 2024-09-17T21:31:45.890 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.890 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.891 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.891 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.891 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.891 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.891 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97780 2024-09-17T21:31:45.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.891 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:45.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:45.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966940+0000", 2024-09-17T21:31:45.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:31:45.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966942+0000", 2024-09-17T21:31:45.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7439999999999999e-06 2024-09-17T21:31:45.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966946+0000", 2024-09-17T21:31:45.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5510000000000001e-06 2024-09-17T21:31:45.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966951+0000", 2024-09-17T21:31:45.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3699999999999997e-06 2024-09-17T21:31:45.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018191+0000", 2024-09-17T21:31:45.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051240290000000001 2024-09-17T21:31:45.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018269+0000", 2024-09-17T21:31:45.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7416999999999994e-05 2024-09-17T21:31:45.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018475+0000", 2024-09-17T21:31:45.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000206486 2024-09-17T21:31:45.894 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.894 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.894 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.895 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97789 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 40960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.895 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:45.895 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.910515631000003, 2024-09-17T21:31:45.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051862143999999999, 2024-09-17T21:31:45.895 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.895 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.895 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.895 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.895 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.895 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.895 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97789 2024-09-17T21:31:45.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.895 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:45.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:45.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966960+0000", 2024-09-17T21:31:45.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5899999999999996e-07 2024-09-17T21:31:45.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966962+0000", 2024-09-17T21:31:45.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9700000000000002e-06 2024-09-17T21:31:45.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966969+0000", 2024-09-17T21:31:45.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6939999999999998e-06 2024-09-17T21:31:45.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966980+0000", 2024-09-17T21:31:45.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0679999999999999e-05 2024-09-17T21:31:45.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018513+0000", 2024-09-17T21:31:45.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051533420000000003 2024-09-17T21:31:45.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018595+0000", 2024-09-17T21:31:45.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1260999999999993e-05 2024-09-17T21:31:45.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018822+0000", 2024-09-17T21:31:45.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022756 2024-09-17T21:31:45.899 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.899 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.899 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.899 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97790 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 90112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.899 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:31:45.899 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.910485133000002, 2024-09-17T21:31:45.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052173479000000002, 2024-09-17T21:31:45.899 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.899 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.900 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.900 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.900 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.900 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.900 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97790 2024-09-17T21:31:45.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.900 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:31:45.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966990+0000", 2024-09-17T21:31:45.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966991+0000", 2024-09-17T21:31:45.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9500000000000003e-07 2024-09-17T21:31:45.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.966993+0000", 2024-09-17T21:31:45.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.238e-06 2024-09-17T21:31:45.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967000+0000", 2024-09-17T21:31:45.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9500000000000004e-06 2024-09-17T21:31:45.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967005+0000", 2024-09-17T21:31:45.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:31:45.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018852+0000", 2024-09-17T21:31:45.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.051847250999999997 2024-09-17T21:31:45.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.018927+0000", 2024-09-17T21:31:45.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4944000000000006e-05 2024-09-17T21:31:45.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019164+0000", 2024-09-17T21:31:45.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000236803 2024-09-17T21:31:45.903 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.903 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.903 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.904 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97791 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 139264~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.904 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:31:45.904 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.910463389, 2024-09-17T21:31:45.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052464594000000003, 2024-09-17T21:31:45.904 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.904 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.904 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.904 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97791 2024-09-17T21:31:45.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.904 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:31:45.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967012+0000", 2024-09-17T21:31:45.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967013+0000", 2024-09-17T21:31:45.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:31:45.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967014+0000", 2024-09-17T21:31:45.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8679999999999999e-06 2024-09-17T21:31:45.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967020+0000", 2024-09-17T21:31:45.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.322e-06 2024-09-17T21:31:45.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967024+0000", 2024-09-17T21:31:45.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2660000000000003e-06 2024-09-17T21:31:45.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019193+0000", 2024-09-17T21:31:45.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052169010000000002 2024-09-17T21:31:45.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019269+0000", 2024-09-17T21:31:45.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6149999999999994e-05 2024-09-17T21:31:45.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019477+0000", 2024-09-17T21:31:45.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020753699999999999 2024-09-17T21:31:45.908 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.908 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.908 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.908 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97792 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 188416~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.908 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:45.908 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.91044565, 2024-09-17T21:31:45.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052764958000000001, 2024-09-17T21:31:45.908 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.908 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.908 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.909 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97792 2024-09-17T21:31:45.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.909 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:45.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:45.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967030+0000", 2024-09-17T21:31:45.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0900000000000002e-07 2024-09-17T21:31:45.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967032+0000", 2024-09-17T21:31:45.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9740000000000001e-06 2024-09-17T21:31:45.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967039+0000", 2024-09-17T21:31:45.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3679999999999998e-06 2024-09-17T21:31:45.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967043+0000", 2024-09-17T21:31:45.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3379999999999998e-06 2024-09-17T21:31:45.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019505+0000", 2024-09-17T21:31:45.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052462264000000002 2024-09-17T21:31:45.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019582+0000", 2024-09-17T21:31:45.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7146999999999998e-05 2024-09-17T21:31:45.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019795+0000", 2024-09-17T21:31:45.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000212458 2024-09-17T21:31:45.912 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.912 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.912 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.913 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97793 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 237568~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.913 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:31:45.913 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.910424149999997, 2024-09-17T21:31:45.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053070436999999998, 2024-09-17T21:31:45.913 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.913 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.913 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.913 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.913 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.913 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.913 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97793 2024-09-17T21:31:45.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.914 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:31:45.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967051+0000", 2024-09-17T21:31:45.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967052+0000", 2024-09-17T21:31:45.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7500000000000001e-07 2024-09-17T21:31:45.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967054+0000", 2024-09-17T21:31:45.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:31:45.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967059+0000", 2024-09-17T21:31:45.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5840000000000002e-06 2024-09-17T21:31:45.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967064+0000", 2024-09-17T21:31:45.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:31:45.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019826+0000", 2024-09-17T21:31:45.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.052762396000000003 2024-09-17T21:31:45.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.019900+0000", 2024-09-17T21:31:45.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3594e-05 2024-09-17T21:31:45.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020122+0000", 2024-09-17T21:31:45.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022185999999999999 2024-09-17T21:31:45.917 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.917 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.917 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.917 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97794 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 286720~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.917 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:45.918 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.910404014000001, 2024-09-17T21:31:45.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053453734000000003, 2024-09-17T21:31:45.918 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.918 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.918 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.918 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97794 2024-09-17T21:31:45.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.918 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:45.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:45.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967072+0000", 2024-09-17T21:31:45.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2199999999999999e-07 2024-09-17T21:31:45.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967074+0000", 2024-09-17T21:31:45.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8419999999999999e-06 2024-09-17T21:31:45.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967080+0000", 2024-09-17T21:31:45.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5660000000000003e-06 2024-09-17T21:31:45.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967120+0000", 2024-09-17T21:31:45.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9504e-05 2024-09-17T21:31:45.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020189+0000", 2024-09-17T21:31:45.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053069144999999998 2024-09-17T21:31:45.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020287+0000", 2024-09-17T21:31:45.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8259999999999998e-05 2024-09-17T21:31:45.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020525+0000", 2024-09-17T21:31:45.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023799499999999999 2024-09-17T21:31:45.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.922 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.922 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97795 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 335872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.922 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:45.922 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.910345880000001, 2024-09-17T21:31:45.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053707207999999999, 2024-09-17T21:31:45.922 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.922 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.922 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.922 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.923 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97795 2024-09-17T21:31:45.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.923 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:45.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:45.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967130+0000", 2024-09-17T21:31:45.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:31:45.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967139+0000", 2024-09-17T21:31:45.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5790000000000004e-06 2024-09-17T21:31:45.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967145+0000", 2024-09-17T21:31:45.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3550000000000001e-06 2024-09-17T21:31:45.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967155+0000", 2024-09-17T21:31:45.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8859999999999999e-06 2024-09-17T21:31:45.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020555+0000", 2024-09-17T21:31:45.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053400359000000001 2024-09-17T21:31:45.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020630+0000", 2024-09-17T21:31:45.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4894999999999994e-05 2024-09-17T21:31:45.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020837+0000", 2024-09-17T21:31:45.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020670500000000001 2024-09-17T21:31:45.926 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.926 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.926 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.927 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97796 3.19 3:9d38086a:::rbd_data.1120c318df8c.0000000000000001:24 [read 385024~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.927 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:45.927 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.910313619999997, 2024-09-17T21:31:45.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053990227000000002, 2024-09-17T21:31:45.927 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.927 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.927 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.927 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97796 2024-09-17T21:31:45.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.927 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:45.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:45.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967162+0000", 2024-09-17T21:31:45.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:31:45.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967164+0000", 2024-09-17T21:31:45.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0370000000000001e-06 2024-09-17T21:31:45.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967171+0000", 2024-09-17T21:31:45.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2190000000000001e-06 2024-09-17T21:31:45.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967175+0000", 2024-09-17T21:31:45.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2599999999999999e-06 2024-09-17T21:31:45.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020866+0000", 2024-09-17T21:31:45.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053691539000000003 2024-09-17T21:31:45.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.020942+0000", 2024-09-17T21:31:45.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5487000000000004e-05 2024-09-17T21:31:45.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021152+0000", 2024-09-17T21:31:45.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021021599999999999 2024-09-17T21:31:45.931 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.931 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.931 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.931 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97805 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 8192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.931 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:45.931 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.91029494, 2024-09-17T21:31:45.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054331771000000001, 2024-09-17T21:31:45.932 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.932 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.932 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.932 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97805 2024-09-17T21:31:45.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.932 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:45.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:45.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967181+0000", 2024-09-17T21:31:45.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:31:45.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967183+0000", 2024-09-17T21:31:45.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6840000000000001e-06 2024-09-17T21:31:45.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967189+0000", 2024-09-17T21:31:45.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0299999999999999e-06 2024-09-17T21:31:45.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967193+0000", 2024-09-17T21:31:45.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-17T21:31:45.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021182+0000", 2024-09-17T21:31:45.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.053989319000000001 2024-09-17T21:31:45.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021261+0000", 2024-09-17T21:31:45.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8460000000000004e-05 2024-09-17T21:31:45.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021512+0000", 2024-09-17T21:31:45.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025158500000000002 2024-09-17T21:31:45.935 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.935 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.936 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.936 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97806 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.936 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:45.936 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.910276936999999, 2024-09-17T21:31:45.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054654343000000001, 2024-09-17T21:31:45.936 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.936 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.936 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.937 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97806 2024-09-17T21:31:45.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.937 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:45.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:45.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967199+0000", 2024-09-17T21:31:45.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:31:45.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967201+0000", 2024-09-17T21:31:45.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.691e-06 2024-09-17T21:31:45.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967206+0000", 2024-09-17T21:31:45.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7780000000000004e-06 2024-09-17T21:31:45.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967224+0000", 2024-09-17T21:31:45.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7357999999999999e-05 2024-09-17T21:31:45.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021546+0000", 2024-09-17T21:31:45.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054322237000000002 2024-09-17T21:31:45.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021618+0000", 2024-09-17T21:31:45.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2007000000000004e-05 2024-09-17T21:31:45.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021853+0000", 2024-09-17T21:31:45.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023495800000000001 2024-09-17T21:31:45.940 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.940 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.940 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.941 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97807 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 106496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.941 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:31:45.941 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.910244536999997, 2024-09-17T21:31:45.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055014183000000001, 2024-09-17T21:31:45.941 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.941 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.941 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.941 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.941 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.942 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97807 2024-09-17T21:31:45.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.942 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:31:45.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967231+0000", 2024-09-17T21:31:45.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967232+0000", 2024-09-17T21:31:45.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:31:45.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967234+0000", 2024-09-17T21:31:45.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.283e-06 2024-09-17T21:31:45.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967240+0000", 2024-09-17T21:31:45.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.144e-06 2024-09-17T21:31:45.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967252+0000", 2024-09-17T21:31:45.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2322e-05 2024-09-17T21:31:45.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021882+0000", 2024-09-17T21:31:45.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.054629952000000002 2024-09-17T21:31:45.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.021952+0000", 2024-09-17T21:31:45.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9919999999999995e-05 2024-09-17T21:31:45.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022245+0000", 2024-09-17T21:31:45.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00029302100000000003 2024-09-17T21:31:45.945 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.945 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.945 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.946 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97808 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 155648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.946 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:45.946 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.910215938, 2024-09-17T21:31:45.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055447485999999997, 2024-09-17T21:31:45.946 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.946 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.946 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.946 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.946 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.946 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.946 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97808 2024-09-17T21:31:45.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.947 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:45.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:45.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967260+0000", 2024-09-17T21:31:45.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4299999999999998e-07 2024-09-17T21:31:45.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967267+0000", 2024-09-17T21:31:45.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9820000000000002e-06 2024-09-17T21:31:45.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967273+0000", 2024-09-17T21:31:45.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7710000000000003e-06 2024-09-17T21:31:45.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967277+0000", 2024-09-17T21:31:45.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5050000000000001e-06 2024-09-17T21:31:45.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022297+0000", 2024-09-17T21:31:45.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055020131999999999 2024-09-17T21:31:45.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022379+0000", 2024-09-17T21:31:45.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1810000000000004e-05 2024-09-17T21:31:45.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022707+0000", 2024-09-17T21:31:45.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00032784300000000003 2024-09-17T21:31:45.950 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.950 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.950 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.950 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97809 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.950 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:31:45.950 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.910192422999998, 2024-09-17T21:31:45.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056012109999999997, 2024-09-17T21:31:45.950 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.950 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.950 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.951 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97809 2024-09-17T21:31:45.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.951 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:31:45.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967283+0000", 2024-09-17T21:31:45.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967284+0000", 2024-09-17T21:31:45.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:31:45.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967285+0000", 2024-09-17T21:31:45.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.826e-06 2024-09-17T21:31:45.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967291+0000", 2024-09-17T21:31:45.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:31:45.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967302+0000", 2024-09-17T21:31:45.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1114999999999999e-05 2024-09-17T21:31:45.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022901+0000", 2024-09-17T21:31:45.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.055599335 2024-09-17T21:31:45.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.022990+0000", 2024-09-17T21:31:45.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8671999999999994e-05 2024-09-17T21:31:45.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023295+0000", 2024-09-17T21:31:45.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00030554900000000001 2024-09-17T21:31:45.954 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.954 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.954 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.955 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97810 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 253952~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.955 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:45.955 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.910165995, 2024-09-17T21:31:45.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056355668999999997, 2024-09-17T21:31:45.955 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.955 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.955 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.955 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97810 2024-09-17T21:31:45.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.955 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:45.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:45.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967310+0000", 2024-09-17T21:31:45.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0200000000000002e-07 2024-09-17T21:31:45.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967313+0000", 2024-09-17T21:31:45.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4549999999999998e-06 2024-09-17T21:31:45.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967319+0000", 2024-09-17T21:31:45.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3080000000000004e-06 2024-09-17T21:31:45.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967324+0000", 2024-09-17T21:31:45.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0159999999999999e-06 2024-09-17T21:31:45.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023342+0000", 2024-09-17T21:31:45.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056018635999999997 2024-09-17T21:31:45.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023423+0000", 2024-09-17T21:31:45.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0473e-05 2024-09-17T21:31:45.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023665+0000", 2024-09-17T21:31:45.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000242279 2024-09-17T21:31:45.959 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.959 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.959 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.959 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97811 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 303104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.959 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:45.959 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.910139737000002, 2024-09-17T21:31:45.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056743978, 2024-09-17T21:31:45.959 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.960 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.960 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.960 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.960 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.960 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.960 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97811 2024-09-17T21:31:45.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.960 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:45.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:45.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967336+0000", 2024-09-17T21:31:45.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1300000000000001e-07 2024-09-17T21:31:45.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967338+0000", 2024-09-17T21:31:45.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9740000000000001e-06 2024-09-17T21:31:45.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967346+0000", 2024-09-17T21:31:45.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8129999999999996e-06 2024-09-17T21:31:45.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967351+0000", 2024-09-17T21:31:45.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6129999999999997e-06 2024-09-17T21:31:45.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023750+0000", 2024-09-17T21:31:45.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056399796000000002 2024-09-17T21:31:45.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.023839+0000", 2024-09-17T21:31:45.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.852e-05 2024-09-17T21:31:45.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024080+0000", 2024-09-17T21:31:45.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024084900000000001 2024-09-17T21:31:45.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.964 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.964 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.964 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:97812 3.19 3:9aa132b7:::rbd_data.1120c318df8c.0000000000000005:24 [read 352256~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:31:45.964 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:31:45.964 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.910119121000001, 2024-09-17T21:31:45.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.057024455000000002, 2024-09-17T21:31:45.964 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.964 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.964 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.965 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 97812 2024-09-17T21:31:45.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.965 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:31:45.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967356+0000", 2024-09-17T21:31:45.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967357+0000", 2024-09-17T21:31:45.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T21:31:45.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967359+0000", 2024-09-17T21:31:45.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0660000000000002e-06 2024-09-17T21:31:45.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967365+0000", 2024-09-17T21:31:45.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:31:45.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:44.967369+0000", 2024-09-17T21:31:45.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5029999999999999e-06 2024-09-17T21:31:45.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024109+0000", 2024-09-17T21:31:45.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.056740038 2024-09-17T21:31:45.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024175+0000", 2024-09-17T21:31:45.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5106999999999999e-05 2024-09-17T21:31:45.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:30:45.024381+0000", 2024-09-17T21:31:45.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020629700000000001 2024-09-17T21:31:45.968 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.968 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.968 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.968 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4384.0:112033 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-09-17T21:31:45.969 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T21:31:11.421210+0000", 2024-09-17T21:31:45.969 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.456265354999999, 2024-09-17T21:31:45.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0104142869999999, 2024-09-17T21:31:45.969 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T21:31:45.969 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T21:31:45.969 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T21:31:45.969 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T21:31:45.969 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4384", 2024-09-17T21:31:45.969 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/4128710039", 2024-09-17T21:31:45.969 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 112033 2024-09-17T21:31:45.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.969 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T21:31:45.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T21:31:45.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421210+0000", 2024-09-17T21:31:45.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T21:31:45.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421210+0000", 2024-09-17T21:31:45.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T21:31:45.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T21:31:45.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421212+0000", 2024-09-17T21:31:45.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.376e-06 2024-09-17T21:31:45.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T21:31:45.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421217+0000", 2024-09-17T21:31:45.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.259e-06 2024-09-17T21:31:45.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T21:31:45.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421228+0000", 2024-09-17T21:31:45.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0944000000000001e-05 2024-09-17T21:31:45.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T21:31:45.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421238+0000", 2024-09-17T21:31:45.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:31:45.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421626+0000", 2024-09-17T21:31:45.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00038804099999999999 2024-09-17T21:31:45.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-17T21:31:45.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:11.421637+0000", 2024-09-17T21:31:45.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1783000000000001e-05 2024-09-17T21:31:45.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T21:31:45.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:12.431245+0000", 2024-09-17T21:31:45.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.00960765 2024-09-17T21:31:45.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T21:31:45.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:12.431413+0000", 2024-09-17T21:31:45.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000167784 2024-09-17T21:31:45.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T21:31:45.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T21:31:45.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T21:31:45.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T21:31:12.431624+0000", 2024-09-17T21:31:45.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000211704 2024-09-17T21:31:45.973 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.973 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.974 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.974 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T21:31:45.974 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T21:31:45.974 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:31:46.602 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is unset 2024-09-17T21:31:48.619 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:31:48.913 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-17T21:31:49.024 INFO:teuthology.orchestra.run.smithi129.stdout:mon.a: {} 2024-09-17T21:31:49.024 INFO:teuthology.orchestra.run.smithi129.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T21:31:49.034 INFO:teuthology.orchestra.run.smithi129.stdout:mon.b: {} 2024-09-17T21:31:49.034 INFO:teuthology.orchestra.run.smithi129.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T21:31:49.042 INFO:teuthology.orchestra.run.smithi129.stdout:mon.c: {} 2024-09-17T21:31:49.043 INFO:teuthology.orchestra.run.smithi129.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T21:31:49.053 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-17T21:31:49.610 INFO:teuthology.orchestra.run.smithi129.stderr:noscrub is unset 2024-09-17T21:31:49.610 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T21:31:49.630 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-17T21:31:49.631 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:31:50.615 INFO:teuthology.orchestra.run.smithi129.stderr:nodeep-scrub is unset 2024-09-17T21:31:50.616 INFO:teuthology.orchestra.run.smithi129.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-17T21:31:50.634 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-17T21:31:51.621 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T21:31:51.637 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-17T21:31:52.419 INFO:teuthology.orchestra.run.smithi129.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-17T21:31:52.435 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T21:31:53.416 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T21:31:53.432 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-17T21:31:54.629 INFO:teuthology.orchestra.run.smithi129.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-17T21:31:54.648 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T21:31:55.636 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T21:31:55.654 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-17T21:31:56.646 INFO:teuthology.orchestra.run.smithi129.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-17T21:31:56.663 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T21:31:57.663 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T21:31:57.679 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-17T21:31:58.659 INFO:teuthology.orchestra.run.smithi129.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-17T21:31:58.676 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-17T21:31:59.668 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.6 to 1 (10000) 2024-09-17T21:31:59.684 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-17T21:32:00.678 INFO:teuthology.orchestra.run.smithi129.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-17T21:32:00.693 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-17T21:32:01.683 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.7 to 1 (10000) 2024-09-17T21:32:01.699 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-17T21:32:02.686 INFO:teuthology.orchestra.run.smithi129.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-17T21:32:02.704 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T21:32:03.696 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T21:32:03.712 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-17T21:32:04.703 INFO:teuthology.orchestra.run.smithi129.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-17T21:32:04.721 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-17T21:32:04.721 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-17T21:32:04.721 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-17T21:32:04.722 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-17T21:32:04.722 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T21:32:04.722 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T21:32:05.012 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:32:05.013 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":154,"fsid":"f6d3a80c-8a1f-4716-b25d-e97b28890c0d","created":"2024-09-17T21:29:12.606637+0000","modified":"2024-09-17T21:32:04.695621+0000","last_up_change":"2024-09-17T21:30:27.721456+0000","last_in_change":"2024-09-17T21:31:10.410190+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T21:29:26.876763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T21:29:31.078466+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T21:29:39.892418+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":136,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ad067b51-9402-4f69-a0f0-08fbf068dcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6801","nonce":3635367421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6803","nonce":3635367421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6807","nonce":3635367421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6805","nonce":3635367421}]},"public_addr":"172.21.15.129:6801/3635367421","cluster_addr":"172.21.15.129:6803/3635367421","heartbeat_back_addr":"172.21.15.129:6807/3635367421","heartbeat_front_addr":"172.21.15.129:6805/3635367421","state":["exists","up"]},{"osd":1,"uuid":"30385fc0-ef5f-45f5-a3ec-37e890d4826e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":47,"up_from":63,"up_thru":104,"down_at":48,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":842542464},{"type":"v1","addr":"172.21.15.129:6819","nonce":842542464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":842542464},{"type":"v1","addr":"172.21.15.129:6821","nonce":842542464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":842542464},{"type":"v1","addr":"172.21.15.129:6825","nonce":842542464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":842542464},{"type":"v1","addr":"172.21.15.129:6823","nonce":842542464}]},"public_addr":"172.21.15.129:6819/842542464","cluster_addr":"172.21.15.129:6821/842542464","heartbeat_back_addr":"172.21.15.129:6825/842542464","heartbeat_front_addr":"172.21.15.129:6823/842542464","state":["exists","up"]},{"osd":2,"uuid":"76be9b2b-b63f-402a-9ab5-2564bd671ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6809","nonce":1790706405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6811","nonce":1790706405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6815","nonce":1790706405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6813","nonce":1790706405}]},"public_addr":"172.21.15.129:6809/1790706405","cluster_addr":"172.21.15.129:6811/1790706405","heartbeat_back_addr":"172.21.15.129:6815/1790706405","heartbeat_front_addr":"172.21.15.129:6813/1790706405","state":["exists","up"]},{"osd":3,"uuid":"d6c34e3b-1ac6-415c-9c56-38ae77a5c9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6827","nonce":393361453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6829","nonce":393361453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6833","nonce":393361453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6831","nonce":393361453}]},"public_addr":"172.21.15.129:6827/393361453","cluster_addr":"172.21.15.129:6829/393361453","heartbeat_back_addr":"172.21.15.129:6833/393361453","heartbeat_front_addr":"172.21.15.129:6831/393361453","state":["exists","up"]},{"osd":4,"uuid":"7d722e8c-c497-4f0c-9f61-d9826f8c93d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6809","nonce":2337546183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6811","nonce":2337546183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6815","nonce":2337546183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6813","nonce":2337546183}]},"public_addr":"172.21.15.171:6809/2337546183","cluster_addr":"172.21.15.171:6811/2337546183","heartbeat_back_addr":"172.21.15.171:6815/2337546183","heartbeat_front_addr":"172.21.15.171:6813/2337546183","state":["exists","up"]},{"osd":5,"uuid":"954f801b-3ac9-410a-a375-eb1c533744de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6817","nonce":1510320351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6819","nonce":1510320351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6823","nonce":1510320351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6821","nonce":1510320351}]},"public_addr":"172.21.15.171:6817/1510320351","cluster_addr":"172.21.15.171:6819/1510320351","heartbeat_back_addr":"172.21.15.171:6823/1510320351","heartbeat_front_addr":"172.21.15.171:6821/1510320351","state":["exists","up"]},{"osd":6,"uuid":"21837515-65a2-4f21-96d6-67cfc4589358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6825","nonce":471043930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6827","nonce":471043930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6831","nonce":471043930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6829","nonce":471043930}]},"public_addr":"172.21.15.171:6825/471043930","cluster_addr":"172.21.15.171:6827/471043930","heartbeat_back_addr":"172.21.15.171:6831/471043930","heartbeat_front_addr":"172.21.15.171:6829/471043930","state":["exists","up"]},{"osd":7,"uuid":"67af7eea-8803-425d-ae50-30a6ab91a711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6801","nonce":1770619177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6803","nonce":1770619177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6807","nonce":1770619177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6805","nonce":1770619177}]},"public_addr":"172.21.15.171:6801/1770619177","cluster_addr":"172.21.15.171:6803/1770619177","heartbeat_back_addr":"172.21.15.171:6807/1770619177","heartbeat_front_addr":"172.21.15.171:6805/1770619177","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:21.445879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.240967+0000","dead_epoch":48},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.039097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.372014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.049543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.009384+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.048598+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.046345+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:32:05.026 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T21:32:05.026 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T21:32:05.314 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:32:05.314 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":154,"fsid":"f6d3a80c-8a1f-4716-b25d-e97b28890c0d","created":"2024-09-17T21:29:12.606637+0000","modified":"2024-09-17T21:32:04.695621+0000","last_up_change":"2024-09-17T21:30:27.721456+0000","last_in_change":"2024-09-17T21:31:10.410190+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T21:29:26.876763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T21:29:31.078466+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T21:29:39.892418+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":136,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ad067b51-9402-4f69-a0f0-08fbf068dcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6801","nonce":3635367421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6803","nonce":3635367421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6807","nonce":3635367421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6805","nonce":3635367421}]},"public_addr":"172.21.15.129:6801/3635367421","cluster_addr":"172.21.15.129:6803/3635367421","heartbeat_back_addr":"172.21.15.129:6807/3635367421","heartbeat_front_addr":"172.21.15.129:6805/3635367421","state":["exists","up"]},{"osd":1,"uuid":"30385fc0-ef5f-45f5-a3ec-37e890d4826e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":47,"up_from":63,"up_thru":104,"down_at":48,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":842542464},{"type":"v1","addr":"172.21.15.129:6819","nonce":842542464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":842542464},{"type":"v1","addr":"172.21.15.129:6821","nonce":842542464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":842542464},{"type":"v1","addr":"172.21.15.129:6825","nonce":842542464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":842542464},{"type":"v1","addr":"172.21.15.129:6823","nonce":842542464}]},"public_addr":"172.21.15.129:6819/842542464","cluster_addr":"172.21.15.129:6821/842542464","heartbeat_back_addr":"172.21.15.129:6825/842542464","heartbeat_front_addr":"172.21.15.129:6823/842542464","state":["exists","up"]},{"osd":2,"uuid":"76be9b2b-b63f-402a-9ab5-2564bd671ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6809","nonce":1790706405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6811","nonce":1790706405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6815","nonce":1790706405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6813","nonce":1790706405}]},"public_addr":"172.21.15.129:6809/1790706405","cluster_addr":"172.21.15.129:6811/1790706405","heartbeat_back_addr":"172.21.15.129:6815/1790706405","heartbeat_front_addr":"172.21.15.129:6813/1790706405","state":["exists","up"]},{"osd":3,"uuid":"d6c34e3b-1ac6-415c-9c56-38ae77a5c9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6827","nonce":393361453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6829","nonce":393361453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6833","nonce":393361453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6831","nonce":393361453}]},"public_addr":"172.21.15.129:6827/393361453","cluster_addr":"172.21.15.129:6829/393361453","heartbeat_back_addr":"172.21.15.129:6833/393361453","heartbeat_front_addr":"172.21.15.129:6831/393361453","state":["exists","up"]},{"osd":4,"uuid":"7d722e8c-c497-4f0c-9f61-d9826f8c93d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6809","nonce":2337546183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6811","nonce":2337546183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6815","nonce":2337546183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6813","nonce":2337546183}]},"public_addr":"172.21.15.171:6809/2337546183","cluster_addr":"172.21.15.171:6811/2337546183","heartbeat_back_addr":"172.21.15.171:6815/2337546183","heartbeat_front_addr":"172.21.15.171:6813/2337546183","state":["exists","up"]},{"osd":5,"uuid":"954f801b-3ac9-410a-a375-eb1c533744de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6817","nonce":1510320351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6819","nonce":1510320351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6823","nonce":1510320351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6821","nonce":1510320351}]},"public_addr":"172.21.15.171:6817/1510320351","cluster_addr":"172.21.15.171:6819/1510320351","heartbeat_back_addr":"172.21.15.171:6823/1510320351","heartbeat_front_addr":"172.21.15.171:6821/1510320351","state":["exists","up"]},{"osd":6,"uuid":"21837515-65a2-4f21-96d6-67cfc4589358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6825","nonce":471043930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6827","nonce":471043930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6831","nonce":471043930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6829","nonce":471043930}]},"public_addr":"172.21.15.171:6825/471043930","cluster_addr":"172.21.15.171:6827/471043930","heartbeat_back_addr":"172.21.15.171:6831/471043930","heartbeat_front_addr":"172.21.15.171:6829/471043930","state":["exists","up"]},{"osd":7,"uuid":"67af7eea-8803-425d-ae50-30a6ab91a711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6801","nonce":1770619177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6803","nonce":1770619177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6807","nonce":1770619177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6805","nonce":1770619177}]},"public_addr":"172.21.15.171:6801/1770619177","cluster_addr":"172.21.15.171:6803/1770619177","heartbeat_back_addr":"172.21.15.171:6807/1770619177","heartbeat_front_addr":"172.21.15.171:6805/1770619177","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:21.445879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.240967+0000","dead_epoch":48},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.039097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.372014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.049543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.009384+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.048598+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.046345+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:32:05.327 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T21:32:05.328 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T21:32:05.328 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T21:32:05.328 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T21:32:05.329 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T21:32:05.329 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T21:32:05.330 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-17T21:32:05.330 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-17T21:32:05.575 INFO:teuthology.orchestra.run.smithi129.stdout:55834574886 2024-09-17T21:32:05.575 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T21:32:05.576 INFO:teuthology.orchestra.run.smithi129.stdout:270582939669 2024-09-17T21:32:05.576 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T21:32:05.577 INFO:teuthology.orchestra.run.smithi129.stdout:64424509476 2024-09-17T21:32:05.578 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T21:32:05.578 INFO:teuthology.orchestra.run.smithi129.stdout:64424509476 2024-09-17T21:32:05.579 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T21:32:05.580 INFO:teuthology.orchestra.run.smithi129.stdout:64424509476 2024-09-17T21:32:05.580 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T21:32:05.582 INFO:teuthology.orchestra.run.smithi129.stdout:64424509475 2024-09-17T21:32:05.582 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T21:32:05.583 INFO:teuthology.orchestra.run.smithi129.stdout:64424509475 2024-09-17T21:32:05.583 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T21:32:05.584 INFO:teuthology.orchestra.run.smithi129.stdout:64424509475 2024-09-17T21:32:05.585 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T21:32:06.136 INFO:teuthology.orchestra.run.smithi129.stdout:270582939668 2024-09-17T21:32:06.156 INFO:tasks.ceph.ceph_manager.ceph:need seq 270582939669 got 270582939668 for osd.1 2024-09-17T21:32:06.173 INFO:teuthology.orchestra.run.smithi129.stdout:55834574885 2024-09-17T21:32:06.181 INFO:teuthology.orchestra.run.smithi129.stdout:64424509475 2024-09-17T21:32:06.183 INFO:teuthology.orchestra.run.smithi129.stdout:64424509475 2024-09-17T21:32:06.194 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574886 got 55834574885 for osd.0 2024-09-17T21:32:06.199 INFO:teuthology.orchestra.run.smithi129.stdout:64424509475 2024-09-17T21:32:06.200 INFO:teuthology.orchestra.run.smithi129.stdout:64424509474 2024-09-17T21:32:06.203 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509476 got 64424509475 for osd.2 2024-09-17T21:32:06.207 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509476 got 64424509475 for osd.3 2024-09-17T21:32:06.214 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509476 got 64424509475 for osd.5 2024-09-17T21:32:06.217 INFO:teuthology.orchestra.run.smithi129.stdout:64424509474 2024-09-17T21:32:06.219 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509475 got 64424509474 for osd.4 2024-09-17T21:32:06.220 INFO:teuthology.orchestra.run.smithi129.stdout:64424509474 2024-09-17T21:32:06.232 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509475 got 64424509474 for osd.7 2024-09-17T21:32:06.234 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509475 got 64424509474 for osd.6 2024-09-17T21:32:07.157 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T21:32:07.194 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T21:32:07.204 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T21:32:07.209 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T21:32:07.215 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T21:32:07.221 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T21:32:07.233 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T21:32:07.234 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T21:32:07.683 INFO:teuthology.orchestra.run.smithi129.stdout:270582939668 2024-09-17T21:32:07.704 INFO:tasks.ceph.ceph_manager.ceph:need seq 270582939669 got 270582939668 for osd.1 2024-09-17T21:32:07.778 INFO:teuthology.orchestra.run.smithi129.stdout:64424509477 2024-09-17T21:32:07.778 INFO:teuthology.orchestra.run.smithi129.stdout:55834574886 2024-09-17T21:32:07.794 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574886 got 55834574886 for osd.0 2024-09-17T21:32:07.794 DEBUG:teuthology.parallel:result is None 2024-09-17T21:32:07.796 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509476 got 64424509477 for osd.3 2024-09-17T21:32:07.796 DEBUG:teuthology.parallel:result is None 2024-09-17T21:32:07.824 INFO:teuthology.orchestra.run.smithi129.stdout:64424509476 2024-09-17T21:32:07.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509475 got 64424509476 for osd.4 2024-09-17T21:32:07.838 DEBUG:teuthology.parallel:result is None 2024-09-17T21:32:07.846 INFO:teuthology.orchestra.run.smithi129.stdout:64424509476 2024-09-17T21:32:07.857 INFO:teuthology.orchestra.run.smithi129.stdout:64424509477 2024-09-17T21:32:07.861 INFO:teuthology.orchestra.run.smithi129.stdout:64424509476 2024-09-17T21:32:07.864 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509475 got 64424509476 for osd.7 2024-09-17T21:32:07.864 DEBUG:teuthology.parallel:result is None 2024-09-17T21:32:07.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509476 got 64424509477 for osd.2 2024-09-17T21:32:07.873 DEBUG:teuthology.parallel:result is None 2024-09-17T21:32:07.873 INFO:teuthology.orchestra.run.smithi129.stdout:64424509477 2024-09-17T21:32:07.879 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509475 got 64424509476 for osd.6 2024-09-17T21:32:07.879 DEBUG:teuthology.parallel:result is None 2024-09-17T21:32:07.888 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509476 got 64424509477 for osd.5 2024-09-17T21:32:07.888 DEBUG:teuthology.parallel:result is None 2024-09-17T21:32:08.704 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T21:32:09.004 INFO:teuthology.orchestra.run.smithi129.stdout:270582939670 2024-09-17T21:32:09.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 270582939669 got 270582939670 for osd.1 2024-09-17T21:32:09.018 DEBUG:teuthology.parallel:result is None 2024-09-17T21:32:09.019 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T21:32:09.019 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:32:09.303 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:32:09.303 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-17T21:32:09.315 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-17T21:32:08.887764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":660,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":660,"num_whiteouts":652,"num_read":159422,"num_read_kb":894014,"num_write":21746,"num_write_kb":454251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1208,"num_bytes_recovered":117790844,"num_keys_recovered":327,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19172,"ondisk_log_size":19172,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379400,"kb_used_data":14468,"kb_used_omap":31,"kb_used_meta":364704,"kb_avail":754595320,"statfs":{"total":773094113280,"available":772705607680,"internally_reserved":0,"allocated":14815232,"data_stored":8183763,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":32080,"internal_metadata":373457584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.202366"},"pg_stats":[{"pgid":"3.1f","version":"136'809","reported_seq":5134,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248455+0000","last_change":"2024-09-17T21:31:45.430953+0000","last_active":"2024-09-17T21:32:05.248455+0000","last_peered":"2024-09-17T21:32:05.248455+0000","last_clean":"2024-09-17T21:32:05.248455+0000","last_became_active":"2024-09-17T21:31:15.457506+0000","last_became_peered":"2024-09-17T21:31:15.457506+0000","last_unstale":"2024-09-17T21:32:05.248455+0000","last_undegraded":"2024-09-17T21:32:05.248455+0000","last_fullsized":"2024-09-17T21:32:05.248455+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:08.923954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098455079999999993,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6585,"num_read_kb":63347,"num_write":1151,"num_write_kb":32903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":27321072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"136'562","reported_seq":4234,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125356+0000","last_change":"2024-09-17T21:31:45.418470+0000","last_active":"2024-09-17T21:32:05.125356+0000","last_peered":"2024-09-17T21:32:05.125356+0000","last_clean":"2024-09-17T21:32:05.125356+0000","last_became_active":"2024-09-17T21:29:41.472161+0000","last_became_peered":"2024-09-17T21:29:41.472161+0000","last_unstale":"2024-09-17T21:32:05.125356+0000","last_undegraded":"2024-09-17T21:32:05.125356+0000","last_fullsized":"2024-09-17T21:32:05.125356+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:04:47.812939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079279,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3917,"num_read_kb":31515,"num_write":544,"num_write_kb":11792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"135'768","reported_seq":5402,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704245+0000","last_change":"2024-09-17T21:31:45.436342+0000","last_active":"2024-09-17T21:32:04.704245+0000","last_peered":"2024-09-17T21:32:04.704245+0000","last_clean":"2024-09-17T21:32:04.704245+0000","last_became_active":"2024-09-17T21:31:14.449672+0000","last_became_peered":"2024-09-17T21:31:14.449672+0000","last_unstale":"2024-09-17T21:32:04.704245+0000","last_undegraded":"2024-09-17T21:32:04.704245+0000","last_fullsized":"2024-09-17T21:32:04.704245+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:48:24.256603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010859089999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6612,"num_read_kb":51732,"num_write":893,"num_write_kb":18876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":15268754,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"136'433","reported_seq":2685,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704042+0000","last_change":"2024-09-17T21:31:45.435673+0000","last_active":"2024-09-17T21:32:04.704042+0000","last_peered":"2024-09-17T21:32:04.704042+0000","last_clean":"2024-09-17T21:32:04.704042+0000","last_became_active":"2024-09-17T21:31:14.451037+0000","last_became_peered":"2024-09-17T21:31:14.451037+0000","last_unstale":"2024-09-17T21:32:04.704042+0000","last_undegraded":"2024-09-17T21:32:04.704042+0000","last_fullsized":"2024-09-17T21:32:04.704042+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:41:20.330623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068460700000000005,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":2712,"num_read_kb":12910,"num_write":456,"num_write_kb":9120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":604511,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"136'589","reported_seq":8494,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248591+0000","last_change":"2024-09-17T21:31:45.430352+0000","last_active":"2024-09-17T21:32:05.248591+0000","last_peered":"2024-09-17T21:32:05.248591+0000","last_clean":"2024-09-17T21:32:05.248591+0000","last_became_active":"2024-09-17T21:29:41.476041+0000","last_became_peered":"2024-09-17T21:29:41.476041+0000","last_unstale":"2024-09-17T21:32:05.248591+0000","last_undegraded":"2024-09-17T21:32:05.248591+0000","last_fullsized":"2024-09-17T21:32:05.248591+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:45:15.271044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0092951189999999993,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8770,"num_read_kb":53504,"num_write":580,"num_write_kb":24876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"136'630","reported_seq":6573,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699780+0000","last_change":"2024-09-17T21:31:45.425285+0000","last_active":"2024-09-17T21:32:04.699780+0000","last_peered":"2024-09-17T21:32:04.699780+0000","last_clean":"2024-09-17T21:32:04.699780+0000","last_became_active":"2024-09-17T21:30:28.747031+0000","last_became_peered":"2024-09-17T21:30:28.747031+0000","last_unstale":"2024-09-17T21:32:04.699780+0000","last_undegraded":"2024-09-17T21:32:04.699780+0000","last_fullsized":"2024-09-17T21:32:04.699780+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:26.185752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00134892,"stat_sum":{"num_bytes":19,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":7065,"num_read_kb":21661,"num_write":665,"num_write_kb":6726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":235969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"135'424","reported_seq":4765,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700031+0000","last_change":"2024-09-17T21:31:45.427269+0000","last_active":"2024-09-17T21:32:04.700031+0000","last_peered":"2024-09-17T21:32:04.700031+0000","last_clean":"2024-09-17T21:32:04.700031+0000","last_became_active":"2024-09-17T21:30:28.745634+0000","last_became_peered":"2024-09-17T21:30:28.745634+0000","last_unstale":"2024-09-17T21:32:04.700031+0000","last_undegraded":"2024-09-17T21:32:04.700031+0000","last_fullsized":"2024-09-17T21:32:04.700031+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:58.579935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001320393,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4551,"num_read_kb":56994,"num_write":426,"num_write_kb":45225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":36017677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"136'512","reported_seq":4537,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125260+0000","last_change":"2024-09-17T21:31:45.420094+0000","last_active":"2024-09-17T21:32:05.125260+0000","last_peered":"2024-09-17T21:32:05.125260+0000","last_clean":"2024-09-17T21:32:05.125260+0000","last_became_active":"2024-09-17T21:29:41.472468+0000","last_became_peered":"2024-09-17T21:29:41.472468+0000","last_unstale":"2024-09-17T21:32:05.125260+0000","last_undegraded":"2024-09-17T21:32:05.125260+0000","last_fullsized":"2024-09-17T21:32:05.125260+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:12:23.877166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002623551,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4791,"num_read_kb":27686,"num_write":500,"num_write_kb":13101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'814","reported_seq":3250,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248533+0000","last_change":"2024-09-17T21:31:45.431351+0000","last_active":"2024-09-17T21:32:05.248533+0000","last_peered":"2024-09-17T21:32:05.248533+0000","last_clean":"2024-09-17T21:32:05.248533+0000","last_became_active":"2024-09-17T21:31:15.456240+0000","last_became_peered":"2024-09-17T21:31:15.456240+0000","last_unstale":"2024-09-17T21:32:05.248533+0000","last_undegraded":"2024-09-17T21:32:05.248533+0000","last_fullsized":"2024-09-17T21:32:05.248533+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:31:41.549775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010220406,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4952,"num_read_kb":22844,"num_write":955,"num_write_kb":30092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":9394579,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"136'1106","reported_seq":3090,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704029+0000","last_change":"2024-09-17T21:31:45.435544+0000","last_active":"2024-09-17T21:32:04.704029+0000","last_peered":"2024-09-17T21:32:04.704029+0000","last_clean":"2024-09-17T21:32:04.704029+0000","last_became_active":"2024-09-17T21:31:15.456612+0000","last_became_peered":"2024-09-17T21:31:15.456612+0000","last_unstale":"2024-09-17T21:32:04.704029+0000","last_undegraded":"2024-09-17T21:32:04.704029+0000","last_fullsized":"2024-09-17T21:32:04.704029+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":1106,"log_dups_size":0,"ondisk_log_size":1106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:12:11.830676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051663100000000003,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":5737,"num_read_kb":20891,"num_write":1670,"num_write_kb":12347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3063082,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"135'512","reported_seq":3188,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311120+0000","last_change":"2024-09-17T21:31:45.497882+0000","last_active":"2024-09-17T21:32:05.311120+0000","last_peered":"2024-09-17T21:32:05.311120+0000","last_clean":"2024-09-17T21:32:05.311120+0000","last_became_active":"2024-09-17T21:29:41.478346+0000","last_became_peered":"2024-09-17T21:29:41.478346+0000","last_unstale":"2024-09-17T21:32:05.311120+0000","last_undegraded":"2024-09-17T21:32:05.311120+0000","last_fullsized":"2024-09-17T21:32:05.311120+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:58.424829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017578019999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3628,"num_read_kb":9299,"num_write":426,"num_write_kb":8355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"135'297","reported_seq":2884,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699913+0000","last_change":"2024-09-17T21:31:45.424549+0000","last_active":"2024-09-17T21:32:04.699913+0000","last_peered":"2024-09-17T21:32:04.699913+0000","last_clean":"2024-09-17T21:32:04.699913+0000","last_became_active":"2024-09-17T21:29:41.477611+0000","last_became_peered":"2024-09-17T21:29:41.477611+0000","last_unstale":"2024-09-17T21:32:04.699913+0000","last_undegraded":"2024-09-17T21:32:04.699913+0000","last_fullsized":"2024-09-17T21:32:04.699913+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:59:07.448045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064521399999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2154,"num_read_kb":14079,"num_write":352,"num_write_kb":12746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":280,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.180954+0000","last_change":"2024-09-17T21:29:35.002656+0000","last_active":"2024-09-17T21:32:05.180954+0000","last_peered":"2024-09-17T21:32:05.180954+0000","last_clean":"2024-09-17T21:32:05.180954+0000","last_became_active":"2024-09-17T21:29:32.971677+0000","last_became_peered":"2024-09-17T21:29:32.971677+0000","last_unstale":"2024-09-17T21:32:05.180954+0000","last_undegraded":"2024-09-17T21:32:05.180954+0000","last_fullsized":"2024-09-17T21:32:05.180954+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:53.781657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041121300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"136'848","reported_seq":4463,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248518+0000","last_change":"2024-09-17T21:31:45.431468+0000","last_active":"2024-09-17T21:32:05.248518+0000","last_peered":"2024-09-17T21:32:05.248518+0000","last_clean":"2024-09-17T21:32:05.248518+0000","last_became_active":"2024-09-17T21:30:31.755039+0000","last_became_peered":"2024-09-17T21:30:31.755039+0000","last_unstale":"2024-09-17T21:32:05.248518+0000","last_undegraded":"2024-09-17T21:32:05.248518+0000","last_fullsized":"2024-09-17T21:32:05.248518+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.621804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01026987,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6950,"num_read_kb":18293,"num_write":963,"num_write_kb":8597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":448871,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":213,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700243+0000","last_change":"2024-09-17T21:30:28.745759+0000","last_active":"2024-09-17T21:32:04.700243+0000","last_peered":"2024-09-17T21:32:04.700243+0000","last_clean":"2024-09-17T21:32:04.700243+0000","last_became_active":"2024-09-17T21:30:28.744742+0000","last_became_peered":"2024-09-17T21:30:28.744742+0000","last_unstale":"2024-09-17T21:32:04.700243+0000","last_undegraded":"2024-09-17T21:32:04.700243+0000","last_fullsized":"2024-09-17T21:32:04.700243+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:37:02.014566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"134'592","reported_seq":2794,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125373+0000","last_change":"2024-09-17T21:31:45.420595+0000","last_active":"2024-09-17T21:32:05.125373+0000","last_peered":"2024-09-17T21:32:05.125373+0000","last_clean":"2024-09-17T21:32:05.125373+0000","last_became_active":"2024-09-17T21:29:41.472095+0000","last_became_peered":"2024-09-17T21:29:41.472095+0000","last_unstale":"2024-09-17T21:32:05.125373+0000","last_undegraded":"2024-09-17T21:32:05.125373+0000","last_fullsized":"2024-09-17T21:32:05.125373+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:07:43.653815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028983989999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":3726,"num_read_kb":13082,"num_write":700,"num_write_kb":8995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":282,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311127+0000","last_change":"2024-09-17T21:29:35.078854+0000","last_active":"2024-09-17T21:32:05.311127+0000","last_peered":"2024-09-17T21:32:05.311127+0000","last_clean":"2024-09-17T21:32:05.311127+0000","last_became_active":"2024-09-17T21:29:33.071665+0000","last_became_peered":"2024-09-17T21:29:33.071665+0000","last_unstale":"2024-09-17T21:32:05.311127+0000","last_undegraded":"2024-09-17T21:32:05.311127+0000","last_fullsized":"2024-09-17T21:32:05.311127+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.415687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047969899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"134'805","reported_seq":4875,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700392+0000","last_change":"2024-09-17T21:31:45.426864+0000","last_active":"2024-09-17T21:32:04.700392+0000","last_peered":"2024-09-17T21:32:04.700392+0000","last_clean":"2024-09-17T21:32:04.700392+0000","last_became_active":"2024-09-17T21:30:31.752750+0000","last_became_peered":"2024-09-17T21:30:31.752750+0000","last_unstale":"2024-09-17T21:32:04.700392+0000","last_undegraded":"2024-09-17T21:32:04.700392+0000","last_fullsized":"2024-09-17T21:32:04.700392+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:27:53.025416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094011400000000003,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6270,"num_read_kb":32698,"num_write":1050,"num_write_kb":17424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3231045,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":213,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700238+0000","last_change":"2024-09-17T21:30:28.748319+0000","last_active":"2024-09-17T21:32:04.700238+0000","last_peered":"2024-09-17T21:32:04.700238+0000","last_clean":"2024-09-17T21:32:04.700238+0000","last_became_active":"2024-09-17T21:30:28.747728+0000","last_became_peered":"2024-09-17T21:30:28.747728+0000","last_unstale":"2024-09-17T21:32:04.700238+0000","last_undegraded":"2024-09-17T21:32:04.700238+0000","last_fullsized":"2024-09-17T21:32:04.700238+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:58:18.954005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"135'502","reported_seq":4854,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704234+0000","last_change":"2024-09-17T21:31:45.436069+0000","last_active":"2024-09-17T21:32:04.704234+0000","last_peered":"2024-09-17T21:32:04.704234+0000","last_clean":"2024-09-17T21:32:04.704234+0000","last_became_active":"2024-09-17T21:31:14.455957+0000","last_became_peered":"2024-09-17T21:31:14.455957+0000","last_unstale":"2024-09-17T21:32:04.704234+0000","last_undegraded":"2024-09-17T21:32:04.704234+0000","last_fullsized":"2024-09-17T21:32:04.704234+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:43:55.461066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087537200000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4988,"num_read_kb":31633,"num_write":558,"num_write_kb":9789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":1545182,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"136'856","reported_seq":5026,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699955+0000","last_change":"2024-09-17T21:31:45.424972+0000","last_active":"2024-09-17T21:32:04.699955+0000","last_peered":"2024-09-17T21:32:04.699955+0000","last_clean":"2024-09-17T21:32:04.699955+0000","last_became_active":"2024-09-17T21:29:41.475273+0000","last_became_peered":"2024-09-17T21:29:41.475273+0000","last_unstale":"2024-09-17T21:32:04.699955+0000","last_undegraded":"2024-09-17T21:32:04.699955+0000","last_fullsized":"2024-09-17T21:32:04.699955+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:24:39.329469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001059431,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":6333,"num_read_kb":18110,"num_write":998,"num_write_kb":6659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":116,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.703964+0000","last_change":"2024-09-17T21:31:11.424633+0000","last_active":"2024-09-17T21:32:04.703964+0000","last_peered":"2024-09-17T21:32:04.703964+0000","last_clean":"2024-09-17T21:32:04.703964+0000","last_became_active":"2024-09-17T21:31:11.421737+0000","last_became_peered":"2024-09-17T21:31:11.421737+0000","last_unstale":"2024-09-17T21:32:04.703964+0000","last_undegraded":"2024-09-17T21:32:04.703964+0000","last_fullsized":"2024-09-17T21:32:04.703964+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:07:58.025026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"135'413","reported_seq":2979,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700393+0000","last_change":"2024-09-17T21:31:45.426730+0000","last_active":"2024-09-17T21:32:04.700393+0000","last_peered":"2024-09-17T21:32:04.700393+0000","last_clean":"2024-09-17T21:32:04.700393+0000","last_became_active":"2024-09-17T21:30:28.746690+0000","last_became_peered":"2024-09-17T21:30:28.746690+0000","last_unstale":"2024-09-17T21:32:04.700393+0000","last_undegraded":"2024-09-17T21:32:04.700393+0000","last_fullsized":"2024-09-17T21:32:04.700393+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:53:30.001882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090442500000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3369,"num_read_kb":13331,"num_write":389,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1513,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":241,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700230+0000","last_change":"2024-09-17T21:31:53.979063+0000","last_active":"2024-09-17T21:32:04.700230+0000","last_peered":"2024-09-17T21:32:04.700230+0000","last_clean":"2024-09-17T21:32:04.700230+0000","last_became_active":"2024-09-17T21:31:53.978829+0000","last_became_peered":"2024-09-17T21:31:53.978829+0000","last_unstale":"2024-09-17T21:32:04.700230+0000","last_undegraded":"2024-09-17T21:32:04.700230+0000","last_fullsized":"2024-09-17T21:32:04.700230+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:52.874988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"135'431","reported_seq":3957,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248367+0000","last_change":"2024-09-17T21:31:45.430779+0000","last_active":"2024-09-17T21:32:05.248367+0000","last_peered":"2024-09-17T21:32:05.248367+0000","last_clean":"2024-09-17T21:32:05.248367+0000","last_became_active":"2024-09-17T21:29:41.473964+0000","last_became_peered":"2024-09-17T21:29:41.473964+0000","last_unstale":"2024-09-17T21:32:05.248367+0000","last_undegraded":"2024-09-17T21:32:05.248367+0000","last_fullsized":"2024-09-17T21:32:05.248367+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:45.162957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097061290000000008,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3802,"num_read_kb":64522,"num_write":578,"num_write_kb":39811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":291,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310779+0000","last_change":"2024-09-17T21:30:28.746557+0000","last_active":"2024-09-17T21:32:05.310779+0000","last_peered":"2024-09-17T21:32:05.310779+0000","last_clean":"2024-09-17T21:32:05.310779+0000","last_became_active":"2024-09-17T21:30:28.746261+0000","last_became_peered":"2024-09-17T21:30:28.746261+0000","last_unstale":"2024-09-17T21:32:05.310779+0000","last_undegraded":"2024-09-17T21:32:05.310779+0000","last_fullsized":"2024-09-17T21:32:05.310779+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:42.591689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044115199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"135'333","reported_seq":4194,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125209+0000","last_change":"2024-09-17T21:31:45.418657+0000","last_active":"2024-09-17T21:32:05.125209+0000","last_peered":"2024-09-17T21:32:05.125209+0000","last_clean":"2024-09-17T21:32:05.125209+0000","last_became_active":"2024-09-17T21:31:14.459936+0000","last_became_peered":"2024-09-17T21:31:14.459936+0000","last_unstale":"2024-09-17T21:32:05.125209+0000","last_undegraded":"2024-09-17T21:32:05.125209+0000","last_fullsized":"2024-09-17T21:32:05.125209+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:35.279004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012333649999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":3924,"num_read_kb":72696,"num_write":302,"num_write_kb":18067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":13048770,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":116,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.703960+0000","last_change":"2024-09-17T21:31:11.427558+0000","last_active":"2024-09-17T21:32:04.703960+0000","last_peered":"2024-09-17T21:32:04.703960+0000","last_clean":"2024-09-17T21:32:04.703960+0000","last_became_active":"2024-09-17T21:31:11.422076+0000","last_became_peered":"2024-09-17T21:31:11.422076+0000","last_unstale":"2024-09-17T21:32:04.703960+0000","last_undegraded":"2024-09-17T21:32:04.703960+0000","last_fullsized":"2024-09-17T21:32:04.703960+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:37:14.137712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":373,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310979+0000","last_change":"2024-09-17T21:29:28.554672+0000","last_active":"2024-09-17T21:32:05.310979+0000","last_peered":"2024-09-17T21:32:05.310979+0000","last_clean":"2024-09-17T21:32:05.310979+0000","last_became_active":"2024-09-17T21:29:28.553733+0000","last_became_peered":"2024-09-17T21:29:28.553733+0000","last_unstale":"2024-09-17T21:32:05.310979+0000","last_undegraded":"2024-09-17T21:32:05.310979+0000","last_fullsized":"2024-09-17T21:32:05.310979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_clean_scrub_stamp":"2024-09-17T21:29:26.925225+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:36:42.585477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"135'712","reported_seq":4573,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125288+0000","last_change":"2024-09-17T21:31:45.420733+0000","last_active":"2024-09-17T21:32:05.125288+0000","last_peered":"2024-09-17T21:32:05.125288+0000","last_clean":"2024-09-17T21:32:05.125288+0000","last_became_active":"2024-09-17T21:30:31.753839+0000","last_became_peered":"2024-09-17T21:30:31.753839+0000","last_unstale":"2024-09-17T21:32:05.125288+0000","last_undegraded":"2024-09-17T21:32:05.125288+0000","last_fullsized":"2024-09-17T21:32:05.125288+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:57.420241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030177569999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5120,"num_read_kb":35075,"num_write":882,"num_write_kb":14888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2410412,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"134'412","reported_seq":3583,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699862+0000","last_change":"2024-09-17T21:31:45.424510+0000","last_active":"2024-09-17T21:32:04.699862+0000","last_peered":"2024-09-17T21:32:04.699862+0000","last_clean":"2024-09-17T21:32:04.699862+0000","last_became_active":"2024-09-17T21:29:41.477965+0000","last_became_peered":"2024-09-17T21:29:41.477965+0000","last_unstale":"2024-09-17T21:32:04.699862+0000","last_undegraded":"2024-09-17T21:32:04.699862+0000","last_fullsized":"2024-09-17T21:32:04.699862+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:24:25.503913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062142999999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3781,"num_read_kb":28832,"num_write":427,"num_write_kb":12788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"136'595","reported_seq":4077,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.180946+0000","last_change":"2024-09-17T21:31:45.419838+0000","last_active":"2024-09-17T21:32:05.180946+0000","last_peered":"2024-09-17T21:32:05.180946+0000","last_clean":"2024-09-17T21:32:05.180946+0000","last_became_active":"2024-09-17T21:29:41.473420+0000","last_became_peered":"2024-09-17T21:29:41.473420+0000","last_unstale":"2024-09-17T21:32:05.180946+0000","last_undegraded":"2024-09-17T21:32:05.180946+0000","last_fullsized":"2024-09-17T21:32:05.180946+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:33:22.091302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058409800000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":4583,"num_read_kb":15568,"num_write":610,"num_write_kb":6172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"135'541","reported_seq":7208,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125233+0000","last_change":"2024-09-17T21:31:45.420089+0000","last_active":"2024-09-17T21:32:05.125233+0000","last_peered":"2024-09-17T21:32:05.125233+0000","last_clean":"2024-09-17T21:32:05.125233+0000","last_became_active":"2024-09-17T21:29:41.472354+0000","last_became_peered":"2024-09-17T21:29:41.472354+0000","last_unstale":"2024-09-17T21:32:05.125233+0000","last_undegraded":"2024-09-17T21:32:05.125233+0000","last_fullsized":"2024-09-17T21:32:05.125233+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:30.670868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025676010000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7686,"num_read_kb":30319,"num_write":462,"num_write_kb":3994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"135'661","reported_seq":4876,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.703998+0000","last_change":"2024-09-17T21:31:45.435964+0000","last_active":"2024-09-17T21:32:04.703998+0000","last_peered":"2024-09-17T21:32:04.703998+0000","last_clean":"2024-09-17T21:32:04.703998+0000","last_became_active":"2024-09-17T21:31:15.457822+0000","last_became_peered":"2024-09-17T21:31:15.457822+0000","last_unstale":"2024-09-17T21:32:04.703998+0000","last_undegraded":"2024-09-17T21:32:04.703998+0000","last_fullsized":"2024-09-17T21:32:04.703998+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:45:34.971106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086872700000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7278,"num_read_kb":17574,"num_write":576,"num_write_kb":5172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":2355433,"num_keys_recovered":58,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"135'622","reported_seq":4516,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310922+0000","last_change":"2024-09-17T21:31:45.497982+0000","last_active":"2024-09-17T21:32:05.310922+0000","last_peered":"2024-09-17T21:32:05.310922+0000","last_clean":"2024-09-17T21:32:05.310922+0000","last_became_active":"2024-09-17T21:31:14.457648+0000","last_became_peered":"2024-09-17T21:31:14.457648+0000","last_unstale":"2024-09-17T21:32:05.310922+0000","last_undegraded":"2024-09-17T21:32:05.310922+0000","last_fullsized":"2024-09-17T21:32:05.310922+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:10:28.622328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016040850000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5354,"num_read_kb":12719,"num_write":707,"num_write_kb":4578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2843955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"136'580","reported_seq":4386,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311131+0000","last_change":"2024-09-17T21:31:45.497414+0000","last_active":"2024-09-17T21:32:05.311131+0000","last_peered":"2024-09-17T21:32:05.311131+0000","last_clean":"2024-09-17T21:32:05.311131+0000","last_became_active":"2024-09-17T21:29:41.473736+0000","last_became_peered":"2024-09-17T21:29:41.473736+0000","last_unstale":"2024-09-17T21:32:05.311131+0000","last_undegraded":"2024-09-17T21:32:05.311131+0000","last_fullsized":"2024-09-17T21:32:05.311131+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:12:54.605441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013374140000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5042,"num_read_kb":17595,"num_write":692,"num_write_kb":5927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"135'882","reported_seq":4533,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311122+0000","last_change":"2024-09-17T21:31:45.498420+0000","last_active":"2024-09-17T21:32:05.311122+0000","last_peered":"2024-09-17T21:32:05.311122+0000","last_clean":"2024-09-17T21:32:05.311122+0000","last_became_active":"2024-09-17T21:29:41.473394+0000","last_became_peered":"2024-09-17T21:29:41.473394+0000","last_unstale":"2024-09-17T21:32:05.311122+0000","last_undegraded":"2024-09-17T21:32:05.311122+0000","last_fullsized":"2024-09-17T21:32:05.311122+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:45:31.022697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002038134,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6008,"num_read_kb":8422,"num_write":1180,"num_write_kb":21760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"134'623","reported_seq":2582,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.180906+0000","last_change":"2024-09-17T21:31:45.419674+0000","last_active":"2024-09-17T21:32:05.180906+0000","last_peered":"2024-09-17T21:32:05.180906+0000","last_clean":"2024-09-17T21:32:05.180906+0000","last_became_active":"2024-09-17T21:29:41.471342+0000","last_became_peered":"2024-09-17T21:29:41.471342+0000","last_unstale":"2024-09-17T21:32:05.180906+0000","last_undegraded":"2024-09-17T21:32:05.180906+0000","last_fullsized":"2024-09-17T21:32:05.180906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:23:05.457286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044458799999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3239,"num_read_kb":14667,"num_write":843,"num_write_kb":8817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"136'418","reported_seq":4857,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310899+0000","last_change":"2024-09-17T21:31:45.497400+0000","last_active":"2024-09-17T21:32:05.310899+0000","last_peered":"2024-09-17T21:32:05.310899+0000","last_clean":"2024-09-17T21:32:05.310899+0000","last_became_active":"2024-09-17T21:29:41.473252+0000","last_became_peered":"2024-09-17T21:29:41.473252+0000","last_unstale":"2024-09-17T21:32:05.310899+0000","last_undegraded":"2024-09-17T21:32:05.310899+0000","last_fullsized":"2024-09-17T21:32:05.310899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:36:31.575810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013588999999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4864,"num_read_kb":34676,"num_write":382,"num_write_kb":15080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"136'375","reported_seq":2283,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248376+0000","last_change":"2024-09-17T21:31:45.430386+0000","last_active":"2024-09-17T21:32:05.248376+0000","last_peered":"2024-09-17T21:32:05.248376+0000","last_clean":"2024-09-17T21:32:05.248376+0000","last_became_active":"2024-09-17T21:29:41.477019+0000","last_became_peered":"2024-09-17T21:29:41.477019+0000","last_unstale":"2024-09-17T21:32:05.248376+0000","last_undegraded":"2024-09-17T21:32:05.248376+0000","last_fullsized":"2024-09-17T21:32:05.248376+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:07:16.797282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094423349999999996,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":1879,"num_read_kb":5231,"num_write":351,"num_write_kb":2424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"135'440","reported_seq":4067,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310943+0000","last_change":"2024-09-17T21:31:45.497003+0000","last_active":"2024-09-17T21:32:05.310943+0000","last_peered":"2024-09-17T21:32:05.310943+0000","last_clean":"2024-09-17T21:32:05.310943+0000","last_became_active":"2024-09-17T21:29:41.473182+0000","last_became_peered":"2024-09-17T21:29:41.473182+0000","last_unstale":"2024-09-17T21:32:05.310943+0000","last_undegraded":"2024-09-17T21:32:05.310943+0000","last_fullsized":"2024-09-17T21:32:05.310943+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:55:27.817705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092929600000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3686,"num_read_kb":22454,"num_write":375,"num_write_kb":9447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":656,"num_object_clones":0,"num_object_copies":1312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":656,"num_whiteouts":652,"num_read":159356,"num_read_kb":893959,"num_write":21643,"num_write_kb":452621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1205,"num_bytes_recovered":117790825,"num_keys_recovered":327,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10476,"internal_metadata":0},"log_size":19097,"ondisk_log_size":19097,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509476,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55256,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316584,"statfs":{"total":96636764160,"available":96580182016,"internally_reserved":0,"allocated":1900544,"data_stored":1469370,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28808,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343032,"statfs":{"total":96636764160,"available":96607264768,"internally_reserved":0,"allocated":1818624,"data_stored":867376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41316,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330524,"statfs":{"total":96636764160,"available":96594456576,"internally_reserved":0,"allocated":1847296,"data_stored":879021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509476,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59368,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312472,"statfs":{"total":96636764160,"available":96575971328,"internally_reserved":0,"allocated":1847296,"data_stored":879021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55192,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316648,"statfs":{"total":96636764160,"available":96580247552,"internally_reserved":0,"allocated":1830912,"data_stored":873189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509477,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41316,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330524,"statfs":{"total":96636764160,"available":96594456576,"internally_reserved":0,"allocated":1843200,"data_stored":879002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":63,"seq":270582939670,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42916,"kb_used_data":1808,"kb_used_omap":20,"kb_used_meta":41067,"kb_avail":94328924,"statfs":{"total":96636764160,"available":96592818176,"internally_reserved":0,"allocated":1851392,"data_stored":879040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20925,"internal_metadata":42053187},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55228,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316612,"statfs":{"total":96636764160,"available":96580210688,"internally_reserved":0,"allocated":1875968,"data_stored":1457744,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10476,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:32:09.317 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:32:09.605 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:32:09.605 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-17T21:32:09.617 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-17T21:32:08.887764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":660,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":660,"num_whiteouts":652,"num_read":159422,"num_read_kb":894014,"num_write":21746,"num_write_kb":454251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1208,"num_bytes_recovered":117790844,"num_keys_recovered":327,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19172,"ondisk_log_size":19172,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379400,"kb_used_data":14468,"kb_used_omap":31,"kb_used_meta":364704,"kb_avail":754595320,"statfs":{"total":773094113280,"available":772705607680,"internally_reserved":0,"allocated":14815232,"data_stored":8183763,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":32080,"internal_metadata":373457584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.202366"},"pg_stats":[{"pgid":"3.1f","version":"136'809","reported_seq":5134,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248455+0000","last_change":"2024-09-17T21:31:45.430953+0000","last_active":"2024-09-17T21:32:05.248455+0000","last_peered":"2024-09-17T21:32:05.248455+0000","last_clean":"2024-09-17T21:32:05.248455+0000","last_became_active":"2024-09-17T21:31:15.457506+0000","last_became_peered":"2024-09-17T21:31:15.457506+0000","last_unstale":"2024-09-17T21:32:05.248455+0000","last_undegraded":"2024-09-17T21:32:05.248455+0000","last_fullsized":"2024-09-17T21:32:05.248455+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:08.923954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098455079999999993,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6585,"num_read_kb":63347,"num_write":1151,"num_write_kb":32903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":27321072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"136'562","reported_seq":4234,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125356+0000","last_change":"2024-09-17T21:31:45.418470+0000","last_active":"2024-09-17T21:32:05.125356+0000","last_peered":"2024-09-17T21:32:05.125356+0000","last_clean":"2024-09-17T21:32:05.125356+0000","last_became_active":"2024-09-17T21:29:41.472161+0000","last_became_peered":"2024-09-17T21:29:41.472161+0000","last_unstale":"2024-09-17T21:32:05.125356+0000","last_undegraded":"2024-09-17T21:32:05.125356+0000","last_fullsized":"2024-09-17T21:32:05.125356+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:04:47.812939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079279,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3917,"num_read_kb":31515,"num_write":544,"num_write_kb":11792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"135'768","reported_seq":5402,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704245+0000","last_change":"2024-09-17T21:31:45.436342+0000","last_active":"2024-09-17T21:32:04.704245+0000","last_peered":"2024-09-17T21:32:04.704245+0000","last_clean":"2024-09-17T21:32:04.704245+0000","last_became_active":"2024-09-17T21:31:14.449672+0000","last_became_peered":"2024-09-17T21:31:14.449672+0000","last_unstale":"2024-09-17T21:32:04.704245+0000","last_undegraded":"2024-09-17T21:32:04.704245+0000","last_fullsized":"2024-09-17T21:32:04.704245+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:48:24.256603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010859089999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6612,"num_read_kb":51732,"num_write":893,"num_write_kb":18876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":15268754,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"136'433","reported_seq":2685,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704042+0000","last_change":"2024-09-17T21:31:45.435673+0000","last_active":"2024-09-17T21:32:04.704042+0000","last_peered":"2024-09-17T21:32:04.704042+0000","last_clean":"2024-09-17T21:32:04.704042+0000","last_became_active":"2024-09-17T21:31:14.451037+0000","last_became_peered":"2024-09-17T21:31:14.451037+0000","last_unstale":"2024-09-17T21:32:04.704042+0000","last_undegraded":"2024-09-17T21:32:04.704042+0000","last_fullsized":"2024-09-17T21:32:04.704042+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:41:20.330623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068460700000000005,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":2712,"num_read_kb":12910,"num_write":456,"num_write_kb":9120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":604511,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"136'589","reported_seq":8494,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248591+0000","last_change":"2024-09-17T21:31:45.430352+0000","last_active":"2024-09-17T21:32:05.248591+0000","last_peered":"2024-09-17T21:32:05.248591+0000","last_clean":"2024-09-17T21:32:05.248591+0000","last_became_active":"2024-09-17T21:29:41.476041+0000","last_became_peered":"2024-09-17T21:29:41.476041+0000","last_unstale":"2024-09-17T21:32:05.248591+0000","last_undegraded":"2024-09-17T21:32:05.248591+0000","last_fullsized":"2024-09-17T21:32:05.248591+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:45:15.271044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0092951189999999993,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8770,"num_read_kb":53504,"num_write":580,"num_write_kb":24876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"136'630","reported_seq":6573,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699780+0000","last_change":"2024-09-17T21:31:45.425285+0000","last_active":"2024-09-17T21:32:04.699780+0000","last_peered":"2024-09-17T21:32:04.699780+0000","last_clean":"2024-09-17T21:32:04.699780+0000","last_became_active":"2024-09-17T21:30:28.747031+0000","last_became_peered":"2024-09-17T21:30:28.747031+0000","last_unstale":"2024-09-17T21:32:04.699780+0000","last_undegraded":"2024-09-17T21:32:04.699780+0000","last_fullsized":"2024-09-17T21:32:04.699780+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:26.185752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00134892,"stat_sum":{"num_bytes":19,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":7065,"num_read_kb":21661,"num_write":665,"num_write_kb":6726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":235969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"135'424","reported_seq":4765,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700031+0000","last_change":"2024-09-17T21:31:45.427269+0000","last_active":"2024-09-17T21:32:04.700031+0000","last_peered":"2024-09-17T21:32:04.700031+0000","last_clean":"2024-09-17T21:32:04.700031+0000","last_became_active":"2024-09-17T21:30:28.745634+0000","last_became_peered":"2024-09-17T21:30:28.745634+0000","last_unstale":"2024-09-17T21:32:04.700031+0000","last_undegraded":"2024-09-17T21:32:04.700031+0000","last_fullsized":"2024-09-17T21:32:04.700031+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:58.579935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001320393,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4551,"num_read_kb":56994,"num_write":426,"num_write_kb":45225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":36017677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"136'512","reported_seq":4537,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125260+0000","last_change":"2024-09-17T21:31:45.420094+0000","last_active":"2024-09-17T21:32:05.125260+0000","last_peered":"2024-09-17T21:32:05.125260+0000","last_clean":"2024-09-17T21:32:05.125260+0000","last_became_active":"2024-09-17T21:29:41.472468+0000","last_became_peered":"2024-09-17T21:29:41.472468+0000","last_unstale":"2024-09-17T21:32:05.125260+0000","last_undegraded":"2024-09-17T21:32:05.125260+0000","last_fullsized":"2024-09-17T21:32:05.125260+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:12:23.877166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002623551,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4791,"num_read_kb":27686,"num_write":500,"num_write_kb":13101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'814","reported_seq":3250,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248533+0000","last_change":"2024-09-17T21:31:45.431351+0000","last_active":"2024-09-17T21:32:05.248533+0000","last_peered":"2024-09-17T21:32:05.248533+0000","last_clean":"2024-09-17T21:32:05.248533+0000","last_became_active":"2024-09-17T21:31:15.456240+0000","last_became_peered":"2024-09-17T21:31:15.456240+0000","last_unstale":"2024-09-17T21:32:05.248533+0000","last_undegraded":"2024-09-17T21:32:05.248533+0000","last_fullsized":"2024-09-17T21:32:05.248533+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:31:41.549775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010220406,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4952,"num_read_kb":22844,"num_write":955,"num_write_kb":30092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":9394579,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"136'1106","reported_seq":3090,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704029+0000","last_change":"2024-09-17T21:31:45.435544+0000","last_active":"2024-09-17T21:32:04.704029+0000","last_peered":"2024-09-17T21:32:04.704029+0000","last_clean":"2024-09-17T21:32:04.704029+0000","last_became_active":"2024-09-17T21:31:15.456612+0000","last_became_peered":"2024-09-17T21:31:15.456612+0000","last_unstale":"2024-09-17T21:32:04.704029+0000","last_undegraded":"2024-09-17T21:32:04.704029+0000","last_fullsized":"2024-09-17T21:32:04.704029+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":1106,"log_dups_size":0,"ondisk_log_size":1106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:12:11.830676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051663100000000003,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":5737,"num_read_kb":20891,"num_write":1670,"num_write_kb":12347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3063082,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"135'512","reported_seq":3188,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311120+0000","last_change":"2024-09-17T21:31:45.497882+0000","last_active":"2024-09-17T21:32:05.311120+0000","last_peered":"2024-09-17T21:32:05.311120+0000","last_clean":"2024-09-17T21:32:05.311120+0000","last_became_active":"2024-09-17T21:29:41.478346+0000","last_became_peered":"2024-09-17T21:29:41.478346+0000","last_unstale":"2024-09-17T21:32:05.311120+0000","last_undegraded":"2024-09-17T21:32:05.311120+0000","last_fullsized":"2024-09-17T21:32:05.311120+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:58.424829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017578019999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3628,"num_read_kb":9299,"num_write":426,"num_write_kb":8355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"135'297","reported_seq":2884,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699913+0000","last_change":"2024-09-17T21:31:45.424549+0000","last_active":"2024-09-17T21:32:04.699913+0000","last_peered":"2024-09-17T21:32:04.699913+0000","last_clean":"2024-09-17T21:32:04.699913+0000","last_became_active":"2024-09-17T21:29:41.477611+0000","last_became_peered":"2024-09-17T21:29:41.477611+0000","last_unstale":"2024-09-17T21:32:04.699913+0000","last_undegraded":"2024-09-17T21:32:04.699913+0000","last_fullsized":"2024-09-17T21:32:04.699913+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:59:07.448045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064521399999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2154,"num_read_kb":14079,"num_write":352,"num_write_kb":12746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":280,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.180954+0000","last_change":"2024-09-17T21:29:35.002656+0000","last_active":"2024-09-17T21:32:05.180954+0000","last_peered":"2024-09-17T21:32:05.180954+0000","last_clean":"2024-09-17T21:32:05.180954+0000","last_became_active":"2024-09-17T21:29:32.971677+0000","last_became_peered":"2024-09-17T21:29:32.971677+0000","last_unstale":"2024-09-17T21:32:05.180954+0000","last_undegraded":"2024-09-17T21:32:05.180954+0000","last_fullsized":"2024-09-17T21:32:05.180954+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:53.781657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041121300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"136'848","reported_seq":4463,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248518+0000","last_change":"2024-09-17T21:31:45.431468+0000","last_active":"2024-09-17T21:32:05.248518+0000","last_peered":"2024-09-17T21:32:05.248518+0000","last_clean":"2024-09-17T21:32:05.248518+0000","last_became_active":"2024-09-17T21:30:31.755039+0000","last_became_peered":"2024-09-17T21:30:31.755039+0000","last_unstale":"2024-09-17T21:32:05.248518+0000","last_undegraded":"2024-09-17T21:32:05.248518+0000","last_fullsized":"2024-09-17T21:32:05.248518+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.621804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01026987,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6950,"num_read_kb":18293,"num_write":963,"num_write_kb":8597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":448871,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":213,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700243+0000","last_change":"2024-09-17T21:30:28.745759+0000","last_active":"2024-09-17T21:32:04.700243+0000","last_peered":"2024-09-17T21:32:04.700243+0000","last_clean":"2024-09-17T21:32:04.700243+0000","last_became_active":"2024-09-17T21:30:28.744742+0000","last_became_peered":"2024-09-17T21:30:28.744742+0000","last_unstale":"2024-09-17T21:32:04.700243+0000","last_undegraded":"2024-09-17T21:32:04.700243+0000","last_fullsized":"2024-09-17T21:32:04.700243+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:37:02.014566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"134'592","reported_seq":2794,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125373+0000","last_change":"2024-09-17T21:31:45.420595+0000","last_active":"2024-09-17T21:32:05.125373+0000","last_peered":"2024-09-17T21:32:05.125373+0000","last_clean":"2024-09-17T21:32:05.125373+0000","last_became_active":"2024-09-17T21:29:41.472095+0000","last_became_peered":"2024-09-17T21:29:41.472095+0000","last_unstale":"2024-09-17T21:32:05.125373+0000","last_undegraded":"2024-09-17T21:32:05.125373+0000","last_fullsized":"2024-09-17T21:32:05.125373+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:07:43.653815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028983989999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":3726,"num_read_kb":13082,"num_write":700,"num_write_kb":8995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":282,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311127+0000","last_change":"2024-09-17T21:29:35.078854+0000","last_active":"2024-09-17T21:32:05.311127+0000","last_peered":"2024-09-17T21:32:05.311127+0000","last_clean":"2024-09-17T21:32:05.311127+0000","last_became_active":"2024-09-17T21:29:33.071665+0000","last_became_peered":"2024-09-17T21:29:33.071665+0000","last_unstale":"2024-09-17T21:32:05.311127+0000","last_undegraded":"2024-09-17T21:32:05.311127+0000","last_fullsized":"2024-09-17T21:32:05.311127+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.415687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047969899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"134'805","reported_seq":4875,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700392+0000","last_change":"2024-09-17T21:31:45.426864+0000","last_active":"2024-09-17T21:32:04.700392+0000","last_peered":"2024-09-17T21:32:04.700392+0000","last_clean":"2024-09-17T21:32:04.700392+0000","last_became_active":"2024-09-17T21:30:31.752750+0000","last_became_peered":"2024-09-17T21:30:31.752750+0000","last_unstale":"2024-09-17T21:32:04.700392+0000","last_undegraded":"2024-09-17T21:32:04.700392+0000","last_fullsized":"2024-09-17T21:32:04.700392+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:27:53.025416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094011400000000003,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6270,"num_read_kb":32698,"num_write":1050,"num_write_kb":17424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3231045,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":213,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700238+0000","last_change":"2024-09-17T21:30:28.748319+0000","last_active":"2024-09-17T21:32:04.700238+0000","last_peered":"2024-09-17T21:32:04.700238+0000","last_clean":"2024-09-17T21:32:04.700238+0000","last_became_active":"2024-09-17T21:30:28.747728+0000","last_became_peered":"2024-09-17T21:30:28.747728+0000","last_unstale":"2024-09-17T21:32:04.700238+0000","last_undegraded":"2024-09-17T21:32:04.700238+0000","last_fullsized":"2024-09-17T21:32:04.700238+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:58:18.954005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"135'502","reported_seq":4854,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704234+0000","last_change":"2024-09-17T21:31:45.436069+0000","last_active":"2024-09-17T21:32:04.704234+0000","last_peered":"2024-09-17T21:32:04.704234+0000","last_clean":"2024-09-17T21:32:04.704234+0000","last_became_active":"2024-09-17T21:31:14.455957+0000","last_became_peered":"2024-09-17T21:31:14.455957+0000","last_unstale":"2024-09-17T21:32:04.704234+0000","last_undegraded":"2024-09-17T21:32:04.704234+0000","last_fullsized":"2024-09-17T21:32:04.704234+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:43:55.461066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087537200000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4988,"num_read_kb":31633,"num_write":558,"num_write_kb":9789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":1545182,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"136'856","reported_seq":5026,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699955+0000","last_change":"2024-09-17T21:31:45.424972+0000","last_active":"2024-09-17T21:32:04.699955+0000","last_peered":"2024-09-17T21:32:04.699955+0000","last_clean":"2024-09-17T21:32:04.699955+0000","last_became_active":"2024-09-17T21:29:41.475273+0000","last_became_peered":"2024-09-17T21:29:41.475273+0000","last_unstale":"2024-09-17T21:32:04.699955+0000","last_undegraded":"2024-09-17T21:32:04.699955+0000","last_fullsized":"2024-09-17T21:32:04.699955+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:24:39.329469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001059431,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":6333,"num_read_kb":18110,"num_write":998,"num_write_kb":6659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":116,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.703964+0000","last_change":"2024-09-17T21:31:11.424633+0000","last_active":"2024-09-17T21:32:04.703964+0000","last_peered":"2024-09-17T21:32:04.703964+0000","last_clean":"2024-09-17T21:32:04.703964+0000","last_became_active":"2024-09-17T21:31:11.421737+0000","last_became_peered":"2024-09-17T21:31:11.421737+0000","last_unstale":"2024-09-17T21:32:04.703964+0000","last_undegraded":"2024-09-17T21:32:04.703964+0000","last_fullsized":"2024-09-17T21:32:04.703964+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:07:58.025026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"135'413","reported_seq":2979,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700393+0000","last_change":"2024-09-17T21:31:45.426730+0000","last_active":"2024-09-17T21:32:04.700393+0000","last_peered":"2024-09-17T21:32:04.700393+0000","last_clean":"2024-09-17T21:32:04.700393+0000","last_became_active":"2024-09-17T21:30:28.746690+0000","last_became_peered":"2024-09-17T21:30:28.746690+0000","last_unstale":"2024-09-17T21:32:04.700393+0000","last_undegraded":"2024-09-17T21:32:04.700393+0000","last_fullsized":"2024-09-17T21:32:04.700393+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:53:30.001882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090442500000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3369,"num_read_kb":13331,"num_write":389,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1513,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":241,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700230+0000","last_change":"2024-09-17T21:31:53.979063+0000","last_active":"2024-09-17T21:32:04.700230+0000","last_peered":"2024-09-17T21:32:04.700230+0000","last_clean":"2024-09-17T21:32:04.700230+0000","last_became_active":"2024-09-17T21:31:53.978829+0000","last_became_peered":"2024-09-17T21:31:53.978829+0000","last_unstale":"2024-09-17T21:32:04.700230+0000","last_undegraded":"2024-09-17T21:32:04.700230+0000","last_fullsized":"2024-09-17T21:32:04.700230+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:52.874988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"135'431","reported_seq":3957,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248367+0000","last_change":"2024-09-17T21:31:45.430779+0000","last_active":"2024-09-17T21:32:05.248367+0000","last_peered":"2024-09-17T21:32:05.248367+0000","last_clean":"2024-09-17T21:32:05.248367+0000","last_became_active":"2024-09-17T21:29:41.473964+0000","last_became_peered":"2024-09-17T21:29:41.473964+0000","last_unstale":"2024-09-17T21:32:05.248367+0000","last_undegraded":"2024-09-17T21:32:05.248367+0000","last_fullsized":"2024-09-17T21:32:05.248367+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:45.162957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097061290000000008,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3802,"num_read_kb":64522,"num_write":578,"num_write_kb":39811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":291,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310779+0000","last_change":"2024-09-17T21:30:28.746557+0000","last_active":"2024-09-17T21:32:05.310779+0000","last_peered":"2024-09-17T21:32:05.310779+0000","last_clean":"2024-09-17T21:32:05.310779+0000","last_became_active":"2024-09-17T21:30:28.746261+0000","last_became_peered":"2024-09-17T21:30:28.746261+0000","last_unstale":"2024-09-17T21:32:05.310779+0000","last_undegraded":"2024-09-17T21:32:05.310779+0000","last_fullsized":"2024-09-17T21:32:05.310779+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:42.591689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044115199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"135'333","reported_seq":4194,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125209+0000","last_change":"2024-09-17T21:31:45.418657+0000","last_active":"2024-09-17T21:32:05.125209+0000","last_peered":"2024-09-17T21:32:05.125209+0000","last_clean":"2024-09-17T21:32:05.125209+0000","last_became_active":"2024-09-17T21:31:14.459936+0000","last_became_peered":"2024-09-17T21:31:14.459936+0000","last_unstale":"2024-09-17T21:32:05.125209+0000","last_undegraded":"2024-09-17T21:32:05.125209+0000","last_fullsized":"2024-09-17T21:32:05.125209+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:35.279004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012333649999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":3924,"num_read_kb":72696,"num_write":302,"num_write_kb":18067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":13048770,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":116,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.703960+0000","last_change":"2024-09-17T21:31:11.427558+0000","last_active":"2024-09-17T21:32:04.703960+0000","last_peered":"2024-09-17T21:32:04.703960+0000","last_clean":"2024-09-17T21:32:04.703960+0000","last_became_active":"2024-09-17T21:31:11.422076+0000","last_became_peered":"2024-09-17T21:31:11.422076+0000","last_unstale":"2024-09-17T21:32:04.703960+0000","last_undegraded":"2024-09-17T21:32:04.703960+0000","last_fullsized":"2024-09-17T21:32:04.703960+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:37:14.137712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":373,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310979+0000","last_change":"2024-09-17T21:29:28.554672+0000","last_active":"2024-09-17T21:32:05.310979+0000","last_peered":"2024-09-17T21:32:05.310979+0000","last_clean":"2024-09-17T21:32:05.310979+0000","last_became_active":"2024-09-17T21:29:28.553733+0000","last_became_peered":"2024-09-17T21:29:28.553733+0000","last_unstale":"2024-09-17T21:32:05.310979+0000","last_undegraded":"2024-09-17T21:32:05.310979+0000","last_fullsized":"2024-09-17T21:32:05.310979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_clean_scrub_stamp":"2024-09-17T21:29:26.925225+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:36:42.585477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"135'712","reported_seq":4573,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125288+0000","last_change":"2024-09-17T21:31:45.420733+0000","last_active":"2024-09-17T21:32:05.125288+0000","last_peered":"2024-09-17T21:32:05.125288+0000","last_clean":"2024-09-17T21:32:05.125288+0000","last_became_active":"2024-09-17T21:30:31.753839+0000","last_became_peered":"2024-09-17T21:30:31.753839+0000","last_unstale":"2024-09-17T21:32:05.125288+0000","last_undegraded":"2024-09-17T21:32:05.125288+0000","last_fullsized":"2024-09-17T21:32:05.125288+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:57.420241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030177569999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5120,"num_read_kb":35075,"num_write":882,"num_write_kb":14888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2410412,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"134'412","reported_seq":3583,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699862+0000","last_change":"2024-09-17T21:31:45.424510+0000","last_active":"2024-09-17T21:32:04.699862+0000","last_peered":"2024-09-17T21:32:04.699862+0000","last_clean":"2024-09-17T21:32:04.699862+0000","last_became_active":"2024-09-17T21:29:41.477965+0000","last_became_peered":"2024-09-17T21:29:41.477965+0000","last_unstale":"2024-09-17T21:32:04.699862+0000","last_undegraded":"2024-09-17T21:32:04.699862+0000","last_fullsized":"2024-09-17T21:32:04.699862+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:24:25.503913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062142999999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3781,"num_read_kb":28832,"num_write":427,"num_write_kb":12788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"136'595","reported_seq":4077,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.180946+0000","last_change":"2024-09-17T21:31:45.419838+0000","last_active":"2024-09-17T21:32:05.180946+0000","last_peered":"2024-09-17T21:32:05.180946+0000","last_clean":"2024-09-17T21:32:05.180946+0000","last_became_active":"2024-09-17T21:29:41.473420+0000","last_became_peered":"2024-09-17T21:29:41.473420+0000","last_unstale":"2024-09-17T21:32:05.180946+0000","last_undegraded":"2024-09-17T21:32:05.180946+0000","last_fullsized":"2024-09-17T21:32:05.180946+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:33:22.091302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058409800000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":4583,"num_read_kb":15568,"num_write":610,"num_write_kb":6172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"135'541","reported_seq":7208,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125233+0000","last_change":"2024-09-17T21:31:45.420089+0000","last_active":"2024-09-17T21:32:05.125233+0000","last_peered":"2024-09-17T21:32:05.125233+0000","last_clean":"2024-09-17T21:32:05.125233+0000","last_became_active":"2024-09-17T21:29:41.472354+0000","last_became_peered":"2024-09-17T21:29:41.472354+0000","last_unstale":"2024-09-17T21:32:05.125233+0000","last_undegraded":"2024-09-17T21:32:05.125233+0000","last_fullsized":"2024-09-17T21:32:05.125233+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:30.670868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025676010000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7686,"num_read_kb":30319,"num_write":462,"num_write_kb":3994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"135'661","reported_seq":4876,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.703998+0000","last_change":"2024-09-17T21:31:45.435964+0000","last_active":"2024-09-17T21:32:04.703998+0000","last_peered":"2024-09-17T21:32:04.703998+0000","last_clean":"2024-09-17T21:32:04.703998+0000","last_became_active":"2024-09-17T21:31:15.457822+0000","last_became_peered":"2024-09-17T21:31:15.457822+0000","last_unstale":"2024-09-17T21:32:04.703998+0000","last_undegraded":"2024-09-17T21:32:04.703998+0000","last_fullsized":"2024-09-17T21:32:04.703998+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:45:34.971106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086872700000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7278,"num_read_kb":17574,"num_write":576,"num_write_kb":5172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":2355433,"num_keys_recovered":58,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"135'622","reported_seq":4516,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310922+0000","last_change":"2024-09-17T21:31:45.497982+0000","last_active":"2024-09-17T21:32:05.310922+0000","last_peered":"2024-09-17T21:32:05.310922+0000","last_clean":"2024-09-17T21:32:05.310922+0000","last_became_active":"2024-09-17T21:31:14.457648+0000","last_became_peered":"2024-09-17T21:31:14.457648+0000","last_unstale":"2024-09-17T21:32:05.310922+0000","last_undegraded":"2024-09-17T21:32:05.310922+0000","last_fullsized":"2024-09-17T21:32:05.310922+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:10:28.622328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016040850000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5354,"num_read_kb":12719,"num_write":707,"num_write_kb":4578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2843955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"136'580","reported_seq":4386,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311131+0000","last_change":"2024-09-17T21:31:45.497414+0000","last_active":"2024-09-17T21:32:05.311131+0000","last_peered":"2024-09-17T21:32:05.311131+0000","last_clean":"2024-09-17T21:32:05.311131+0000","last_became_active":"2024-09-17T21:29:41.473736+0000","last_became_peered":"2024-09-17T21:29:41.473736+0000","last_unstale":"2024-09-17T21:32:05.311131+0000","last_undegraded":"2024-09-17T21:32:05.311131+0000","last_fullsized":"2024-09-17T21:32:05.311131+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:12:54.605441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013374140000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5042,"num_read_kb":17595,"num_write":692,"num_write_kb":5927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"135'882","reported_seq":4533,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311122+0000","last_change":"2024-09-17T21:31:45.498420+0000","last_active":"2024-09-17T21:32:05.311122+0000","last_peered":"2024-09-17T21:32:05.311122+0000","last_clean":"2024-09-17T21:32:05.311122+0000","last_became_active":"2024-09-17T21:29:41.473394+0000","last_became_peered":"2024-09-17T21:29:41.473394+0000","last_unstale":"2024-09-17T21:32:05.311122+0000","last_undegraded":"2024-09-17T21:32:05.311122+0000","last_fullsized":"2024-09-17T21:32:05.311122+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:45:31.022697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002038134,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6008,"num_read_kb":8422,"num_write":1180,"num_write_kb":21760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"134'623","reported_seq":2582,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.180906+0000","last_change":"2024-09-17T21:31:45.419674+0000","last_active":"2024-09-17T21:32:05.180906+0000","last_peered":"2024-09-17T21:32:05.180906+0000","last_clean":"2024-09-17T21:32:05.180906+0000","last_became_active":"2024-09-17T21:29:41.471342+0000","last_became_peered":"2024-09-17T21:29:41.471342+0000","last_unstale":"2024-09-17T21:32:05.180906+0000","last_undegraded":"2024-09-17T21:32:05.180906+0000","last_fullsized":"2024-09-17T21:32:05.180906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:23:05.457286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044458799999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3239,"num_read_kb":14667,"num_write":843,"num_write_kb":8817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"136'418","reported_seq":4857,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310899+0000","last_change":"2024-09-17T21:31:45.497400+0000","last_active":"2024-09-17T21:32:05.310899+0000","last_peered":"2024-09-17T21:32:05.310899+0000","last_clean":"2024-09-17T21:32:05.310899+0000","last_became_active":"2024-09-17T21:29:41.473252+0000","last_became_peered":"2024-09-17T21:29:41.473252+0000","last_unstale":"2024-09-17T21:32:05.310899+0000","last_undegraded":"2024-09-17T21:32:05.310899+0000","last_fullsized":"2024-09-17T21:32:05.310899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:36:31.575810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013588999999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4864,"num_read_kb":34676,"num_write":382,"num_write_kb":15080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"136'375","reported_seq":2283,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248376+0000","last_change":"2024-09-17T21:31:45.430386+0000","last_active":"2024-09-17T21:32:05.248376+0000","last_peered":"2024-09-17T21:32:05.248376+0000","last_clean":"2024-09-17T21:32:05.248376+0000","last_became_active":"2024-09-17T21:29:41.477019+0000","last_became_peered":"2024-09-17T21:29:41.477019+0000","last_unstale":"2024-09-17T21:32:05.248376+0000","last_undegraded":"2024-09-17T21:32:05.248376+0000","last_fullsized":"2024-09-17T21:32:05.248376+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:07:16.797282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094423349999999996,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":1879,"num_read_kb":5231,"num_write":351,"num_write_kb":2424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"135'440","reported_seq":4067,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310943+0000","last_change":"2024-09-17T21:31:45.497003+0000","last_active":"2024-09-17T21:32:05.310943+0000","last_peered":"2024-09-17T21:32:05.310943+0000","last_clean":"2024-09-17T21:32:05.310943+0000","last_became_active":"2024-09-17T21:29:41.473182+0000","last_became_peered":"2024-09-17T21:29:41.473182+0000","last_unstale":"2024-09-17T21:32:05.310943+0000","last_undegraded":"2024-09-17T21:32:05.310943+0000","last_fullsized":"2024-09-17T21:32:05.310943+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:55:27.817705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092929600000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3686,"num_read_kb":22454,"num_write":375,"num_write_kb":9447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":656,"num_object_clones":0,"num_object_copies":1312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":656,"num_whiteouts":652,"num_read":159356,"num_read_kb":893959,"num_write":21643,"num_write_kb":452621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1205,"num_bytes_recovered":117790825,"num_keys_recovered":327,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10476,"internal_metadata":0},"log_size":19097,"ondisk_log_size":19097,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509476,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55256,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316584,"statfs":{"total":96636764160,"available":96580182016,"internally_reserved":0,"allocated":1900544,"data_stored":1469370,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28808,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343032,"statfs":{"total":96636764160,"available":96607264768,"internally_reserved":0,"allocated":1818624,"data_stored":867376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41316,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330524,"statfs":{"total":96636764160,"available":96594456576,"internally_reserved":0,"allocated":1847296,"data_stored":879021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509476,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59368,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312472,"statfs":{"total":96636764160,"available":96575971328,"internally_reserved":0,"allocated":1847296,"data_stored":879021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55192,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316648,"statfs":{"total":96636764160,"available":96580247552,"internally_reserved":0,"allocated":1830912,"data_stored":873189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509477,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41316,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330524,"statfs":{"total":96636764160,"available":96594456576,"internally_reserved":0,"allocated":1843200,"data_stored":879002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":63,"seq":270582939670,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42916,"kb_used_data":1808,"kb_used_omap":20,"kb_used_meta":41067,"kb_avail":94328924,"statfs":{"total":96636764160,"available":96592818176,"internally_reserved":0,"allocated":1851392,"data_stored":879040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20925,"internal_metadata":42053187},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55228,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316612,"statfs":{"total":96636764160,"available":96580210688,"internally_reserved":0,"allocated":1875968,"data_stored":1457744,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10476,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:32:09.619 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T21:32:09.619 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-17T21:32:09.923 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-17T21:32:09.932 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T21:32:09.933 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:32:10.216 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:32:10.217 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-17T21:32:10.229 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-17T21:32:08.887764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":660,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":660,"num_whiteouts":652,"num_read":159422,"num_read_kb":894014,"num_write":21746,"num_write_kb":454251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1208,"num_bytes_recovered":117790844,"num_keys_recovered":327,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19172,"ondisk_log_size":19172,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379400,"kb_used_data":14468,"kb_used_omap":31,"kb_used_meta":364704,"kb_avail":754595320,"statfs":{"total":773094113280,"available":772705607680,"internally_reserved":0,"allocated":14815232,"data_stored":8183763,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":32080,"internal_metadata":373457584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.202366"},"pg_stats":[{"pgid":"3.1f","version":"136'809","reported_seq":5134,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248455+0000","last_change":"2024-09-17T21:31:45.430953+0000","last_active":"2024-09-17T21:32:05.248455+0000","last_peered":"2024-09-17T21:32:05.248455+0000","last_clean":"2024-09-17T21:32:05.248455+0000","last_became_active":"2024-09-17T21:31:15.457506+0000","last_became_peered":"2024-09-17T21:31:15.457506+0000","last_unstale":"2024-09-17T21:32:05.248455+0000","last_undegraded":"2024-09-17T21:32:05.248455+0000","last_fullsized":"2024-09-17T21:32:05.248455+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:08.923954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098455079999999993,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6585,"num_read_kb":63347,"num_write":1151,"num_write_kb":32903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":27321072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"136'562","reported_seq":4234,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125356+0000","last_change":"2024-09-17T21:31:45.418470+0000","last_active":"2024-09-17T21:32:05.125356+0000","last_peered":"2024-09-17T21:32:05.125356+0000","last_clean":"2024-09-17T21:32:05.125356+0000","last_became_active":"2024-09-17T21:29:41.472161+0000","last_became_peered":"2024-09-17T21:29:41.472161+0000","last_unstale":"2024-09-17T21:32:05.125356+0000","last_undegraded":"2024-09-17T21:32:05.125356+0000","last_fullsized":"2024-09-17T21:32:05.125356+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:04:47.812939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079279,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3917,"num_read_kb":31515,"num_write":544,"num_write_kb":11792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"135'768","reported_seq":5402,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704245+0000","last_change":"2024-09-17T21:31:45.436342+0000","last_active":"2024-09-17T21:32:04.704245+0000","last_peered":"2024-09-17T21:32:04.704245+0000","last_clean":"2024-09-17T21:32:04.704245+0000","last_became_active":"2024-09-17T21:31:14.449672+0000","last_became_peered":"2024-09-17T21:31:14.449672+0000","last_unstale":"2024-09-17T21:32:04.704245+0000","last_undegraded":"2024-09-17T21:32:04.704245+0000","last_fullsized":"2024-09-17T21:32:04.704245+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:48:24.256603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010859089999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6612,"num_read_kb":51732,"num_write":893,"num_write_kb":18876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":15268754,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"136'433","reported_seq":2685,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704042+0000","last_change":"2024-09-17T21:31:45.435673+0000","last_active":"2024-09-17T21:32:04.704042+0000","last_peered":"2024-09-17T21:32:04.704042+0000","last_clean":"2024-09-17T21:32:04.704042+0000","last_became_active":"2024-09-17T21:31:14.451037+0000","last_became_peered":"2024-09-17T21:31:14.451037+0000","last_unstale":"2024-09-17T21:32:04.704042+0000","last_undegraded":"2024-09-17T21:32:04.704042+0000","last_fullsized":"2024-09-17T21:32:04.704042+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:41:20.330623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068460700000000005,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":2712,"num_read_kb":12910,"num_write":456,"num_write_kb":9120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":604511,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"136'589","reported_seq":8494,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248591+0000","last_change":"2024-09-17T21:31:45.430352+0000","last_active":"2024-09-17T21:32:05.248591+0000","last_peered":"2024-09-17T21:32:05.248591+0000","last_clean":"2024-09-17T21:32:05.248591+0000","last_became_active":"2024-09-17T21:29:41.476041+0000","last_became_peered":"2024-09-17T21:29:41.476041+0000","last_unstale":"2024-09-17T21:32:05.248591+0000","last_undegraded":"2024-09-17T21:32:05.248591+0000","last_fullsized":"2024-09-17T21:32:05.248591+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:45:15.271044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0092951189999999993,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8770,"num_read_kb":53504,"num_write":580,"num_write_kb":24876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"136'630","reported_seq":6573,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699780+0000","last_change":"2024-09-17T21:31:45.425285+0000","last_active":"2024-09-17T21:32:04.699780+0000","last_peered":"2024-09-17T21:32:04.699780+0000","last_clean":"2024-09-17T21:32:04.699780+0000","last_became_active":"2024-09-17T21:30:28.747031+0000","last_became_peered":"2024-09-17T21:30:28.747031+0000","last_unstale":"2024-09-17T21:32:04.699780+0000","last_undegraded":"2024-09-17T21:32:04.699780+0000","last_fullsized":"2024-09-17T21:32:04.699780+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:26.185752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00134892,"stat_sum":{"num_bytes":19,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":7065,"num_read_kb":21661,"num_write":665,"num_write_kb":6726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":235969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"135'424","reported_seq":4765,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700031+0000","last_change":"2024-09-17T21:31:45.427269+0000","last_active":"2024-09-17T21:32:04.700031+0000","last_peered":"2024-09-17T21:32:04.700031+0000","last_clean":"2024-09-17T21:32:04.700031+0000","last_became_active":"2024-09-17T21:30:28.745634+0000","last_became_peered":"2024-09-17T21:30:28.745634+0000","last_unstale":"2024-09-17T21:32:04.700031+0000","last_undegraded":"2024-09-17T21:32:04.700031+0000","last_fullsized":"2024-09-17T21:32:04.700031+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:58.579935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001320393,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4551,"num_read_kb":56994,"num_write":426,"num_write_kb":45225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":36017677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"136'512","reported_seq":4537,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125260+0000","last_change":"2024-09-17T21:31:45.420094+0000","last_active":"2024-09-17T21:32:05.125260+0000","last_peered":"2024-09-17T21:32:05.125260+0000","last_clean":"2024-09-17T21:32:05.125260+0000","last_became_active":"2024-09-17T21:29:41.472468+0000","last_became_peered":"2024-09-17T21:29:41.472468+0000","last_unstale":"2024-09-17T21:32:05.125260+0000","last_undegraded":"2024-09-17T21:32:05.125260+0000","last_fullsized":"2024-09-17T21:32:05.125260+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:12:23.877166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002623551,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4791,"num_read_kb":27686,"num_write":500,"num_write_kb":13101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'814","reported_seq":3250,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248533+0000","last_change":"2024-09-17T21:31:45.431351+0000","last_active":"2024-09-17T21:32:05.248533+0000","last_peered":"2024-09-17T21:32:05.248533+0000","last_clean":"2024-09-17T21:32:05.248533+0000","last_became_active":"2024-09-17T21:31:15.456240+0000","last_became_peered":"2024-09-17T21:31:15.456240+0000","last_unstale":"2024-09-17T21:32:05.248533+0000","last_undegraded":"2024-09-17T21:32:05.248533+0000","last_fullsized":"2024-09-17T21:32:05.248533+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:31:41.549775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010220406,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4952,"num_read_kb":22844,"num_write":955,"num_write_kb":30092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":9394579,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"136'1106","reported_seq":3090,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704029+0000","last_change":"2024-09-17T21:31:45.435544+0000","last_active":"2024-09-17T21:32:04.704029+0000","last_peered":"2024-09-17T21:32:04.704029+0000","last_clean":"2024-09-17T21:32:04.704029+0000","last_became_active":"2024-09-17T21:31:15.456612+0000","last_became_peered":"2024-09-17T21:31:15.456612+0000","last_unstale":"2024-09-17T21:32:04.704029+0000","last_undegraded":"2024-09-17T21:32:04.704029+0000","last_fullsized":"2024-09-17T21:32:04.704029+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":1106,"log_dups_size":0,"ondisk_log_size":1106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:12:11.830676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051663100000000003,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":5737,"num_read_kb":20891,"num_write":1670,"num_write_kb":12347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3063082,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"135'512","reported_seq":3188,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311120+0000","last_change":"2024-09-17T21:31:45.497882+0000","last_active":"2024-09-17T21:32:05.311120+0000","last_peered":"2024-09-17T21:32:05.311120+0000","last_clean":"2024-09-17T21:32:05.311120+0000","last_became_active":"2024-09-17T21:29:41.478346+0000","last_became_peered":"2024-09-17T21:29:41.478346+0000","last_unstale":"2024-09-17T21:32:05.311120+0000","last_undegraded":"2024-09-17T21:32:05.311120+0000","last_fullsized":"2024-09-17T21:32:05.311120+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:58.424829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017578019999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3628,"num_read_kb":9299,"num_write":426,"num_write_kb":8355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"135'297","reported_seq":2884,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699913+0000","last_change":"2024-09-17T21:31:45.424549+0000","last_active":"2024-09-17T21:32:04.699913+0000","last_peered":"2024-09-17T21:32:04.699913+0000","last_clean":"2024-09-17T21:32:04.699913+0000","last_became_active":"2024-09-17T21:29:41.477611+0000","last_became_peered":"2024-09-17T21:29:41.477611+0000","last_unstale":"2024-09-17T21:32:04.699913+0000","last_undegraded":"2024-09-17T21:32:04.699913+0000","last_fullsized":"2024-09-17T21:32:04.699913+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:59:07.448045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064521399999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2154,"num_read_kb":14079,"num_write":352,"num_write_kb":12746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":280,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.180954+0000","last_change":"2024-09-17T21:29:35.002656+0000","last_active":"2024-09-17T21:32:05.180954+0000","last_peered":"2024-09-17T21:32:05.180954+0000","last_clean":"2024-09-17T21:32:05.180954+0000","last_became_active":"2024-09-17T21:29:32.971677+0000","last_became_peered":"2024-09-17T21:29:32.971677+0000","last_unstale":"2024-09-17T21:32:05.180954+0000","last_undegraded":"2024-09-17T21:32:05.180954+0000","last_fullsized":"2024-09-17T21:32:05.180954+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:53.781657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041121300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"136'848","reported_seq":4463,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248518+0000","last_change":"2024-09-17T21:31:45.431468+0000","last_active":"2024-09-17T21:32:05.248518+0000","last_peered":"2024-09-17T21:32:05.248518+0000","last_clean":"2024-09-17T21:32:05.248518+0000","last_became_active":"2024-09-17T21:30:31.755039+0000","last_became_peered":"2024-09-17T21:30:31.755039+0000","last_unstale":"2024-09-17T21:32:05.248518+0000","last_undegraded":"2024-09-17T21:32:05.248518+0000","last_fullsized":"2024-09-17T21:32:05.248518+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.621804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01026987,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6950,"num_read_kb":18293,"num_write":963,"num_write_kb":8597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":448871,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":213,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700243+0000","last_change":"2024-09-17T21:30:28.745759+0000","last_active":"2024-09-17T21:32:04.700243+0000","last_peered":"2024-09-17T21:32:04.700243+0000","last_clean":"2024-09-17T21:32:04.700243+0000","last_became_active":"2024-09-17T21:30:28.744742+0000","last_became_peered":"2024-09-17T21:30:28.744742+0000","last_unstale":"2024-09-17T21:32:04.700243+0000","last_undegraded":"2024-09-17T21:32:04.700243+0000","last_fullsized":"2024-09-17T21:32:04.700243+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:37:02.014566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"134'592","reported_seq":2794,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125373+0000","last_change":"2024-09-17T21:31:45.420595+0000","last_active":"2024-09-17T21:32:05.125373+0000","last_peered":"2024-09-17T21:32:05.125373+0000","last_clean":"2024-09-17T21:32:05.125373+0000","last_became_active":"2024-09-17T21:29:41.472095+0000","last_became_peered":"2024-09-17T21:29:41.472095+0000","last_unstale":"2024-09-17T21:32:05.125373+0000","last_undegraded":"2024-09-17T21:32:05.125373+0000","last_fullsized":"2024-09-17T21:32:05.125373+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:07:43.653815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028983989999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":3726,"num_read_kb":13082,"num_write":700,"num_write_kb":8995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":282,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311127+0000","last_change":"2024-09-17T21:29:35.078854+0000","last_active":"2024-09-17T21:32:05.311127+0000","last_peered":"2024-09-17T21:32:05.311127+0000","last_clean":"2024-09-17T21:32:05.311127+0000","last_became_active":"2024-09-17T21:29:33.071665+0000","last_became_peered":"2024-09-17T21:29:33.071665+0000","last_unstale":"2024-09-17T21:32:05.311127+0000","last_undegraded":"2024-09-17T21:32:05.311127+0000","last_fullsized":"2024-09-17T21:32:05.311127+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.415687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047969899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"134'805","reported_seq":4875,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700392+0000","last_change":"2024-09-17T21:31:45.426864+0000","last_active":"2024-09-17T21:32:04.700392+0000","last_peered":"2024-09-17T21:32:04.700392+0000","last_clean":"2024-09-17T21:32:04.700392+0000","last_became_active":"2024-09-17T21:30:31.752750+0000","last_became_peered":"2024-09-17T21:30:31.752750+0000","last_unstale":"2024-09-17T21:32:04.700392+0000","last_undegraded":"2024-09-17T21:32:04.700392+0000","last_fullsized":"2024-09-17T21:32:04.700392+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:27:53.025416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094011400000000003,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6270,"num_read_kb":32698,"num_write":1050,"num_write_kb":17424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3231045,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":213,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700238+0000","last_change":"2024-09-17T21:30:28.748319+0000","last_active":"2024-09-17T21:32:04.700238+0000","last_peered":"2024-09-17T21:32:04.700238+0000","last_clean":"2024-09-17T21:32:04.700238+0000","last_became_active":"2024-09-17T21:30:28.747728+0000","last_became_peered":"2024-09-17T21:30:28.747728+0000","last_unstale":"2024-09-17T21:32:04.700238+0000","last_undegraded":"2024-09-17T21:32:04.700238+0000","last_fullsized":"2024-09-17T21:32:04.700238+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:58:18.954005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"135'502","reported_seq":4854,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704234+0000","last_change":"2024-09-17T21:31:45.436069+0000","last_active":"2024-09-17T21:32:04.704234+0000","last_peered":"2024-09-17T21:32:04.704234+0000","last_clean":"2024-09-17T21:32:04.704234+0000","last_became_active":"2024-09-17T21:31:14.455957+0000","last_became_peered":"2024-09-17T21:31:14.455957+0000","last_unstale":"2024-09-17T21:32:04.704234+0000","last_undegraded":"2024-09-17T21:32:04.704234+0000","last_fullsized":"2024-09-17T21:32:04.704234+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:43:55.461066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087537200000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4988,"num_read_kb":31633,"num_write":558,"num_write_kb":9789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":1545182,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"136'856","reported_seq":5026,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699955+0000","last_change":"2024-09-17T21:31:45.424972+0000","last_active":"2024-09-17T21:32:04.699955+0000","last_peered":"2024-09-17T21:32:04.699955+0000","last_clean":"2024-09-17T21:32:04.699955+0000","last_became_active":"2024-09-17T21:29:41.475273+0000","last_became_peered":"2024-09-17T21:29:41.475273+0000","last_unstale":"2024-09-17T21:32:04.699955+0000","last_undegraded":"2024-09-17T21:32:04.699955+0000","last_fullsized":"2024-09-17T21:32:04.699955+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:24:39.329469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001059431,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":6333,"num_read_kb":18110,"num_write":998,"num_write_kb":6659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":116,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.703964+0000","last_change":"2024-09-17T21:31:11.424633+0000","last_active":"2024-09-17T21:32:04.703964+0000","last_peered":"2024-09-17T21:32:04.703964+0000","last_clean":"2024-09-17T21:32:04.703964+0000","last_became_active":"2024-09-17T21:31:11.421737+0000","last_became_peered":"2024-09-17T21:31:11.421737+0000","last_unstale":"2024-09-17T21:32:04.703964+0000","last_undegraded":"2024-09-17T21:32:04.703964+0000","last_fullsized":"2024-09-17T21:32:04.703964+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:07:58.025026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"135'413","reported_seq":2979,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700393+0000","last_change":"2024-09-17T21:31:45.426730+0000","last_active":"2024-09-17T21:32:04.700393+0000","last_peered":"2024-09-17T21:32:04.700393+0000","last_clean":"2024-09-17T21:32:04.700393+0000","last_became_active":"2024-09-17T21:30:28.746690+0000","last_became_peered":"2024-09-17T21:30:28.746690+0000","last_unstale":"2024-09-17T21:32:04.700393+0000","last_undegraded":"2024-09-17T21:32:04.700393+0000","last_fullsized":"2024-09-17T21:32:04.700393+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:53:30.001882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090442500000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3369,"num_read_kb":13331,"num_write":389,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1513,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":241,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700230+0000","last_change":"2024-09-17T21:31:53.979063+0000","last_active":"2024-09-17T21:32:04.700230+0000","last_peered":"2024-09-17T21:32:04.700230+0000","last_clean":"2024-09-17T21:32:04.700230+0000","last_became_active":"2024-09-17T21:31:53.978829+0000","last_became_peered":"2024-09-17T21:31:53.978829+0000","last_unstale":"2024-09-17T21:32:04.700230+0000","last_undegraded":"2024-09-17T21:32:04.700230+0000","last_fullsized":"2024-09-17T21:32:04.700230+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:52.874988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"135'431","reported_seq":3957,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248367+0000","last_change":"2024-09-17T21:31:45.430779+0000","last_active":"2024-09-17T21:32:05.248367+0000","last_peered":"2024-09-17T21:32:05.248367+0000","last_clean":"2024-09-17T21:32:05.248367+0000","last_became_active":"2024-09-17T21:29:41.473964+0000","last_became_peered":"2024-09-17T21:29:41.473964+0000","last_unstale":"2024-09-17T21:32:05.248367+0000","last_undegraded":"2024-09-17T21:32:05.248367+0000","last_fullsized":"2024-09-17T21:32:05.248367+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:45.162957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097061290000000008,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3802,"num_read_kb":64522,"num_write":578,"num_write_kb":39811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":291,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310779+0000","last_change":"2024-09-17T21:30:28.746557+0000","last_active":"2024-09-17T21:32:05.310779+0000","last_peered":"2024-09-17T21:32:05.310779+0000","last_clean":"2024-09-17T21:32:05.310779+0000","last_became_active":"2024-09-17T21:30:28.746261+0000","last_became_peered":"2024-09-17T21:30:28.746261+0000","last_unstale":"2024-09-17T21:32:05.310779+0000","last_undegraded":"2024-09-17T21:32:05.310779+0000","last_fullsized":"2024-09-17T21:32:05.310779+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:42.591689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044115199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"135'333","reported_seq":4194,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125209+0000","last_change":"2024-09-17T21:31:45.418657+0000","last_active":"2024-09-17T21:32:05.125209+0000","last_peered":"2024-09-17T21:32:05.125209+0000","last_clean":"2024-09-17T21:32:05.125209+0000","last_became_active":"2024-09-17T21:31:14.459936+0000","last_became_peered":"2024-09-17T21:31:14.459936+0000","last_unstale":"2024-09-17T21:32:05.125209+0000","last_undegraded":"2024-09-17T21:32:05.125209+0000","last_fullsized":"2024-09-17T21:32:05.125209+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:35.279004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012333649999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":3924,"num_read_kb":72696,"num_write":302,"num_write_kb":18067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":13048770,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":116,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.703960+0000","last_change":"2024-09-17T21:31:11.427558+0000","last_active":"2024-09-17T21:32:04.703960+0000","last_peered":"2024-09-17T21:32:04.703960+0000","last_clean":"2024-09-17T21:32:04.703960+0000","last_became_active":"2024-09-17T21:31:11.422076+0000","last_became_peered":"2024-09-17T21:31:11.422076+0000","last_unstale":"2024-09-17T21:32:04.703960+0000","last_undegraded":"2024-09-17T21:32:04.703960+0000","last_fullsized":"2024-09-17T21:32:04.703960+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:37:14.137712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":373,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310979+0000","last_change":"2024-09-17T21:29:28.554672+0000","last_active":"2024-09-17T21:32:05.310979+0000","last_peered":"2024-09-17T21:32:05.310979+0000","last_clean":"2024-09-17T21:32:05.310979+0000","last_became_active":"2024-09-17T21:29:28.553733+0000","last_became_peered":"2024-09-17T21:29:28.553733+0000","last_unstale":"2024-09-17T21:32:05.310979+0000","last_undegraded":"2024-09-17T21:32:05.310979+0000","last_fullsized":"2024-09-17T21:32:05.310979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_clean_scrub_stamp":"2024-09-17T21:29:26.925225+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:36:42.585477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"135'712","reported_seq":4573,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125288+0000","last_change":"2024-09-17T21:31:45.420733+0000","last_active":"2024-09-17T21:32:05.125288+0000","last_peered":"2024-09-17T21:32:05.125288+0000","last_clean":"2024-09-17T21:32:05.125288+0000","last_became_active":"2024-09-17T21:30:31.753839+0000","last_became_peered":"2024-09-17T21:30:31.753839+0000","last_unstale":"2024-09-17T21:32:05.125288+0000","last_undegraded":"2024-09-17T21:32:05.125288+0000","last_fullsized":"2024-09-17T21:32:05.125288+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:57.420241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030177569999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5120,"num_read_kb":35075,"num_write":882,"num_write_kb":14888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2410412,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"134'412","reported_seq":3583,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699862+0000","last_change":"2024-09-17T21:31:45.424510+0000","last_active":"2024-09-17T21:32:04.699862+0000","last_peered":"2024-09-17T21:32:04.699862+0000","last_clean":"2024-09-17T21:32:04.699862+0000","last_became_active":"2024-09-17T21:29:41.477965+0000","last_became_peered":"2024-09-17T21:29:41.477965+0000","last_unstale":"2024-09-17T21:32:04.699862+0000","last_undegraded":"2024-09-17T21:32:04.699862+0000","last_fullsized":"2024-09-17T21:32:04.699862+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:24:25.503913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062142999999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3781,"num_read_kb":28832,"num_write":427,"num_write_kb":12788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"136'595","reported_seq":4077,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.180946+0000","last_change":"2024-09-17T21:31:45.419838+0000","last_active":"2024-09-17T21:32:05.180946+0000","last_peered":"2024-09-17T21:32:05.180946+0000","last_clean":"2024-09-17T21:32:05.180946+0000","last_became_active":"2024-09-17T21:29:41.473420+0000","last_became_peered":"2024-09-17T21:29:41.473420+0000","last_unstale":"2024-09-17T21:32:05.180946+0000","last_undegraded":"2024-09-17T21:32:05.180946+0000","last_fullsized":"2024-09-17T21:32:05.180946+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:33:22.091302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058409800000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":4583,"num_read_kb":15568,"num_write":610,"num_write_kb":6172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"135'541","reported_seq":7208,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125233+0000","last_change":"2024-09-17T21:31:45.420089+0000","last_active":"2024-09-17T21:32:05.125233+0000","last_peered":"2024-09-17T21:32:05.125233+0000","last_clean":"2024-09-17T21:32:05.125233+0000","last_became_active":"2024-09-17T21:29:41.472354+0000","last_became_peered":"2024-09-17T21:29:41.472354+0000","last_unstale":"2024-09-17T21:32:05.125233+0000","last_undegraded":"2024-09-17T21:32:05.125233+0000","last_fullsized":"2024-09-17T21:32:05.125233+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:30.670868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025676010000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7686,"num_read_kb":30319,"num_write":462,"num_write_kb":3994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"135'661","reported_seq":4876,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.703998+0000","last_change":"2024-09-17T21:31:45.435964+0000","last_active":"2024-09-17T21:32:04.703998+0000","last_peered":"2024-09-17T21:32:04.703998+0000","last_clean":"2024-09-17T21:32:04.703998+0000","last_became_active":"2024-09-17T21:31:15.457822+0000","last_became_peered":"2024-09-17T21:31:15.457822+0000","last_unstale":"2024-09-17T21:32:04.703998+0000","last_undegraded":"2024-09-17T21:32:04.703998+0000","last_fullsized":"2024-09-17T21:32:04.703998+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:45:34.971106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086872700000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7278,"num_read_kb":17574,"num_write":576,"num_write_kb":5172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":2355433,"num_keys_recovered":58,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"135'622","reported_seq":4516,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310922+0000","last_change":"2024-09-17T21:31:45.497982+0000","last_active":"2024-09-17T21:32:05.310922+0000","last_peered":"2024-09-17T21:32:05.310922+0000","last_clean":"2024-09-17T21:32:05.310922+0000","last_became_active":"2024-09-17T21:31:14.457648+0000","last_became_peered":"2024-09-17T21:31:14.457648+0000","last_unstale":"2024-09-17T21:32:05.310922+0000","last_undegraded":"2024-09-17T21:32:05.310922+0000","last_fullsized":"2024-09-17T21:32:05.310922+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:10:28.622328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016040850000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5354,"num_read_kb":12719,"num_write":707,"num_write_kb":4578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2843955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"136'580","reported_seq":4386,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311131+0000","last_change":"2024-09-17T21:31:45.497414+0000","last_active":"2024-09-17T21:32:05.311131+0000","last_peered":"2024-09-17T21:32:05.311131+0000","last_clean":"2024-09-17T21:32:05.311131+0000","last_became_active":"2024-09-17T21:29:41.473736+0000","last_became_peered":"2024-09-17T21:29:41.473736+0000","last_unstale":"2024-09-17T21:32:05.311131+0000","last_undegraded":"2024-09-17T21:32:05.311131+0000","last_fullsized":"2024-09-17T21:32:05.311131+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:12:54.605441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013374140000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5042,"num_read_kb":17595,"num_write":692,"num_write_kb":5927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"135'882","reported_seq":4533,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311122+0000","last_change":"2024-09-17T21:31:45.498420+0000","last_active":"2024-09-17T21:32:05.311122+0000","last_peered":"2024-09-17T21:32:05.311122+0000","last_clean":"2024-09-17T21:32:05.311122+0000","last_became_active":"2024-09-17T21:29:41.473394+0000","last_became_peered":"2024-09-17T21:29:41.473394+0000","last_unstale":"2024-09-17T21:32:05.311122+0000","last_undegraded":"2024-09-17T21:32:05.311122+0000","last_fullsized":"2024-09-17T21:32:05.311122+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:45:31.022697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002038134,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6008,"num_read_kb":8422,"num_write":1180,"num_write_kb":21760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"134'623","reported_seq":2582,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.180906+0000","last_change":"2024-09-17T21:31:45.419674+0000","last_active":"2024-09-17T21:32:05.180906+0000","last_peered":"2024-09-17T21:32:05.180906+0000","last_clean":"2024-09-17T21:32:05.180906+0000","last_became_active":"2024-09-17T21:29:41.471342+0000","last_became_peered":"2024-09-17T21:29:41.471342+0000","last_unstale":"2024-09-17T21:32:05.180906+0000","last_undegraded":"2024-09-17T21:32:05.180906+0000","last_fullsized":"2024-09-17T21:32:05.180906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:23:05.457286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044458799999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3239,"num_read_kb":14667,"num_write":843,"num_write_kb":8817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"136'418","reported_seq":4857,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310899+0000","last_change":"2024-09-17T21:31:45.497400+0000","last_active":"2024-09-17T21:32:05.310899+0000","last_peered":"2024-09-17T21:32:05.310899+0000","last_clean":"2024-09-17T21:32:05.310899+0000","last_became_active":"2024-09-17T21:29:41.473252+0000","last_became_peered":"2024-09-17T21:29:41.473252+0000","last_unstale":"2024-09-17T21:32:05.310899+0000","last_undegraded":"2024-09-17T21:32:05.310899+0000","last_fullsized":"2024-09-17T21:32:05.310899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:36:31.575810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013588999999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4864,"num_read_kb":34676,"num_write":382,"num_write_kb":15080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"136'375","reported_seq":2283,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248376+0000","last_change":"2024-09-17T21:31:45.430386+0000","last_active":"2024-09-17T21:32:05.248376+0000","last_peered":"2024-09-17T21:32:05.248376+0000","last_clean":"2024-09-17T21:32:05.248376+0000","last_became_active":"2024-09-17T21:29:41.477019+0000","last_became_peered":"2024-09-17T21:29:41.477019+0000","last_unstale":"2024-09-17T21:32:05.248376+0000","last_undegraded":"2024-09-17T21:32:05.248376+0000","last_fullsized":"2024-09-17T21:32:05.248376+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:07:16.797282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094423349999999996,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":1879,"num_read_kb":5231,"num_write":351,"num_write_kb":2424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"135'440","reported_seq":4067,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310943+0000","last_change":"2024-09-17T21:31:45.497003+0000","last_active":"2024-09-17T21:32:05.310943+0000","last_peered":"2024-09-17T21:32:05.310943+0000","last_clean":"2024-09-17T21:32:05.310943+0000","last_became_active":"2024-09-17T21:29:41.473182+0000","last_became_peered":"2024-09-17T21:29:41.473182+0000","last_unstale":"2024-09-17T21:32:05.310943+0000","last_undegraded":"2024-09-17T21:32:05.310943+0000","last_fullsized":"2024-09-17T21:32:05.310943+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:55:27.817705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092929600000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3686,"num_read_kb":22454,"num_write":375,"num_write_kb":9447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":656,"num_object_clones":0,"num_object_copies":1312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":656,"num_whiteouts":652,"num_read":159356,"num_read_kb":893959,"num_write":21643,"num_write_kb":452621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1205,"num_bytes_recovered":117790825,"num_keys_recovered":327,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10476,"internal_metadata":0},"log_size":19097,"ondisk_log_size":19097,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509476,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55256,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316584,"statfs":{"total":96636764160,"available":96580182016,"internally_reserved":0,"allocated":1900544,"data_stored":1469370,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28808,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343032,"statfs":{"total":96636764160,"available":96607264768,"internally_reserved":0,"allocated":1818624,"data_stored":867376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41316,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330524,"statfs":{"total":96636764160,"available":96594456576,"internally_reserved":0,"allocated":1847296,"data_stored":879021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509476,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59368,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312472,"statfs":{"total":96636764160,"available":96575971328,"internally_reserved":0,"allocated":1847296,"data_stored":879021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55192,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316648,"statfs":{"total":96636764160,"available":96580247552,"internally_reserved":0,"allocated":1830912,"data_stored":873189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509477,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41316,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330524,"statfs":{"total":96636764160,"available":96594456576,"internally_reserved":0,"allocated":1843200,"data_stored":879002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":63,"seq":270582939670,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42916,"kb_used_data":1808,"kb_used_omap":20,"kb_used_meta":41067,"kb_avail":94328924,"statfs":{"total":96636764160,"available":96592818176,"internally_reserved":0,"allocated":1851392,"data_stored":879040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20925,"internal_metadata":42053187},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55228,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316612,"statfs":{"total":96636764160,"available":96580210688,"internally_reserved":0,"allocated":1875968,"data_stored":1457744,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10476,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:32:10.231 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:32:10.515 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:32:10.516 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-17T21:32:10.527 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-17T21:32:08.887764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":660,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":660,"num_whiteouts":652,"num_read":159422,"num_read_kb":894014,"num_write":21746,"num_write_kb":454251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1208,"num_bytes_recovered":117790844,"num_keys_recovered":327,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19172,"ondisk_log_size":19172,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379400,"kb_used_data":14468,"kb_used_omap":31,"kb_used_meta":364704,"kb_avail":754595320,"statfs":{"total":773094113280,"available":772705607680,"internally_reserved":0,"allocated":14815232,"data_stored":8183763,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":32080,"internal_metadata":373457584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.202366"},"pg_stats":[{"pgid":"3.1f","version":"136'809","reported_seq":5134,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248455+0000","last_change":"2024-09-17T21:31:45.430953+0000","last_active":"2024-09-17T21:32:05.248455+0000","last_peered":"2024-09-17T21:32:05.248455+0000","last_clean":"2024-09-17T21:32:05.248455+0000","last_became_active":"2024-09-17T21:31:15.457506+0000","last_became_peered":"2024-09-17T21:31:15.457506+0000","last_unstale":"2024-09-17T21:32:05.248455+0000","last_undegraded":"2024-09-17T21:32:05.248455+0000","last_fullsized":"2024-09-17T21:32:05.248455+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:08.923954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098455079999999993,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6585,"num_read_kb":63347,"num_write":1151,"num_write_kb":32903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":27321072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"136'562","reported_seq":4234,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125356+0000","last_change":"2024-09-17T21:31:45.418470+0000","last_active":"2024-09-17T21:32:05.125356+0000","last_peered":"2024-09-17T21:32:05.125356+0000","last_clean":"2024-09-17T21:32:05.125356+0000","last_became_active":"2024-09-17T21:29:41.472161+0000","last_became_peered":"2024-09-17T21:29:41.472161+0000","last_unstale":"2024-09-17T21:32:05.125356+0000","last_undegraded":"2024-09-17T21:32:05.125356+0000","last_fullsized":"2024-09-17T21:32:05.125356+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:04:47.812939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079279,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3917,"num_read_kb":31515,"num_write":544,"num_write_kb":11792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"135'768","reported_seq":5402,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704245+0000","last_change":"2024-09-17T21:31:45.436342+0000","last_active":"2024-09-17T21:32:04.704245+0000","last_peered":"2024-09-17T21:32:04.704245+0000","last_clean":"2024-09-17T21:32:04.704245+0000","last_became_active":"2024-09-17T21:31:14.449672+0000","last_became_peered":"2024-09-17T21:31:14.449672+0000","last_unstale":"2024-09-17T21:32:04.704245+0000","last_undegraded":"2024-09-17T21:32:04.704245+0000","last_fullsized":"2024-09-17T21:32:04.704245+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:48:24.256603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010859089999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6612,"num_read_kb":51732,"num_write":893,"num_write_kb":18876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":15268754,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"136'433","reported_seq":2685,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704042+0000","last_change":"2024-09-17T21:31:45.435673+0000","last_active":"2024-09-17T21:32:04.704042+0000","last_peered":"2024-09-17T21:32:04.704042+0000","last_clean":"2024-09-17T21:32:04.704042+0000","last_became_active":"2024-09-17T21:31:14.451037+0000","last_became_peered":"2024-09-17T21:31:14.451037+0000","last_unstale":"2024-09-17T21:32:04.704042+0000","last_undegraded":"2024-09-17T21:32:04.704042+0000","last_fullsized":"2024-09-17T21:32:04.704042+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:41:20.330623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068460700000000005,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":2712,"num_read_kb":12910,"num_write":456,"num_write_kb":9120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":604511,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"136'589","reported_seq":8494,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248591+0000","last_change":"2024-09-17T21:31:45.430352+0000","last_active":"2024-09-17T21:32:05.248591+0000","last_peered":"2024-09-17T21:32:05.248591+0000","last_clean":"2024-09-17T21:32:05.248591+0000","last_became_active":"2024-09-17T21:29:41.476041+0000","last_became_peered":"2024-09-17T21:29:41.476041+0000","last_unstale":"2024-09-17T21:32:05.248591+0000","last_undegraded":"2024-09-17T21:32:05.248591+0000","last_fullsized":"2024-09-17T21:32:05.248591+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:45:15.271044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0092951189999999993,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8770,"num_read_kb":53504,"num_write":580,"num_write_kb":24876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"136'630","reported_seq":6573,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699780+0000","last_change":"2024-09-17T21:31:45.425285+0000","last_active":"2024-09-17T21:32:04.699780+0000","last_peered":"2024-09-17T21:32:04.699780+0000","last_clean":"2024-09-17T21:32:04.699780+0000","last_became_active":"2024-09-17T21:30:28.747031+0000","last_became_peered":"2024-09-17T21:30:28.747031+0000","last_unstale":"2024-09-17T21:32:04.699780+0000","last_undegraded":"2024-09-17T21:32:04.699780+0000","last_fullsized":"2024-09-17T21:32:04.699780+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:26.185752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00134892,"stat_sum":{"num_bytes":19,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":7065,"num_read_kb":21661,"num_write":665,"num_write_kb":6726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":235969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"135'424","reported_seq":4765,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700031+0000","last_change":"2024-09-17T21:31:45.427269+0000","last_active":"2024-09-17T21:32:04.700031+0000","last_peered":"2024-09-17T21:32:04.700031+0000","last_clean":"2024-09-17T21:32:04.700031+0000","last_became_active":"2024-09-17T21:30:28.745634+0000","last_became_peered":"2024-09-17T21:30:28.745634+0000","last_unstale":"2024-09-17T21:32:04.700031+0000","last_undegraded":"2024-09-17T21:32:04.700031+0000","last_fullsized":"2024-09-17T21:32:04.700031+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:58.579935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001320393,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4551,"num_read_kb":56994,"num_write":426,"num_write_kb":45225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":36017677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"136'512","reported_seq":4537,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125260+0000","last_change":"2024-09-17T21:31:45.420094+0000","last_active":"2024-09-17T21:32:05.125260+0000","last_peered":"2024-09-17T21:32:05.125260+0000","last_clean":"2024-09-17T21:32:05.125260+0000","last_became_active":"2024-09-17T21:29:41.472468+0000","last_became_peered":"2024-09-17T21:29:41.472468+0000","last_unstale":"2024-09-17T21:32:05.125260+0000","last_undegraded":"2024-09-17T21:32:05.125260+0000","last_fullsized":"2024-09-17T21:32:05.125260+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:12:23.877166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002623551,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4791,"num_read_kb":27686,"num_write":500,"num_write_kb":13101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'814","reported_seq":3250,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248533+0000","last_change":"2024-09-17T21:31:45.431351+0000","last_active":"2024-09-17T21:32:05.248533+0000","last_peered":"2024-09-17T21:32:05.248533+0000","last_clean":"2024-09-17T21:32:05.248533+0000","last_became_active":"2024-09-17T21:31:15.456240+0000","last_became_peered":"2024-09-17T21:31:15.456240+0000","last_unstale":"2024-09-17T21:32:05.248533+0000","last_undegraded":"2024-09-17T21:32:05.248533+0000","last_fullsized":"2024-09-17T21:32:05.248533+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:31:41.549775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010220406,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4952,"num_read_kb":22844,"num_write":955,"num_write_kb":30092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":9394579,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"136'1106","reported_seq":3090,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704029+0000","last_change":"2024-09-17T21:31:45.435544+0000","last_active":"2024-09-17T21:32:04.704029+0000","last_peered":"2024-09-17T21:32:04.704029+0000","last_clean":"2024-09-17T21:32:04.704029+0000","last_became_active":"2024-09-17T21:31:15.456612+0000","last_became_peered":"2024-09-17T21:31:15.456612+0000","last_unstale":"2024-09-17T21:32:04.704029+0000","last_undegraded":"2024-09-17T21:32:04.704029+0000","last_fullsized":"2024-09-17T21:32:04.704029+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":1106,"log_dups_size":0,"ondisk_log_size":1106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:12:11.830676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051663100000000003,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":5737,"num_read_kb":20891,"num_write":1670,"num_write_kb":12347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3063082,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"135'512","reported_seq":3188,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311120+0000","last_change":"2024-09-17T21:31:45.497882+0000","last_active":"2024-09-17T21:32:05.311120+0000","last_peered":"2024-09-17T21:32:05.311120+0000","last_clean":"2024-09-17T21:32:05.311120+0000","last_became_active":"2024-09-17T21:29:41.478346+0000","last_became_peered":"2024-09-17T21:29:41.478346+0000","last_unstale":"2024-09-17T21:32:05.311120+0000","last_undegraded":"2024-09-17T21:32:05.311120+0000","last_fullsized":"2024-09-17T21:32:05.311120+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:58.424829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017578019999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3628,"num_read_kb":9299,"num_write":426,"num_write_kb":8355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"135'297","reported_seq":2884,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699913+0000","last_change":"2024-09-17T21:31:45.424549+0000","last_active":"2024-09-17T21:32:04.699913+0000","last_peered":"2024-09-17T21:32:04.699913+0000","last_clean":"2024-09-17T21:32:04.699913+0000","last_became_active":"2024-09-17T21:29:41.477611+0000","last_became_peered":"2024-09-17T21:29:41.477611+0000","last_unstale":"2024-09-17T21:32:04.699913+0000","last_undegraded":"2024-09-17T21:32:04.699913+0000","last_fullsized":"2024-09-17T21:32:04.699913+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:59:07.448045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064521399999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2154,"num_read_kb":14079,"num_write":352,"num_write_kb":12746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":280,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.180954+0000","last_change":"2024-09-17T21:29:35.002656+0000","last_active":"2024-09-17T21:32:05.180954+0000","last_peered":"2024-09-17T21:32:05.180954+0000","last_clean":"2024-09-17T21:32:05.180954+0000","last_became_active":"2024-09-17T21:29:32.971677+0000","last_became_peered":"2024-09-17T21:29:32.971677+0000","last_unstale":"2024-09-17T21:32:05.180954+0000","last_undegraded":"2024-09-17T21:32:05.180954+0000","last_fullsized":"2024-09-17T21:32:05.180954+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:53.781657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041121300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"136'848","reported_seq":4463,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248518+0000","last_change":"2024-09-17T21:31:45.431468+0000","last_active":"2024-09-17T21:32:05.248518+0000","last_peered":"2024-09-17T21:32:05.248518+0000","last_clean":"2024-09-17T21:32:05.248518+0000","last_became_active":"2024-09-17T21:30:31.755039+0000","last_became_peered":"2024-09-17T21:30:31.755039+0000","last_unstale":"2024-09-17T21:32:05.248518+0000","last_undegraded":"2024-09-17T21:32:05.248518+0000","last_fullsized":"2024-09-17T21:32:05.248518+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.621804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01026987,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6950,"num_read_kb":18293,"num_write":963,"num_write_kb":8597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":448871,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":213,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700243+0000","last_change":"2024-09-17T21:30:28.745759+0000","last_active":"2024-09-17T21:32:04.700243+0000","last_peered":"2024-09-17T21:32:04.700243+0000","last_clean":"2024-09-17T21:32:04.700243+0000","last_became_active":"2024-09-17T21:30:28.744742+0000","last_became_peered":"2024-09-17T21:30:28.744742+0000","last_unstale":"2024-09-17T21:32:04.700243+0000","last_undegraded":"2024-09-17T21:32:04.700243+0000","last_fullsized":"2024-09-17T21:32:04.700243+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:37:02.014566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"134'592","reported_seq":2794,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125373+0000","last_change":"2024-09-17T21:31:45.420595+0000","last_active":"2024-09-17T21:32:05.125373+0000","last_peered":"2024-09-17T21:32:05.125373+0000","last_clean":"2024-09-17T21:32:05.125373+0000","last_became_active":"2024-09-17T21:29:41.472095+0000","last_became_peered":"2024-09-17T21:29:41.472095+0000","last_unstale":"2024-09-17T21:32:05.125373+0000","last_undegraded":"2024-09-17T21:32:05.125373+0000","last_fullsized":"2024-09-17T21:32:05.125373+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:07:43.653815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028983989999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":3726,"num_read_kb":13082,"num_write":700,"num_write_kb":8995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":282,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311127+0000","last_change":"2024-09-17T21:29:35.078854+0000","last_active":"2024-09-17T21:32:05.311127+0000","last_peered":"2024-09-17T21:32:05.311127+0000","last_clean":"2024-09-17T21:32:05.311127+0000","last_became_active":"2024-09-17T21:29:33.071665+0000","last_became_peered":"2024-09-17T21:29:33.071665+0000","last_unstale":"2024-09-17T21:32:05.311127+0000","last_undegraded":"2024-09-17T21:32:05.311127+0000","last_fullsized":"2024-09-17T21:32:05.311127+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.415687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047969899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"134'805","reported_seq":4875,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700392+0000","last_change":"2024-09-17T21:31:45.426864+0000","last_active":"2024-09-17T21:32:04.700392+0000","last_peered":"2024-09-17T21:32:04.700392+0000","last_clean":"2024-09-17T21:32:04.700392+0000","last_became_active":"2024-09-17T21:30:31.752750+0000","last_became_peered":"2024-09-17T21:30:31.752750+0000","last_unstale":"2024-09-17T21:32:04.700392+0000","last_undegraded":"2024-09-17T21:32:04.700392+0000","last_fullsized":"2024-09-17T21:32:04.700392+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:27:53.025416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094011400000000003,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6270,"num_read_kb":32698,"num_write":1050,"num_write_kb":17424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3231045,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":213,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700238+0000","last_change":"2024-09-17T21:30:28.748319+0000","last_active":"2024-09-17T21:32:04.700238+0000","last_peered":"2024-09-17T21:32:04.700238+0000","last_clean":"2024-09-17T21:32:04.700238+0000","last_became_active":"2024-09-17T21:30:28.747728+0000","last_became_peered":"2024-09-17T21:30:28.747728+0000","last_unstale":"2024-09-17T21:32:04.700238+0000","last_undegraded":"2024-09-17T21:32:04.700238+0000","last_fullsized":"2024-09-17T21:32:04.700238+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:58:18.954005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"135'502","reported_seq":4854,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704234+0000","last_change":"2024-09-17T21:31:45.436069+0000","last_active":"2024-09-17T21:32:04.704234+0000","last_peered":"2024-09-17T21:32:04.704234+0000","last_clean":"2024-09-17T21:32:04.704234+0000","last_became_active":"2024-09-17T21:31:14.455957+0000","last_became_peered":"2024-09-17T21:31:14.455957+0000","last_unstale":"2024-09-17T21:32:04.704234+0000","last_undegraded":"2024-09-17T21:32:04.704234+0000","last_fullsized":"2024-09-17T21:32:04.704234+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:43:55.461066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087537200000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4988,"num_read_kb":31633,"num_write":558,"num_write_kb":9789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":1545182,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"136'856","reported_seq":5026,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699955+0000","last_change":"2024-09-17T21:31:45.424972+0000","last_active":"2024-09-17T21:32:04.699955+0000","last_peered":"2024-09-17T21:32:04.699955+0000","last_clean":"2024-09-17T21:32:04.699955+0000","last_became_active":"2024-09-17T21:29:41.475273+0000","last_became_peered":"2024-09-17T21:29:41.475273+0000","last_unstale":"2024-09-17T21:32:04.699955+0000","last_undegraded":"2024-09-17T21:32:04.699955+0000","last_fullsized":"2024-09-17T21:32:04.699955+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:24:39.329469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001059431,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":6333,"num_read_kb":18110,"num_write":998,"num_write_kb":6659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":116,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.703964+0000","last_change":"2024-09-17T21:31:11.424633+0000","last_active":"2024-09-17T21:32:04.703964+0000","last_peered":"2024-09-17T21:32:04.703964+0000","last_clean":"2024-09-17T21:32:04.703964+0000","last_became_active":"2024-09-17T21:31:11.421737+0000","last_became_peered":"2024-09-17T21:31:11.421737+0000","last_unstale":"2024-09-17T21:32:04.703964+0000","last_undegraded":"2024-09-17T21:32:04.703964+0000","last_fullsized":"2024-09-17T21:32:04.703964+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:07:58.025026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"135'413","reported_seq":2979,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700393+0000","last_change":"2024-09-17T21:31:45.426730+0000","last_active":"2024-09-17T21:32:04.700393+0000","last_peered":"2024-09-17T21:32:04.700393+0000","last_clean":"2024-09-17T21:32:04.700393+0000","last_became_active":"2024-09-17T21:30:28.746690+0000","last_became_peered":"2024-09-17T21:30:28.746690+0000","last_unstale":"2024-09-17T21:32:04.700393+0000","last_undegraded":"2024-09-17T21:32:04.700393+0000","last_fullsized":"2024-09-17T21:32:04.700393+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:53:30.001882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090442500000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3369,"num_read_kb":13331,"num_write":389,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1513,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":241,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700230+0000","last_change":"2024-09-17T21:31:53.979063+0000","last_active":"2024-09-17T21:32:04.700230+0000","last_peered":"2024-09-17T21:32:04.700230+0000","last_clean":"2024-09-17T21:32:04.700230+0000","last_became_active":"2024-09-17T21:31:53.978829+0000","last_became_peered":"2024-09-17T21:31:53.978829+0000","last_unstale":"2024-09-17T21:32:04.700230+0000","last_undegraded":"2024-09-17T21:32:04.700230+0000","last_fullsized":"2024-09-17T21:32:04.700230+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:52.874988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"135'431","reported_seq":3957,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248367+0000","last_change":"2024-09-17T21:31:45.430779+0000","last_active":"2024-09-17T21:32:05.248367+0000","last_peered":"2024-09-17T21:32:05.248367+0000","last_clean":"2024-09-17T21:32:05.248367+0000","last_became_active":"2024-09-17T21:29:41.473964+0000","last_became_peered":"2024-09-17T21:29:41.473964+0000","last_unstale":"2024-09-17T21:32:05.248367+0000","last_undegraded":"2024-09-17T21:32:05.248367+0000","last_fullsized":"2024-09-17T21:32:05.248367+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:45.162957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097061290000000008,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3802,"num_read_kb":64522,"num_write":578,"num_write_kb":39811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":291,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310779+0000","last_change":"2024-09-17T21:30:28.746557+0000","last_active":"2024-09-17T21:32:05.310779+0000","last_peered":"2024-09-17T21:32:05.310779+0000","last_clean":"2024-09-17T21:32:05.310779+0000","last_became_active":"2024-09-17T21:30:28.746261+0000","last_became_peered":"2024-09-17T21:30:28.746261+0000","last_unstale":"2024-09-17T21:32:05.310779+0000","last_undegraded":"2024-09-17T21:32:05.310779+0000","last_fullsized":"2024-09-17T21:32:05.310779+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:42.591689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044115199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"135'333","reported_seq":4194,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125209+0000","last_change":"2024-09-17T21:31:45.418657+0000","last_active":"2024-09-17T21:32:05.125209+0000","last_peered":"2024-09-17T21:32:05.125209+0000","last_clean":"2024-09-17T21:32:05.125209+0000","last_became_active":"2024-09-17T21:31:14.459936+0000","last_became_peered":"2024-09-17T21:31:14.459936+0000","last_unstale":"2024-09-17T21:32:05.125209+0000","last_undegraded":"2024-09-17T21:32:05.125209+0000","last_fullsized":"2024-09-17T21:32:05.125209+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:35.279004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012333649999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":3924,"num_read_kb":72696,"num_write":302,"num_write_kb":18067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":13048770,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":116,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.703960+0000","last_change":"2024-09-17T21:31:11.427558+0000","last_active":"2024-09-17T21:32:04.703960+0000","last_peered":"2024-09-17T21:32:04.703960+0000","last_clean":"2024-09-17T21:32:04.703960+0000","last_became_active":"2024-09-17T21:31:11.422076+0000","last_became_peered":"2024-09-17T21:31:11.422076+0000","last_unstale":"2024-09-17T21:32:04.703960+0000","last_undegraded":"2024-09-17T21:32:04.703960+0000","last_fullsized":"2024-09-17T21:32:04.703960+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:37:14.137712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":373,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310979+0000","last_change":"2024-09-17T21:29:28.554672+0000","last_active":"2024-09-17T21:32:05.310979+0000","last_peered":"2024-09-17T21:32:05.310979+0000","last_clean":"2024-09-17T21:32:05.310979+0000","last_became_active":"2024-09-17T21:29:28.553733+0000","last_became_peered":"2024-09-17T21:29:28.553733+0000","last_unstale":"2024-09-17T21:32:05.310979+0000","last_undegraded":"2024-09-17T21:32:05.310979+0000","last_fullsized":"2024-09-17T21:32:05.310979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_clean_scrub_stamp":"2024-09-17T21:29:26.925225+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:36:42.585477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"135'712","reported_seq":4573,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125288+0000","last_change":"2024-09-17T21:31:45.420733+0000","last_active":"2024-09-17T21:32:05.125288+0000","last_peered":"2024-09-17T21:32:05.125288+0000","last_clean":"2024-09-17T21:32:05.125288+0000","last_became_active":"2024-09-17T21:30:31.753839+0000","last_became_peered":"2024-09-17T21:30:31.753839+0000","last_unstale":"2024-09-17T21:32:05.125288+0000","last_undegraded":"2024-09-17T21:32:05.125288+0000","last_fullsized":"2024-09-17T21:32:05.125288+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:57.420241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030177569999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5120,"num_read_kb":35075,"num_write":882,"num_write_kb":14888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2410412,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"134'412","reported_seq":3583,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699862+0000","last_change":"2024-09-17T21:31:45.424510+0000","last_active":"2024-09-17T21:32:04.699862+0000","last_peered":"2024-09-17T21:32:04.699862+0000","last_clean":"2024-09-17T21:32:04.699862+0000","last_became_active":"2024-09-17T21:29:41.477965+0000","last_became_peered":"2024-09-17T21:29:41.477965+0000","last_unstale":"2024-09-17T21:32:04.699862+0000","last_undegraded":"2024-09-17T21:32:04.699862+0000","last_fullsized":"2024-09-17T21:32:04.699862+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:24:25.503913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062142999999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3781,"num_read_kb":28832,"num_write":427,"num_write_kb":12788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"136'595","reported_seq":4077,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.180946+0000","last_change":"2024-09-17T21:31:45.419838+0000","last_active":"2024-09-17T21:32:05.180946+0000","last_peered":"2024-09-17T21:32:05.180946+0000","last_clean":"2024-09-17T21:32:05.180946+0000","last_became_active":"2024-09-17T21:29:41.473420+0000","last_became_peered":"2024-09-17T21:29:41.473420+0000","last_unstale":"2024-09-17T21:32:05.180946+0000","last_undegraded":"2024-09-17T21:32:05.180946+0000","last_fullsized":"2024-09-17T21:32:05.180946+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:33:22.091302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058409800000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":4583,"num_read_kb":15568,"num_write":610,"num_write_kb":6172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"135'541","reported_seq":7208,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125233+0000","last_change":"2024-09-17T21:31:45.420089+0000","last_active":"2024-09-17T21:32:05.125233+0000","last_peered":"2024-09-17T21:32:05.125233+0000","last_clean":"2024-09-17T21:32:05.125233+0000","last_became_active":"2024-09-17T21:29:41.472354+0000","last_became_peered":"2024-09-17T21:29:41.472354+0000","last_unstale":"2024-09-17T21:32:05.125233+0000","last_undegraded":"2024-09-17T21:32:05.125233+0000","last_fullsized":"2024-09-17T21:32:05.125233+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:30.670868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025676010000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7686,"num_read_kb":30319,"num_write":462,"num_write_kb":3994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"135'661","reported_seq":4876,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.703998+0000","last_change":"2024-09-17T21:31:45.435964+0000","last_active":"2024-09-17T21:32:04.703998+0000","last_peered":"2024-09-17T21:32:04.703998+0000","last_clean":"2024-09-17T21:32:04.703998+0000","last_became_active":"2024-09-17T21:31:15.457822+0000","last_became_peered":"2024-09-17T21:31:15.457822+0000","last_unstale":"2024-09-17T21:32:04.703998+0000","last_undegraded":"2024-09-17T21:32:04.703998+0000","last_fullsized":"2024-09-17T21:32:04.703998+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:45:34.971106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086872700000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7278,"num_read_kb":17574,"num_write":576,"num_write_kb":5172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":2355433,"num_keys_recovered":58,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"135'622","reported_seq":4516,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310922+0000","last_change":"2024-09-17T21:31:45.497982+0000","last_active":"2024-09-17T21:32:05.310922+0000","last_peered":"2024-09-17T21:32:05.310922+0000","last_clean":"2024-09-17T21:32:05.310922+0000","last_became_active":"2024-09-17T21:31:14.457648+0000","last_became_peered":"2024-09-17T21:31:14.457648+0000","last_unstale":"2024-09-17T21:32:05.310922+0000","last_undegraded":"2024-09-17T21:32:05.310922+0000","last_fullsized":"2024-09-17T21:32:05.310922+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:10:28.622328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016040850000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5354,"num_read_kb":12719,"num_write":707,"num_write_kb":4578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2843955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"136'580","reported_seq":4386,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311131+0000","last_change":"2024-09-17T21:31:45.497414+0000","last_active":"2024-09-17T21:32:05.311131+0000","last_peered":"2024-09-17T21:32:05.311131+0000","last_clean":"2024-09-17T21:32:05.311131+0000","last_became_active":"2024-09-17T21:29:41.473736+0000","last_became_peered":"2024-09-17T21:29:41.473736+0000","last_unstale":"2024-09-17T21:32:05.311131+0000","last_undegraded":"2024-09-17T21:32:05.311131+0000","last_fullsized":"2024-09-17T21:32:05.311131+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:12:54.605441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013374140000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5042,"num_read_kb":17595,"num_write":692,"num_write_kb":5927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"135'882","reported_seq":4533,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311122+0000","last_change":"2024-09-17T21:31:45.498420+0000","last_active":"2024-09-17T21:32:05.311122+0000","last_peered":"2024-09-17T21:32:05.311122+0000","last_clean":"2024-09-17T21:32:05.311122+0000","last_became_active":"2024-09-17T21:29:41.473394+0000","last_became_peered":"2024-09-17T21:29:41.473394+0000","last_unstale":"2024-09-17T21:32:05.311122+0000","last_undegraded":"2024-09-17T21:32:05.311122+0000","last_fullsized":"2024-09-17T21:32:05.311122+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:45:31.022697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002038134,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6008,"num_read_kb":8422,"num_write":1180,"num_write_kb":21760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"134'623","reported_seq":2582,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.180906+0000","last_change":"2024-09-17T21:31:45.419674+0000","last_active":"2024-09-17T21:32:05.180906+0000","last_peered":"2024-09-17T21:32:05.180906+0000","last_clean":"2024-09-17T21:32:05.180906+0000","last_became_active":"2024-09-17T21:29:41.471342+0000","last_became_peered":"2024-09-17T21:29:41.471342+0000","last_unstale":"2024-09-17T21:32:05.180906+0000","last_undegraded":"2024-09-17T21:32:05.180906+0000","last_fullsized":"2024-09-17T21:32:05.180906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:23:05.457286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044458799999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3239,"num_read_kb":14667,"num_write":843,"num_write_kb":8817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"136'418","reported_seq":4857,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310899+0000","last_change":"2024-09-17T21:31:45.497400+0000","last_active":"2024-09-17T21:32:05.310899+0000","last_peered":"2024-09-17T21:32:05.310899+0000","last_clean":"2024-09-17T21:32:05.310899+0000","last_became_active":"2024-09-17T21:29:41.473252+0000","last_became_peered":"2024-09-17T21:29:41.473252+0000","last_unstale":"2024-09-17T21:32:05.310899+0000","last_undegraded":"2024-09-17T21:32:05.310899+0000","last_fullsized":"2024-09-17T21:32:05.310899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:36:31.575810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013588999999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4864,"num_read_kb":34676,"num_write":382,"num_write_kb":15080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"136'375","reported_seq":2283,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248376+0000","last_change":"2024-09-17T21:31:45.430386+0000","last_active":"2024-09-17T21:32:05.248376+0000","last_peered":"2024-09-17T21:32:05.248376+0000","last_clean":"2024-09-17T21:32:05.248376+0000","last_became_active":"2024-09-17T21:29:41.477019+0000","last_became_peered":"2024-09-17T21:29:41.477019+0000","last_unstale":"2024-09-17T21:32:05.248376+0000","last_undegraded":"2024-09-17T21:32:05.248376+0000","last_fullsized":"2024-09-17T21:32:05.248376+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:07:16.797282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094423349999999996,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":1879,"num_read_kb":5231,"num_write":351,"num_write_kb":2424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"135'440","reported_seq":4067,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310943+0000","last_change":"2024-09-17T21:31:45.497003+0000","last_active":"2024-09-17T21:32:05.310943+0000","last_peered":"2024-09-17T21:32:05.310943+0000","last_clean":"2024-09-17T21:32:05.310943+0000","last_became_active":"2024-09-17T21:29:41.473182+0000","last_became_peered":"2024-09-17T21:29:41.473182+0000","last_unstale":"2024-09-17T21:32:05.310943+0000","last_undegraded":"2024-09-17T21:32:05.310943+0000","last_fullsized":"2024-09-17T21:32:05.310943+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:55:27.817705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092929600000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3686,"num_read_kb":22454,"num_write":375,"num_write_kb":9447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":656,"num_object_clones":0,"num_object_copies":1312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":656,"num_whiteouts":652,"num_read":159356,"num_read_kb":893959,"num_write":21643,"num_write_kb":452621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1205,"num_bytes_recovered":117790825,"num_keys_recovered":327,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10476,"internal_metadata":0},"log_size":19097,"ondisk_log_size":19097,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509476,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55256,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316584,"statfs":{"total":96636764160,"available":96580182016,"internally_reserved":0,"allocated":1900544,"data_stored":1469370,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28808,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343032,"statfs":{"total":96636764160,"available":96607264768,"internally_reserved":0,"allocated":1818624,"data_stored":867376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41316,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330524,"statfs":{"total":96636764160,"available":96594456576,"internally_reserved":0,"allocated":1847296,"data_stored":879021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509476,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59368,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312472,"statfs":{"total":96636764160,"available":96575971328,"internally_reserved":0,"allocated":1847296,"data_stored":879021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55192,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316648,"statfs":{"total":96636764160,"available":96580247552,"internally_reserved":0,"allocated":1830912,"data_stored":873189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509477,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41316,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330524,"statfs":{"total":96636764160,"available":96594456576,"internally_reserved":0,"allocated":1843200,"data_stored":879002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":63,"seq":270582939670,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42916,"kb_used_data":1808,"kb_used_omap":20,"kb_used_meta":41067,"kb_avail":94328924,"statfs":{"total":96636764160,"available":96592818176,"internally_reserved":0,"allocated":1851392,"data_stored":879040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20925,"internal_metadata":42053187},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55228,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316612,"statfs":{"total":96636764160,"available":96580210688,"internally_reserved":0,"allocated":1875968,"data_stored":1457744,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10476,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:32:10.529 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T21:32:10.530 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:32:10.816 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:32:10.817 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-17T21:32:10.829 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-17T21:32:08.887764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":660,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":660,"num_whiteouts":652,"num_read":159422,"num_read_kb":894014,"num_write":21746,"num_write_kb":454251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1208,"num_bytes_recovered":117790844,"num_keys_recovered":327,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19172,"ondisk_log_size":19172,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379400,"kb_used_data":14468,"kb_used_omap":31,"kb_used_meta":364704,"kb_avail":754595320,"statfs":{"total":773094113280,"available":772705607680,"internally_reserved":0,"allocated":14815232,"data_stored":8183763,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":32080,"internal_metadata":373457584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.202366"},"pg_stats":[{"pgid":"3.1f","version":"136'809","reported_seq":5134,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248455+0000","last_change":"2024-09-17T21:31:45.430953+0000","last_active":"2024-09-17T21:32:05.248455+0000","last_peered":"2024-09-17T21:32:05.248455+0000","last_clean":"2024-09-17T21:32:05.248455+0000","last_became_active":"2024-09-17T21:31:15.457506+0000","last_became_peered":"2024-09-17T21:31:15.457506+0000","last_unstale":"2024-09-17T21:32:05.248455+0000","last_undegraded":"2024-09-17T21:32:05.248455+0000","last_fullsized":"2024-09-17T21:32:05.248455+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:08.923954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098455079999999993,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6585,"num_read_kb":63347,"num_write":1151,"num_write_kb":32903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":27321072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"136'562","reported_seq":4234,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125356+0000","last_change":"2024-09-17T21:31:45.418470+0000","last_active":"2024-09-17T21:32:05.125356+0000","last_peered":"2024-09-17T21:32:05.125356+0000","last_clean":"2024-09-17T21:32:05.125356+0000","last_became_active":"2024-09-17T21:29:41.472161+0000","last_became_peered":"2024-09-17T21:29:41.472161+0000","last_unstale":"2024-09-17T21:32:05.125356+0000","last_undegraded":"2024-09-17T21:32:05.125356+0000","last_fullsized":"2024-09-17T21:32:05.125356+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:04:47.812939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079279,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3917,"num_read_kb":31515,"num_write":544,"num_write_kb":11792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"135'768","reported_seq":5402,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704245+0000","last_change":"2024-09-17T21:31:45.436342+0000","last_active":"2024-09-17T21:32:04.704245+0000","last_peered":"2024-09-17T21:32:04.704245+0000","last_clean":"2024-09-17T21:32:04.704245+0000","last_became_active":"2024-09-17T21:31:14.449672+0000","last_became_peered":"2024-09-17T21:31:14.449672+0000","last_unstale":"2024-09-17T21:32:04.704245+0000","last_undegraded":"2024-09-17T21:32:04.704245+0000","last_fullsized":"2024-09-17T21:32:04.704245+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:48:24.256603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010859089999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6612,"num_read_kb":51732,"num_write":893,"num_write_kb":18876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":15268754,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"136'433","reported_seq":2685,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704042+0000","last_change":"2024-09-17T21:31:45.435673+0000","last_active":"2024-09-17T21:32:04.704042+0000","last_peered":"2024-09-17T21:32:04.704042+0000","last_clean":"2024-09-17T21:32:04.704042+0000","last_became_active":"2024-09-17T21:31:14.451037+0000","last_became_peered":"2024-09-17T21:31:14.451037+0000","last_unstale":"2024-09-17T21:32:04.704042+0000","last_undegraded":"2024-09-17T21:32:04.704042+0000","last_fullsized":"2024-09-17T21:32:04.704042+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:41:20.330623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068460700000000005,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":2712,"num_read_kb":12910,"num_write":456,"num_write_kb":9120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":604511,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"136'589","reported_seq":8494,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248591+0000","last_change":"2024-09-17T21:31:45.430352+0000","last_active":"2024-09-17T21:32:05.248591+0000","last_peered":"2024-09-17T21:32:05.248591+0000","last_clean":"2024-09-17T21:32:05.248591+0000","last_became_active":"2024-09-17T21:29:41.476041+0000","last_became_peered":"2024-09-17T21:29:41.476041+0000","last_unstale":"2024-09-17T21:32:05.248591+0000","last_undegraded":"2024-09-17T21:32:05.248591+0000","last_fullsized":"2024-09-17T21:32:05.248591+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:45:15.271044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0092951189999999993,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8770,"num_read_kb":53504,"num_write":580,"num_write_kb":24876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"136'630","reported_seq":6573,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699780+0000","last_change":"2024-09-17T21:31:45.425285+0000","last_active":"2024-09-17T21:32:04.699780+0000","last_peered":"2024-09-17T21:32:04.699780+0000","last_clean":"2024-09-17T21:32:04.699780+0000","last_became_active":"2024-09-17T21:30:28.747031+0000","last_became_peered":"2024-09-17T21:30:28.747031+0000","last_unstale":"2024-09-17T21:32:04.699780+0000","last_undegraded":"2024-09-17T21:32:04.699780+0000","last_fullsized":"2024-09-17T21:32:04.699780+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:26.185752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00134892,"stat_sum":{"num_bytes":19,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":7065,"num_read_kb":21661,"num_write":665,"num_write_kb":6726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":235969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"135'424","reported_seq":4765,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700031+0000","last_change":"2024-09-17T21:31:45.427269+0000","last_active":"2024-09-17T21:32:04.700031+0000","last_peered":"2024-09-17T21:32:04.700031+0000","last_clean":"2024-09-17T21:32:04.700031+0000","last_became_active":"2024-09-17T21:30:28.745634+0000","last_became_peered":"2024-09-17T21:30:28.745634+0000","last_unstale":"2024-09-17T21:32:04.700031+0000","last_undegraded":"2024-09-17T21:32:04.700031+0000","last_fullsized":"2024-09-17T21:32:04.700031+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:58.579935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001320393,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4551,"num_read_kb":56994,"num_write":426,"num_write_kb":45225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":36017677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"136'512","reported_seq":4537,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125260+0000","last_change":"2024-09-17T21:31:45.420094+0000","last_active":"2024-09-17T21:32:05.125260+0000","last_peered":"2024-09-17T21:32:05.125260+0000","last_clean":"2024-09-17T21:32:05.125260+0000","last_became_active":"2024-09-17T21:29:41.472468+0000","last_became_peered":"2024-09-17T21:29:41.472468+0000","last_unstale":"2024-09-17T21:32:05.125260+0000","last_undegraded":"2024-09-17T21:32:05.125260+0000","last_fullsized":"2024-09-17T21:32:05.125260+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:12:23.877166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002623551,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4791,"num_read_kb":27686,"num_write":500,"num_write_kb":13101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'814","reported_seq":3250,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248533+0000","last_change":"2024-09-17T21:31:45.431351+0000","last_active":"2024-09-17T21:32:05.248533+0000","last_peered":"2024-09-17T21:32:05.248533+0000","last_clean":"2024-09-17T21:32:05.248533+0000","last_became_active":"2024-09-17T21:31:15.456240+0000","last_became_peered":"2024-09-17T21:31:15.456240+0000","last_unstale":"2024-09-17T21:32:05.248533+0000","last_undegraded":"2024-09-17T21:32:05.248533+0000","last_fullsized":"2024-09-17T21:32:05.248533+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:31:41.549775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010220406,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4952,"num_read_kb":22844,"num_write":955,"num_write_kb":30092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":9394579,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"136'1106","reported_seq":3090,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704029+0000","last_change":"2024-09-17T21:31:45.435544+0000","last_active":"2024-09-17T21:32:04.704029+0000","last_peered":"2024-09-17T21:32:04.704029+0000","last_clean":"2024-09-17T21:32:04.704029+0000","last_became_active":"2024-09-17T21:31:15.456612+0000","last_became_peered":"2024-09-17T21:31:15.456612+0000","last_unstale":"2024-09-17T21:32:04.704029+0000","last_undegraded":"2024-09-17T21:32:04.704029+0000","last_fullsized":"2024-09-17T21:32:04.704029+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":1106,"log_dups_size":0,"ondisk_log_size":1106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:12:11.830676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051663100000000003,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":5737,"num_read_kb":20891,"num_write":1670,"num_write_kb":12347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3063082,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"135'512","reported_seq":3188,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311120+0000","last_change":"2024-09-17T21:31:45.497882+0000","last_active":"2024-09-17T21:32:05.311120+0000","last_peered":"2024-09-17T21:32:05.311120+0000","last_clean":"2024-09-17T21:32:05.311120+0000","last_became_active":"2024-09-17T21:29:41.478346+0000","last_became_peered":"2024-09-17T21:29:41.478346+0000","last_unstale":"2024-09-17T21:32:05.311120+0000","last_undegraded":"2024-09-17T21:32:05.311120+0000","last_fullsized":"2024-09-17T21:32:05.311120+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:58.424829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017578019999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3628,"num_read_kb":9299,"num_write":426,"num_write_kb":8355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"135'297","reported_seq":2884,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699913+0000","last_change":"2024-09-17T21:31:45.424549+0000","last_active":"2024-09-17T21:32:04.699913+0000","last_peered":"2024-09-17T21:32:04.699913+0000","last_clean":"2024-09-17T21:32:04.699913+0000","last_became_active":"2024-09-17T21:29:41.477611+0000","last_became_peered":"2024-09-17T21:29:41.477611+0000","last_unstale":"2024-09-17T21:32:04.699913+0000","last_undegraded":"2024-09-17T21:32:04.699913+0000","last_fullsized":"2024-09-17T21:32:04.699913+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:59:07.448045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064521399999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2154,"num_read_kb":14079,"num_write":352,"num_write_kb":12746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":280,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.180954+0000","last_change":"2024-09-17T21:29:35.002656+0000","last_active":"2024-09-17T21:32:05.180954+0000","last_peered":"2024-09-17T21:32:05.180954+0000","last_clean":"2024-09-17T21:32:05.180954+0000","last_became_active":"2024-09-17T21:29:32.971677+0000","last_became_peered":"2024-09-17T21:29:32.971677+0000","last_unstale":"2024-09-17T21:32:05.180954+0000","last_undegraded":"2024-09-17T21:32:05.180954+0000","last_fullsized":"2024-09-17T21:32:05.180954+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:53.781657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041121300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"136'848","reported_seq":4463,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248518+0000","last_change":"2024-09-17T21:31:45.431468+0000","last_active":"2024-09-17T21:32:05.248518+0000","last_peered":"2024-09-17T21:32:05.248518+0000","last_clean":"2024-09-17T21:32:05.248518+0000","last_became_active":"2024-09-17T21:30:31.755039+0000","last_became_peered":"2024-09-17T21:30:31.755039+0000","last_unstale":"2024-09-17T21:32:05.248518+0000","last_undegraded":"2024-09-17T21:32:05.248518+0000","last_fullsized":"2024-09-17T21:32:05.248518+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.621804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01026987,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6950,"num_read_kb":18293,"num_write":963,"num_write_kb":8597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":448871,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":213,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700243+0000","last_change":"2024-09-17T21:30:28.745759+0000","last_active":"2024-09-17T21:32:04.700243+0000","last_peered":"2024-09-17T21:32:04.700243+0000","last_clean":"2024-09-17T21:32:04.700243+0000","last_became_active":"2024-09-17T21:30:28.744742+0000","last_became_peered":"2024-09-17T21:30:28.744742+0000","last_unstale":"2024-09-17T21:32:04.700243+0000","last_undegraded":"2024-09-17T21:32:04.700243+0000","last_fullsized":"2024-09-17T21:32:04.700243+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:37:02.014566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"134'592","reported_seq":2794,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125373+0000","last_change":"2024-09-17T21:31:45.420595+0000","last_active":"2024-09-17T21:32:05.125373+0000","last_peered":"2024-09-17T21:32:05.125373+0000","last_clean":"2024-09-17T21:32:05.125373+0000","last_became_active":"2024-09-17T21:29:41.472095+0000","last_became_peered":"2024-09-17T21:29:41.472095+0000","last_unstale":"2024-09-17T21:32:05.125373+0000","last_undegraded":"2024-09-17T21:32:05.125373+0000","last_fullsized":"2024-09-17T21:32:05.125373+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:07:43.653815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028983989999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":3726,"num_read_kb":13082,"num_write":700,"num_write_kb":8995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":282,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311127+0000","last_change":"2024-09-17T21:29:35.078854+0000","last_active":"2024-09-17T21:32:05.311127+0000","last_peered":"2024-09-17T21:32:05.311127+0000","last_clean":"2024-09-17T21:32:05.311127+0000","last_became_active":"2024-09-17T21:29:33.071665+0000","last_became_peered":"2024-09-17T21:29:33.071665+0000","last_unstale":"2024-09-17T21:32:05.311127+0000","last_undegraded":"2024-09-17T21:32:05.311127+0000","last_fullsized":"2024-09-17T21:32:05.311127+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.415687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047969899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"134'805","reported_seq":4875,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700392+0000","last_change":"2024-09-17T21:31:45.426864+0000","last_active":"2024-09-17T21:32:04.700392+0000","last_peered":"2024-09-17T21:32:04.700392+0000","last_clean":"2024-09-17T21:32:04.700392+0000","last_became_active":"2024-09-17T21:30:31.752750+0000","last_became_peered":"2024-09-17T21:30:31.752750+0000","last_unstale":"2024-09-17T21:32:04.700392+0000","last_undegraded":"2024-09-17T21:32:04.700392+0000","last_fullsized":"2024-09-17T21:32:04.700392+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:27:53.025416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094011400000000003,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6270,"num_read_kb":32698,"num_write":1050,"num_write_kb":17424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3231045,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":213,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700238+0000","last_change":"2024-09-17T21:30:28.748319+0000","last_active":"2024-09-17T21:32:04.700238+0000","last_peered":"2024-09-17T21:32:04.700238+0000","last_clean":"2024-09-17T21:32:04.700238+0000","last_became_active":"2024-09-17T21:30:28.747728+0000","last_became_peered":"2024-09-17T21:30:28.747728+0000","last_unstale":"2024-09-17T21:32:04.700238+0000","last_undegraded":"2024-09-17T21:32:04.700238+0000","last_fullsized":"2024-09-17T21:32:04.700238+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:58:18.954005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"135'502","reported_seq":4854,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704234+0000","last_change":"2024-09-17T21:31:45.436069+0000","last_active":"2024-09-17T21:32:04.704234+0000","last_peered":"2024-09-17T21:32:04.704234+0000","last_clean":"2024-09-17T21:32:04.704234+0000","last_became_active":"2024-09-17T21:31:14.455957+0000","last_became_peered":"2024-09-17T21:31:14.455957+0000","last_unstale":"2024-09-17T21:32:04.704234+0000","last_undegraded":"2024-09-17T21:32:04.704234+0000","last_fullsized":"2024-09-17T21:32:04.704234+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:43:55.461066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087537200000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4988,"num_read_kb":31633,"num_write":558,"num_write_kb":9789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":1545182,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"136'856","reported_seq":5026,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699955+0000","last_change":"2024-09-17T21:31:45.424972+0000","last_active":"2024-09-17T21:32:04.699955+0000","last_peered":"2024-09-17T21:32:04.699955+0000","last_clean":"2024-09-17T21:32:04.699955+0000","last_became_active":"2024-09-17T21:29:41.475273+0000","last_became_peered":"2024-09-17T21:29:41.475273+0000","last_unstale":"2024-09-17T21:32:04.699955+0000","last_undegraded":"2024-09-17T21:32:04.699955+0000","last_fullsized":"2024-09-17T21:32:04.699955+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:24:39.329469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001059431,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":6333,"num_read_kb":18110,"num_write":998,"num_write_kb":6659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":116,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.703964+0000","last_change":"2024-09-17T21:31:11.424633+0000","last_active":"2024-09-17T21:32:04.703964+0000","last_peered":"2024-09-17T21:32:04.703964+0000","last_clean":"2024-09-17T21:32:04.703964+0000","last_became_active":"2024-09-17T21:31:11.421737+0000","last_became_peered":"2024-09-17T21:31:11.421737+0000","last_unstale":"2024-09-17T21:32:04.703964+0000","last_undegraded":"2024-09-17T21:32:04.703964+0000","last_fullsized":"2024-09-17T21:32:04.703964+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:07:58.025026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"135'413","reported_seq":2979,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700393+0000","last_change":"2024-09-17T21:31:45.426730+0000","last_active":"2024-09-17T21:32:04.700393+0000","last_peered":"2024-09-17T21:32:04.700393+0000","last_clean":"2024-09-17T21:32:04.700393+0000","last_became_active":"2024-09-17T21:30:28.746690+0000","last_became_peered":"2024-09-17T21:30:28.746690+0000","last_unstale":"2024-09-17T21:32:04.700393+0000","last_undegraded":"2024-09-17T21:32:04.700393+0000","last_fullsized":"2024-09-17T21:32:04.700393+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:53:30.001882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090442500000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3369,"num_read_kb":13331,"num_write":389,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1513,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":241,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700230+0000","last_change":"2024-09-17T21:31:53.979063+0000","last_active":"2024-09-17T21:32:04.700230+0000","last_peered":"2024-09-17T21:32:04.700230+0000","last_clean":"2024-09-17T21:32:04.700230+0000","last_became_active":"2024-09-17T21:31:53.978829+0000","last_became_peered":"2024-09-17T21:31:53.978829+0000","last_unstale":"2024-09-17T21:32:04.700230+0000","last_undegraded":"2024-09-17T21:32:04.700230+0000","last_fullsized":"2024-09-17T21:32:04.700230+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:52.874988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"135'431","reported_seq":3957,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248367+0000","last_change":"2024-09-17T21:31:45.430779+0000","last_active":"2024-09-17T21:32:05.248367+0000","last_peered":"2024-09-17T21:32:05.248367+0000","last_clean":"2024-09-17T21:32:05.248367+0000","last_became_active":"2024-09-17T21:29:41.473964+0000","last_became_peered":"2024-09-17T21:29:41.473964+0000","last_unstale":"2024-09-17T21:32:05.248367+0000","last_undegraded":"2024-09-17T21:32:05.248367+0000","last_fullsized":"2024-09-17T21:32:05.248367+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:45.162957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097061290000000008,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3802,"num_read_kb":64522,"num_write":578,"num_write_kb":39811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":291,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310779+0000","last_change":"2024-09-17T21:30:28.746557+0000","last_active":"2024-09-17T21:32:05.310779+0000","last_peered":"2024-09-17T21:32:05.310779+0000","last_clean":"2024-09-17T21:32:05.310779+0000","last_became_active":"2024-09-17T21:30:28.746261+0000","last_became_peered":"2024-09-17T21:30:28.746261+0000","last_unstale":"2024-09-17T21:32:05.310779+0000","last_undegraded":"2024-09-17T21:32:05.310779+0000","last_fullsized":"2024-09-17T21:32:05.310779+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:42.591689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044115199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"135'333","reported_seq":4194,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125209+0000","last_change":"2024-09-17T21:31:45.418657+0000","last_active":"2024-09-17T21:32:05.125209+0000","last_peered":"2024-09-17T21:32:05.125209+0000","last_clean":"2024-09-17T21:32:05.125209+0000","last_became_active":"2024-09-17T21:31:14.459936+0000","last_became_peered":"2024-09-17T21:31:14.459936+0000","last_unstale":"2024-09-17T21:32:05.125209+0000","last_undegraded":"2024-09-17T21:32:05.125209+0000","last_fullsized":"2024-09-17T21:32:05.125209+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:35.279004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012333649999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":3924,"num_read_kb":72696,"num_write":302,"num_write_kb":18067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":13048770,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":116,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.703960+0000","last_change":"2024-09-17T21:31:11.427558+0000","last_active":"2024-09-17T21:32:04.703960+0000","last_peered":"2024-09-17T21:32:04.703960+0000","last_clean":"2024-09-17T21:32:04.703960+0000","last_became_active":"2024-09-17T21:31:11.422076+0000","last_became_peered":"2024-09-17T21:31:11.422076+0000","last_unstale":"2024-09-17T21:32:04.703960+0000","last_undegraded":"2024-09-17T21:32:04.703960+0000","last_fullsized":"2024-09-17T21:32:04.703960+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:37:14.137712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":373,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310979+0000","last_change":"2024-09-17T21:29:28.554672+0000","last_active":"2024-09-17T21:32:05.310979+0000","last_peered":"2024-09-17T21:32:05.310979+0000","last_clean":"2024-09-17T21:32:05.310979+0000","last_became_active":"2024-09-17T21:29:28.553733+0000","last_became_peered":"2024-09-17T21:29:28.553733+0000","last_unstale":"2024-09-17T21:32:05.310979+0000","last_undegraded":"2024-09-17T21:32:05.310979+0000","last_fullsized":"2024-09-17T21:32:05.310979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_clean_scrub_stamp":"2024-09-17T21:29:26.925225+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:36:42.585477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"135'712","reported_seq":4573,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125288+0000","last_change":"2024-09-17T21:31:45.420733+0000","last_active":"2024-09-17T21:32:05.125288+0000","last_peered":"2024-09-17T21:32:05.125288+0000","last_clean":"2024-09-17T21:32:05.125288+0000","last_became_active":"2024-09-17T21:30:31.753839+0000","last_became_peered":"2024-09-17T21:30:31.753839+0000","last_unstale":"2024-09-17T21:32:05.125288+0000","last_undegraded":"2024-09-17T21:32:05.125288+0000","last_fullsized":"2024-09-17T21:32:05.125288+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:57.420241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030177569999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5120,"num_read_kb":35075,"num_write":882,"num_write_kb":14888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2410412,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"134'412","reported_seq":3583,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699862+0000","last_change":"2024-09-17T21:31:45.424510+0000","last_active":"2024-09-17T21:32:04.699862+0000","last_peered":"2024-09-17T21:32:04.699862+0000","last_clean":"2024-09-17T21:32:04.699862+0000","last_became_active":"2024-09-17T21:29:41.477965+0000","last_became_peered":"2024-09-17T21:29:41.477965+0000","last_unstale":"2024-09-17T21:32:04.699862+0000","last_undegraded":"2024-09-17T21:32:04.699862+0000","last_fullsized":"2024-09-17T21:32:04.699862+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:24:25.503913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062142999999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3781,"num_read_kb":28832,"num_write":427,"num_write_kb":12788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"136'595","reported_seq":4077,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.180946+0000","last_change":"2024-09-17T21:31:45.419838+0000","last_active":"2024-09-17T21:32:05.180946+0000","last_peered":"2024-09-17T21:32:05.180946+0000","last_clean":"2024-09-17T21:32:05.180946+0000","last_became_active":"2024-09-17T21:29:41.473420+0000","last_became_peered":"2024-09-17T21:29:41.473420+0000","last_unstale":"2024-09-17T21:32:05.180946+0000","last_undegraded":"2024-09-17T21:32:05.180946+0000","last_fullsized":"2024-09-17T21:32:05.180946+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:33:22.091302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058409800000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":4583,"num_read_kb":15568,"num_write":610,"num_write_kb":6172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"135'541","reported_seq":7208,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125233+0000","last_change":"2024-09-17T21:31:45.420089+0000","last_active":"2024-09-17T21:32:05.125233+0000","last_peered":"2024-09-17T21:32:05.125233+0000","last_clean":"2024-09-17T21:32:05.125233+0000","last_became_active":"2024-09-17T21:29:41.472354+0000","last_became_peered":"2024-09-17T21:29:41.472354+0000","last_unstale":"2024-09-17T21:32:05.125233+0000","last_undegraded":"2024-09-17T21:32:05.125233+0000","last_fullsized":"2024-09-17T21:32:05.125233+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:30.670868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025676010000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7686,"num_read_kb":30319,"num_write":462,"num_write_kb":3994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"135'661","reported_seq":4876,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.703998+0000","last_change":"2024-09-17T21:31:45.435964+0000","last_active":"2024-09-17T21:32:04.703998+0000","last_peered":"2024-09-17T21:32:04.703998+0000","last_clean":"2024-09-17T21:32:04.703998+0000","last_became_active":"2024-09-17T21:31:15.457822+0000","last_became_peered":"2024-09-17T21:31:15.457822+0000","last_unstale":"2024-09-17T21:32:04.703998+0000","last_undegraded":"2024-09-17T21:32:04.703998+0000","last_fullsized":"2024-09-17T21:32:04.703998+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:45:34.971106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086872700000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7278,"num_read_kb":17574,"num_write":576,"num_write_kb":5172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":2355433,"num_keys_recovered":58,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"135'622","reported_seq":4516,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310922+0000","last_change":"2024-09-17T21:31:45.497982+0000","last_active":"2024-09-17T21:32:05.310922+0000","last_peered":"2024-09-17T21:32:05.310922+0000","last_clean":"2024-09-17T21:32:05.310922+0000","last_became_active":"2024-09-17T21:31:14.457648+0000","last_became_peered":"2024-09-17T21:31:14.457648+0000","last_unstale":"2024-09-17T21:32:05.310922+0000","last_undegraded":"2024-09-17T21:32:05.310922+0000","last_fullsized":"2024-09-17T21:32:05.310922+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:10:28.622328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016040850000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5354,"num_read_kb":12719,"num_write":707,"num_write_kb":4578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2843955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"136'580","reported_seq":4386,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311131+0000","last_change":"2024-09-17T21:31:45.497414+0000","last_active":"2024-09-17T21:32:05.311131+0000","last_peered":"2024-09-17T21:32:05.311131+0000","last_clean":"2024-09-17T21:32:05.311131+0000","last_became_active":"2024-09-17T21:29:41.473736+0000","last_became_peered":"2024-09-17T21:29:41.473736+0000","last_unstale":"2024-09-17T21:32:05.311131+0000","last_undegraded":"2024-09-17T21:32:05.311131+0000","last_fullsized":"2024-09-17T21:32:05.311131+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:12:54.605441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013374140000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5042,"num_read_kb":17595,"num_write":692,"num_write_kb":5927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"135'882","reported_seq":4533,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311122+0000","last_change":"2024-09-17T21:31:45.498420+0000","last_active":"2024-09-17T21:32:05.311122+0000","last_peered":"2024-09-17T21:32:05.311122+0000","last_clean":"2024-09-17T21:32:05.311122+0000","last_became_active":"2024-09-17T21:29:41.473394+0000","last_became_peered":"2024-09-17T21:29:41.473394+0000","last_unstale":"2024-09-17T21:32:05.311122+0000","last_undegraded":"2024-09-17T21:32:05.311122+0000","last_fullsized":"2024-09-17T21:32:05.311122+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:45:31.022697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002038134,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6008,"num_read_kb":8422,"num_write":1180,"num_write_kb":21760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"134'623","reported_seq":2582,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.180906+0000","last_change":"2024-09-17T21:31:45.419674+0000","last_active":"2024-09-17T21:32:05.180906+0000","last_peered":"2024-09-17T21:32:05.180906+0000","last_clean":"2024-09-17T21:32:05.180906+0000","last_became_active":"2024-09-17T21:29:41.471342+0000","last_became_peered":"2024-09-17T21:29:41.471342+0000","last_unstale":"2024-09-17T21:32:05.180906+0000","last_undegraded":"2024-09-17T21:32:05.180906+0000","last_fullsized":"2024-09-17T21:32:05.180906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:23:05.457286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044458799999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3239,"num_read_kb":14667,"num_write":843,"num_write_kb":8817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"136'418","reported_seq":4857,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310899+0000","last_change":"2024-09-17T21:31:45.497400+0000","last_active":"2024-09-17T21:32:05.310899+0000","last_peered":"2024-09-17T21:32:05.310899+0000","last_clean":"2024-09-17T21:32:05.310899+0000","last_became_active":"2024-09-17T21:29:41.473252+0000","last_became_peered":"2024-09-17T21:29:41.473252+0000","last_unstale":"2024-09-17T21:32:05.310899+0000","last_undegraded":"2024-09-17T21:32:05.310899+0000","last_fullsized":"2024-09-17T21:32:05.310899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:36:31.575810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013588999999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4864,"num_read_kb":34676,"num_write":382,"num_write_kb":15080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"136'375","reported_seq":2283,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248376+0000","last_change":"2024-09-17T21:31:45.430386+0000","last_active":"2024-09-17T21:32:05.248376+0000","last_peered":"2024-09-17T21:32:05.248376+0000","last_clean":"2024-09-17T21:32:05.248376+0000","last_became_active":"2024-09-17T21:29:41.477019+0000","last_became_peered":"2024-09-17T21:29:41.477019+0000","last_unstale":"2024-09-17T21:32:05.248376+0000","last_undegraded":"2024-09-17T21:32:05.248376+0000","last_fullsized":"2024-09-17T21:32:05.248376+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:07:16.797282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094423349999999996,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":1879,"num_read_kb":5231,"num_write":351,"num_write_kb":2424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"135'440","reported_seq":4067,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310943+0000","last_change":"2024-09-17T21:31:45.497003+0000","last_active":"2024-09-17T21:32:05.310943+0000","last_peered":"2024-09-17T21:32:05.310943+0000","last_clean":"2024-09-17T21:32:05.310943+0000","last_became_active":"2024-09-17T21:29:41.473182+0000","last_became_peered":"2024-09-17T21:29:41.473182+0000","last_unstale":"2024-09-17T21:32:05.310943+0000","last_undegraded":"2024-09-17T21:32:05.310943+0000","last_fullsized":"2024-09-17T21:32:05.310943+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:55:27.817705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092929600000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3686,"num_read_kb":22454,"num_write":375,"num_write_kb":9447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":656,"num_object_clones":0,"num_object_copies":1312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":656,"num_whiteouts":652,"num_read":159356,"num_read_kb":893959,"num_write":21643,"num_write_kb":452621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1205,"num_bytes_recovered":117790825,"num_keys_recovered":327,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10476,"internal_metadata":0},"log_size":19097,"ondisk_log_size":19097,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509476,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55256,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316584,"statfs":{"total":96636764160,"available":96580182016,"internally_reserved":0,"allocated":1900544,"data_stored":1469370,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28808,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343032,"statfs":{"total":96636764160,"available":96607264768,"internally_reserved":0,"allocated":1818624,"data_stored":867376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41316,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330524,"statfs":{"total":96636764160,"available":96594456576,"internally_reserved":0,"allocated":1847296,"data_stored":879021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509476,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59368,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312472,"statfs":{"total":96636764160,"available":96575971328,"internally_reserved":0,"allocated":1847296,"data_stored":879021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55192,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316648,"statfs":{"total":96636764160,"available":96580247552,"internally_reserved":0,"allocated":1830912,"data_stored":873189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509477,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41316,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330524,"statfs":{"total":96636764160,"available":96594456576,"internally_reserved":0,"allocated":1843200,"data_stored":879002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":63,"seq":270582939670,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42916,"kb_used_data":1808,"kb_used_omap":20,"kb_used_meta":41067,"kb_avail":94328924,"statfs":{"total":96636764160,"available":96592818176,"internally_reserved":0,"allocated":1851392,"data_stored":879040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20925,"internal_metadata":42053187},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55228,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316612,"statfs":{"total":96636764160,"available":96580210688,"internally_reserved":0,"allocated":1875968,"data_stored":1457744,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10476,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:32:10.831 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T21:32:11.115 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:32:11.115 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":154,"fsid":"f6d3a80c-8a1f-4716-b25d-e97b28890c0d","created":"2024-09-17T21:29:12.606637+0000","modified":"2024-09-17T21:32:04.695621+0000","last_up_change":"2024-09-17T21:30:27.721456+0000","last_in_change":"2024-09-17T21:31:10.410190+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T21:29:26.876763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T21:29:31.078466+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T21:29:39.892418+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":136,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ad067b51-9402-4f69-a0f0-08fbf068dcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6801","nonce":3635367421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6803","nonce":3635367421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6807","nonce":3635367421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3635367421},{"type":"v1","addr":"172.21.15.129:6805","nonce":3635367421}]},"public_addr":"172.21.15.129:6801/3635367421","cluster_addr":"172.21.15.129:6803/3635367421","heartbeat_back_addr":"172.21.15.129:6807/3635367421","heartbeat_front_addr":"172.21.15.129:6805/3635367421","state":["exists","up"]},{"osd":1,"uuid":"30385fc0-ef5f-45f5-a3ec-37e890d4826e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":47,"up_from":63,"up_thru":104,"down_at":48,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":842542464},{"type":"v1","addr":"172.21.15.129:6819","nonce":842542464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":842542464},{"type":"v1","addr":"172.21.15.129:6821","nonce":842542464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":842542464},{"type":"v1","addr":"172.21.15.129:6825","nonce":842542464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":842542464},{"type":"v1","addr":"172.21.15.129:6823","nonce":842542464}]},"public_addr":"172.21.15.129:6819/842542464","cluster_addr":"172.21.15.129:6821/842542464","heartbeat_back_addr":"172.21.15.129:6825/842542464","heartbeat_front_addr":"172.21.15.129:6823/842542464","state":["exists","up"]},{"osd":2,"uuid":"76be9b2b-b63f-402a-9ab5-2564bd671ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6809","nonce":1790706405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6811","nonce":1790706405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6815","nonce":1790706405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1790706405},{"type":"v1","addr":"172.21.15.129:6813","nonce":1790706405}]},"public_addr":"172.21.15.129:6809/1790706405","cluster_addr":"172.21.15.129:6811/1790706405","heartbeat_back_addr":"172.21.15.129:6815/1790706405","heartbeat_front_addr":"172.21.15.129:6813/1790706405","state":["exists","up"]},{"osd":3,"uuid":"d6c34e3b-1ac6-415c-9c56-38ae77a5c9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6827","nonce":393361453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6829","nonce":393361453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6833","nonce":393361453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":393361453},{"type":"v1","addr":"172.21.15.129:6831","nonce":393361453}]},"public_addr":"172.21.15.129:6827/393361453","cluster_addr":"172.21.15.129:6829/393361453","heartbeat_back_addr":"172.21.15.129:6833/393361453","heartbeat_front_addr":"172.21.15.129:6831/393361453","state":["exists","up"]},{"osd":4,"uuid":"7d722e8c-c497-4f0c-9f61-d9826f8c93d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6809","nonce":2337546183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6811","nonce":2337546183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6815","nonce":2337546183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2337546183},{"type":"v1","addr":"172.21.15.171:6813","nonce":2337546183}]},"public_addr":"172.21.15.171:6809/2337546183","cluster_addr":"172.21.15.171:6811/2337546183","heartbeat_back_addr":"172.21.15.171:6815/2337546183","heartbeat_front_addr":"172.21.15.171:6813/2337546183","state":["exists","up"]},{"osd":5,"uuid":"954f801b-3ac9-410a-a375-eb1c533744de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6817","nonce":1510320351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6819","nonce":1510320351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6823","nonce":1510320351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1510320351},{"type":"v1","addr":"172.21.15.171:6821","nonce":1510320351}]},"public_addr":"172.21.15.171:6817/1510320351","cluster_addr":"172.21.15.171:6819/1510320351","heartbeat_back_addr":"172.21.15.171:6823/1510320351","heartbeat_front_addr":"172.21.15.171:6821/1510320351","state":["exists","up"]},{"osd":6,"uuid":"21837515-65a2-4f21-96d6-67cfc4589358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6825","nonce":471043930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6827","nonce":471043930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6831","nonce":471043930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":471043930},{"type":"v1","addr":"172.21.15.171:6829","nonce":471043930}]},"public_addr":"172.21.15.171:6825/471043930","cluster_addr":"172.21.15.171:6827/471043930","heartbeat_back_addr":"172.21.15.171:6831/471043930","heartbeat_front_addr":"172.21.15.171:6829/471043930","state":["exists","up"]},{"osd":7,"uuid":"67af7eea-8803-425d-ae50-30a6ab91a711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6801","nonce":1770619177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6803","nonce":1770619177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6807","nonce":1770619177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1770619177},{"type":"v1","addr":"172.21.15.171:6805","nonce":1770619177}]},"public_addr":"172.21.15.171:6801/1770619177","cluster_addr":"172.21.15.171:6803/1770619177","heartbeat_back_addr":"172.21.15.171:6807/1770619177","heartbeat_front_addr":"172.21.15.171:6805/1770619177","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:21.445879+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.240967+0000","dead_epoch":48},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.039097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.372014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.049543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.009384+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.048598+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:29:22.046345+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:32:12.128 INFO:tasks.ceph:Scrubbing osd.0 2024-09-17T21:32:12.129 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:32:12.265 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:32:12.266 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:32:12.266 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:32:12.277 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-17T21:32:12.563 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 0 to deep-scrub 2024-09-17T21:32:12.577 INFO:tasks.ceph:Scrubbing osd.1 2024-09-17T21:32:12.577 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:32:12.700 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:32:12.700 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:32:12.700 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:32:12.711 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-17T21:32:12.998 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 1 to deep-scrub 2024-09-17T21:32:13.011 INFO:tasks.ceph:Scrubbing osd.2 2024-09-17T21:32:13.011 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:32:13.134 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:32:13.134 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:32:13.134 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:32:13.145 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-17T21:32:13.431 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 2 to deep-scrub 2024-09-17T21:32:13.444 INFO:tasks.ceph:Scrubbing osd.3 2024-09-17T21:32:13.444 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:32:13.567 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:32:13.567 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:32:13.567 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:32:13.578 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-17T21:32:13.864 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 3 to deep-scrub 2024-09-17T21:32:13.877 INFO:tasks.ceph:Scrubbing osd.4 2024-09-17T21:32:13.878 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:32:14.001 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:32:14.001 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:32:14.001 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:32:14.012 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-17T21:32:14.303 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 4 to deep-scrub 2024-09-17T21:32:14.317 INFO:tasks.ceph:Scrubbing osd.5 2024-09-17T21:32:14.317 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:32:14.441 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:32:14.441 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:32:14.441 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:32:14.451 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-17T21:32:14.738 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 5 to deep-scrub 2024-09-17T21:32:14.752 INFO:tasks.ceph:Scrubbing osd.6 2024-09-17T21:32:14.752 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:32:14.875 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:32:14.875 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:32:14.875 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:32:14.885 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-17T21:32:15.171 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 6 to deep-scrub 2024-09-17T21:32:15.186 INFO:tasks.ceph:Scrubbing osd.7 2024-09-17T21:32:15.186 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:32:15.311 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T21:32:15.312 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:32:15.312 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T21:32:15.322 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-17T21:32:15.607 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 7 to deep-scrub 2024-09-17T21:32:15.620 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:32:15.906 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:32:15.906 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-17T21:32:15.918 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-17T21:32:14.888686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":660,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":660,"num_whiteouts":652,"num_read":159422,"num_read_kb":894014,"num_write":21746,"num_write_kb":454251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1208,"num_bytes_recovered":117790844,"num_keys_recovered":327,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19187,"ondisk_log_size":19187,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379448,"kb_used_data":14528,"kb_used_omap":31,"kb_used_meta":364704,"kb_avail":754595272,"statfs":{"total":773094113280,"available":772705558528,"internally_reserved":0,"allocated":14876672,"data_stored":8212828,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":32080,"internal_metadata":373457584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.190382"},"pg_stats":[{"pgid":"3.1f","version":"136'809","reported_seq":5134,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248455+0000","last_change":"2024-09-17T21:31:45.430953+0000","last_active":"2024-09-17T21:32:05.248455+0000","last_peered":"2024-09-17T21:32:05.248455+0000","last_clean":"2024-09-17T21:32:05.248455+0000","last_became_active":"2024-09-17T21:31:15.457506+0000","last_became_peered":"2024-09-17T21:31:15.457506+0000","last_unstale":"2024-09-17T21:32:05.248455+0000","last_undegraded":"2024-09-17T21:32:05.248455+0000","last_fullsized":"2024-09-17T21:32:05.248455+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:08.923954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098455079999999993,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6585,"num_read_kb":63347,"num_write":1151,"num_write_kb":32903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":27321072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"136'562","reported_seq":4234,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125356+0000","last_change":"2024-09-17T21:31:45.418470+0000","last_active":"2024-09-17T21:32:05.125356+0000","last_peered":"2024-09-17T21:32:05.125356+0000","last_clean":"2024-09-17T21:32:05.125356+0000","last_became_active":"2024-09-17T21:29:41.472161+0000","last_became_peered":"2024-09-17T21:29:41.472161+0000","last_unstale":"2024-09-17T21:32:05.125356+0000","last_undegraded":"2024-09-17T21:32:05.125356+0000","last_fullsized":"2024-09-17T21:32:05.125356+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:04:47.812939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079279,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3917,"num_read_kb":31515,"num_write":544,"num_write_kb":11792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"135'768","reported_seq":5402,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704245+0000","last_change":"2024-09-17T21:31:45.436342+0000","last_active":"2024-09-17T21:32:04.704245+0000","last_peered":"2024-09-17T21:32:04.704245+0000","last_clean":"2024-09-17T21:32:04.704245+0000","last_became_active":"2024-09-17T21:31:14.449672+0000","last_became_peered":"2024-09-17T21:31:14.449672+0000","last_unstale":"2024-09-17T21:32:04.704245+0000","last_undegraded":"2024-09-17T21:32:04.704245+0000","last_fullsized":"2024-09-17T21:32:04.704245+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:48:24.256603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010859089999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6612,"num_read_kb":51732,"num_write":893,"num_write_kb":18876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":15268754,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"136'433","reported_seq":2685,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704042+0000","last_change":"2024-09-17T21:31:45.435673+0000","last_active":"2024-09-17T21:32:04.704042+0000","last_peered":"2024-09-17T21:32:04.704042+0000","last_clean":"2024-09-17T21:32:04.704042+0000","last_became_active":"2024-09-17T21:31:14.451037+0000","last_became_peered":"2024-09-17T21:31:14.451037+0000","last_unstale":"2024-09-17T21:32:04.704042+0000","last_undegraded":"2024-09-17T21:32:04.704042+0000","last_fullsized":"2024-09-17T21:32:04.704042+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:41:20.330623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068460700000000005,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":2712,"num_read_kb":12910,"num_write":456,"num_write_kb":9120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":604511,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"136'589","reported_seq":8495,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248611+0000","last_change":"2024-09-17T21:31:45.430352+0000","last_active":"2024-09-17T21:32:05.248611+0000","last_peered":"2024-09-17T21:32:05.248611+0000","last_clean":"2024-09-17T21:32:05.248611+0000","last_became_active":"2024-09-17T21:29:41.476041+0000","last_became_peered":"2024-09-17T21:29:41.476041+0000","last_unstale":"2024-09-17T21:32:05.248611+0000","last_undegraded":"2024-09-17T21:32:05.248611+0000","last_fullsized":"2024-09-17T21:32:05.248611+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0092951189999999993,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8770,"num_read_kb":53504,"num_write":580,"num_write_kb":24876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"136'630","reported_seq":6574,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699877+0000","last_change":"2024-09-17T21:31:45.425285+0000","last_active":"2024-09-17T21:32:04.699877+0000","last_peered":"2024-09-17T21:32:04.699877+0000","last_clean":"2024-09-17T21:32:04.699877+0000","last_became_active":"2024-09-17T21:30:28.747031+0000","last_became_peered":"2024-09-17T21:30:28.747031+0000","last_unstale":"2024-09-17T21:32:04.699877+0000","last_undegraded":"2024-09-17T21:32:04.699877+0000","last_fullsized":"2024-09-17T21:32:04.699877+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:26.185752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00134892,"stat_sum":{"num_bytes":19,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":7065,"num_read_kb":21661,"num_write":665,"num_write_kb":6726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":235969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"135'424","reported_seq":4765,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700031+0000","last_change":"2024-09-17T21:31:45.427269+0000","last_active":"2024-09-17T21:32:04.700031+0000","last_peered":"2024-09-17T21:32:04.700031+0000","last_clean":"2024-09-17T21:32:04.700031+0000","last_became_active":"2024-09-17T21:30:28.745634+0000","last_became_peered":"2024-09-17T21:30:28.745634+0000","last_unstale":"2024-09-17T21:32:04.700031+0000","last_undegraded":"2024-09-17T21:32:04.700031+0000","last_fullsized":"2024-09-17T21:32:04.700031+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:58.579935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001320393,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4551,"num_read_kb":56994,"num_write":426,"num_write_kb":45225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":36017677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"136'512","reported_seq":4537,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125260+0000","last_change":"2024-09-17T21:31:45.420094+0000","last_active":"2024-09-17T21:32:05.125260+0000","last_peered":"2024-09-17T21:32:05.125260+0000","last_clean":"2024-09-17T21:32:05.125260+0000","last_became_active":"2024-09-17T21:29:41.472468+0000","last_became_peered":"2024-09-17T21:29:41.472468+0000","last_unstale":"2024-09-17T21:32:05.125260+0000","last_undegraded":"2024-09-17T21:32:05.125260+0000","last_fullsized":"2024-09-17T21:32:05.125260+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:12:23.877166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002623551,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4791,"num_read_kb":27686,"num_write":500,"num_write_kb":13101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'814","reported_seq":3251,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248573+0000","last_change":"2024-09-17T21:31:45.431351+0000","last_active":"2024-09-17T21:32:05.248573+0000","last_peered":"2024-09-17T21:32:05.248573+0000","last_clean":"2024-09-17T21:32:05.248573+0000","last_became_active":"2024-09-17T21:31:15.456240+0000","last_became_peered":"2024-09-17T21:31:15.456240+0000","last_unstale":"2024-09-17T21:32:05.248573+0000","last_undegraded":"2024-09-17T21:32:05.248573+0000","last_fullsized":"2024-09-17T21:32:05.248573+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:31:41.549775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010220406,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4952,"num_read_kb":22844,"num_write":955,"num_write_kb":30092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":9394579,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"136'1106","reported_seq":3090,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704029+0000","last_change":"2024-09-17T21:31:45.435544+0000","last_active":"2024-09-17T21:32:04.704029+0000","last_peered":"2024-09-17T21:32:04.704029+0000","last_clean":"2024-09-17T21:32:04.704029+0000","last_became_active":"2024-09-17T21:31:15.456612+0000","last_became_peered":"2024-09-17T21:31:15.456612+0000","last_unstale":"2024-09-17T21:32:04.704029+0000","last_undegraded":"2024-09-17T21:32:04.704029+0000","last_fullsized":"2024-09-17T21:32:04.704029+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":1106,"log_dups_size":0,"ondisk_log_size":1106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:12:11.830676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051663100000000003,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":5737,"num_read_kb":20891,"num_write":1670,"num_write_kb":12347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3063082,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"135'512","reported_seq":3188,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311120+0000","last_change":"2024-09-17T21:31:45.497882+0000","last_active":"2024-09-17T21:32:05.311120+0000","last_peered":"2024-09-17T21:32:05.311120+0000","last_clean":"2024-09-17T21:32:05.311120+0000","last_became_active":"2024-09-17T21:29:41.478346+0000","last_became_peered":"2024-09-17T21:29:41.478346+0000","last_unstale":"2024-09-17T21:32:05.311120+0000","last_undegraded":"2024-09-17T21:32:05.311120+0000","last_fullsized":"2024-09-17T21:32:05.311120+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:58.424829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017578019999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3628,"num_read_kb":9299,"num_write":426,"num_write_kb":8355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"135'297","reported_seq":2884,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699913+0000","last_change":"2024-09-17T21:31:45.424549+0000","last_active":"2024-09-17T21:32:04.699913+0000","last_peered":"2024-09-17T21:32:04.699913+0000","last_clean":"2024-09-17T21:32:04.699913+0000","last_became_active":"2024-09-17T21:29:41.477611+0000","last_became_peered":"2024-09-17T21:29:41.477611+0000","last_unstale":"2024-09-17T21:32:04.699913+0000","last_undegraded":"2024-09-17T21:32:04.699913+0000","last_fullsized":"2024-09-17T21:32:04.699913+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:59:07.448045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064521399999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2154,"num_read_kb":14079,"num_write":352,"num_write_kb":12746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":280,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.180954+0000","last_change":"2024-09-17T21:29:35.002656+0000","last_active":"2024-09-17T21:32:05.180954+0000","last_peered":"2024-09-17T21:32:05.180954+0000","last_clean":"2024-09-17T21:32:05.180954+0000","last_became_active":"2024-09-17T21:29:32.971677+0000","last_became_peered":"2024-09-17T21:29:32.971677+0000","last_unstale":"2024-09-17T21:32:05.180954+0000","last_undegraded":"2024-09-17T21:32:05.180954+0000","last_fullsized":"2024-09-17T21:32:05.180954+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:53.781657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041121300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"154'863","reported_seq":4487,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:13.235036+0000","last_change":"2024-09-17T21:32:13.235036+0000","last_active":"2024-09-17T21:32:13.235036+0000","last_peered":"2024-09-17T21:32:13.235036+0000","last_clean":"2024-09-17T21:32:13.235036+0000","last_became_active":"2024-09-17T21:30:31.755039+0000","last_became_peered":"2024-09-17T21:30:31.755039+0000","last_unstale":"2024-09-17T21:32:13.235036+0000","last_undegraded":"2024-09-17T21:32:13.235036+0000","last_fullsized":"2024-09-17T21:32:13.235036+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'863","last_scrub_stamp":"2024-09-17T21:32:13.234973+0000","last_deep_scrub":"154'863","last_deep_scrub_stamp":"2024-09-17T21:32:13.234973+0000","last_clean_scrub_stamp":"2024-09-17T21:32:13.234973+0000","objects_scrubbed":15,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:09:05.153455+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.01026987,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6950,"num_read_kb":18293,"num_write":963,"num_write_kb":8597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":448871,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":213,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700243+0000","last_change":"2024-09-17T21:30:28.745759+0000","last_active":"2024-09-17T21:32:04.700243+0000","last_peered":"2024-09-17T21:32:04.700243+0000","last_clean":"2024-09-17T21:32:04.700243+0000","last_became_active":"2024-09-17T21:30:28.744742+0000","last_became_peered":"2024-09-17T21:30:28.744742+0000","last_unstale":"2024-09-17T21:32:04.700243+0000","last_undegraded":"2024-09-17T21:32:04.700243+0000","last_fullsized":"2024-09-17T21:32:04.700243+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:37:02.014566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"134'592","reported_seq":2794,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125373+0000","last_change":"2024-09-17T21:31:45.420595+0000","last_active":"2024-09-17T21:32:05.125373+0000","last_peered":"2024-09-17T21:32:05.125373+0000","last_clean":"2024-09-17T21:32:05.125373+0000","last_became_active":"2024-09-17T21:29:41.472095+0000","last_became_peered":"2024-09-17T21:29:41.472095+0000","last_unstale":"2024-09-17T21:32:05.125373+0000","last_undegraded":"2024-09-17T21:32:05.125373+0000","last_fullsized":"2024-09-17T21:32:05.125373+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:07:43.653815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028983989999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":3726,"num_read_kb":13082,"num_write":700,"num_write_kb":8995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":282,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311127+0000","last_change":"2024-09-17T21:29:35.078854+0000","last_active":"2024-09-17T21:32:05.311127+0000","last_peered":"2024-09-17T21:32:05.311127+0000","last_clean":"2024-09-17T21:32:05.311127+0000","last_became_active":"2024-09-17T21:29:33.071665+0000","last_became_peered":"2024-09-17T21:29:33.071665+0000","last_unstale":"2024-09-17T21:32:05.311127+0000","last_undegraded":"2024-09-17T21:32:05.311127+0000","last_fullsized":"2024-09-17T21:32:05.311127+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:17.415687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047969899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"134'805","reported_seq":4875,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700392+0000","last_change":"2024-09-17T21:31:45.426864+0000","last_active":"2024-09-17T21:32:04.700392+0000","last_peered":"2024-09-17T21:32:04.700392+0000","last_clean":"2024-09-17T21:32:04.700392+0000","last_became_active":"2024-09-17T21:30:31.752750+0000","last_became_peered":"2024-09-17T21:30:31.752750+0000","last_unstale":"2024-09-17T21:32:04.700392+0000","last_undegraded":"2024-09-17T21:32:04.700392+0000","last_fullsized":"2024-09-17T21:32:04.700392+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:27:53.025416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094011400000000003,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6270,"num_read_kb":32698,"num_write":1050,"num_write_kb":17424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3231045,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":213,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700238+0000","last_change":"2024-09-17T21:30:28.748319+0000","last_active":"2024-09-17T21:32:04.700238+0000","last_peered":"2024-09-17T21:32:04.700238+0000","last_clean":"2024-09-17T21:32:04.700238+0000","last_became_active":"2024-09-17T21:30:28.747728+0000","last_became_peered":"2024-09-17T21:30:28.747728+0000","last_unstale":"2024-09-17T21:32:04.700238+0000","last_undegraded":"2024-09-17T21:32:04.700238+0000","last_fullsized":"2024-09-17T21:32:04.700238+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:58:18.954005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"135'502","reported_seq":4854,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.704234+0000","last_change":"2024-09-17T21:31:45.436069+0000","last_active":"2024-09-17T21:32:04.704234+0000","last_peered":"2024-09-17T21:32:04.704234+0000","last_clean":"2024-09-17T21:32:04.704234+0000","last_became_active":"2024-09-17T21:31:14.455957+0000","last_became_peered":"2024-09-17T21:31:14.455957+0000","last_unstale":"2024-09-17T21:32:04.704234+0000","last_undegraded":"2024-09-17T21:32:04.704234+0000","last_fullsized":"2024-09-17T21:32:04.704234+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:43:55.461066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087537200000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4988,"num_read_kb":31633,"num_write":558,"num_write_kb":9789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":1545182,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"136'856","reported_seq":5026,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699955+0000","last_change":"2024-09-17T21:31:45.424972+0000","last_active":"2024-09-17T21:32:04.699955+0000","last_peered":"2024-09-17T21:32:04.699955+0000","last_clean":"2024-09-17T21:32:04.699955+0000","last_became_active":"2024-09-17T21:29:41.475273+0000","last_became_peered":"2024-09-17T21:29:41.475273+0000","last_unstale":"2024-09-17T21:32:04.699955+0000","last_undegraded":"2024-09-17T21:32:04.699955+0000","last_fullsized":"2024-09-17T21:32:04.699955+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:24:39.329469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001059431,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":6333,"num_read_kb":18110,"num_write":998,"num_write_kb":6659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":116,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.703964+0000","last_change":"2024-09-17T21:31:11.424633+0000","last_active":"2024-09-17T21:32:04.703964+0000","last_peered":"2024-09-17T21:32:04.703964+0000","last_clean":"2024-09-17T21:32:04.703964+0000","last_became_active":"2024-09-17T21:31:11.421737+0000","last_became_peered":"2024-09-17T21:31:11.421737+0000","last_unstale":"2024-09-17T21:32:04.703964+0000","last_undegraded":"2024-09-17T21:32:04.703964+0000","last_fullsized":"2024-09-17T21:32:04.703964+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:07:58.025026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"135'413","reported_seq":2979,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700393+0000","last_change":"2024-09-17T21:31:45.426730+0000","last_active":"2024-09-17T21:32:04.700393+0000","last_peered":"2024-09-17T21:32:04.700393+0000","last_clean":"2024-09-17T21:32:04.700393+0000","last_became_active":"2024-09-17T21:30:28.746690+0000","last_became_peered":"2024-09-17T21:30:28.746690+0000","last_unstale":"2024-09-17T21:32:04.700393+0000","last_undegraded":"2024-09-17T21:32:04.700393+0000","last_fullsized":"2024-09-17T21:32:04.700393+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:53:30.001882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090442500000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3369,"num_read_kb":13331,"num_write":389,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1513,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":241,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.700230+0000","last_change":"2024-09-17T21:31:53.979063+0000","last_active":"2024-09-17T21:32:04.700230+0000","last_peered":"2024-09-17T21:32:04.700230+0000","last_clean":"2024-09-17T21:32:04.700230+0000","last_became_active":"2024-09-17T21:31:53.978829+0000","last_became_peered":"2024-09-17T21:31:53.978829+0000","last_unstale":"2024-09-17T21:32:04.700230+0000","last_undegraded":"2024-09-17T21:32:04.700230+0000","last_fullsized":"2024-09-17T21:32:04.700230+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:52.874988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"135'431","reported_seq":3957,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248367+0000","last_change":"2024-09-17T21:31:45.430779+0000","last_active":"2024-09-17T21:32:05.248367+0000","last_peered":"2024-09-17T21:32:05.248367+0000","last_clean":"2024-09-17T21:32:05.248367+0000","last_became_active":"2024-09-17T21:29:41.473964+0000","last_became_peered":"2024-09-17T21:29:41.473964+0000","last_unstale":"2024-09-17T21:32:05.248367+0000","last_undegraded":"2024-09-17T21:32:05.248367+0000","last_fullsized":"2024-09-17T21:32:05.248367+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:45.162957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097061290000000008,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3802,"num_read_kb":64522,"num_write":578,"num_write_kb":39811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":291,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310779+0000","last_change":"2024-09-17T21:30:28.746557+0000","last_active":"2024-09-17T21:32:05.310779+0000","last_peered":"2024-09-17T21:32:05.310779+0000","last_clean":"2024-09-17T21:32:05.310779+0000","last_became_active":"2024-09-17T21:30:28.746261+0000","last_became_peered":"2024-09-17T21:30:28.746261+0000","last_unstale":"2024-09-17T21:32:05.310779+0000","last_undegraded":"2024-09-17T21:32:05.310779+0000","last_fullsized":"2024-09-17T21:32:05.310779+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:42.591689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044115199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"135'333","reported_seq":4194,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125209+0000","last_change":"2024-09-17T21:31:45.418657+0000","last_active":"2024-09-17T21:32:05.125209+0000","last_peered":"2024-09-17T21:32:05.125209+0000","last_clean":"2024-09-17T21:32:05.125209+0000","last_became_active":"2024-09-17T21:31:14.459936+0000","last_became_peered":"2024-09-17T21:31:14.459936+0000","last_unstale":"2024-09-17T21:32:05.125209+0000","last_undegraded":"2024-09-17T21:32:05.125209+0000","last_fullsized":"2024-09-17T21:32:05.125209+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:35.279004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012333649999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":3924,"num_read_kb":72696,"num_write":302,"num_write_kb":18067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":13048770,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":116,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.703960+0000","last_change":"2024-09-17T21:31:11.427558+0000","last_active":"2024-09-17T21:32:04.703960+0000","last_peered":"2024-09-17T21:32:04.703960+0000","last_clean":"2024-09-17T21:32:04.703960+0000","last_became_active":"2024-09-17T21:31:11.422076+0000","last_became_peered":"2024-09-17T21:31:11.422076+0000","last_unstale":"2024-09-17T21:32:04.703960+0000","last_undegraded":"2024-09-17T21:32:04.703960+0000","last_fullsized":"2024-09-17T21:32:04.703960+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:31.953070+0000","last_clean_scrub_stamp":"2024-09-17T21:29:31.953070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:37:14.137712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":373,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310979+0000","last_change":"2024-09-17T21:29:28.554672+0000","last_active":"2024-09-17T21:32:05.310979+0000","last_peered":"2024-09-17T21:32:05.310979+0000","last_clean":"2024-09-17T21:32:05.310979+0000","last_became_active":"2024-09-17T21:29:28.553733+0000","last_became_peered":"2024-09-17T21:29:28.553733+0000","last_unstale":"2024-09-17T21:32:05.310979+0000","last_undegraded":"2024-09-17T21:32:05.310979+0000","last_fullsized":"2024-09-17T21:32:05.310979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:26.925225+0000","last_clean_scrub_stamp":"2024-09-17T21:29:26.925225+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:36:42.585477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"135'712","reported_seq":4573,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125288+0000","last_change":"2024-09-17T21:31:45.420733+0000","last_active":"2024-09-17T21:32:05.125288+0000","last_peered":"2024-09-17T21:32:05.125288+0000","last_clean":"2024-09-17T21:32:05.125288+0000","last_became_active":"2024-09-17T21:30:31.753839+0000","last_became_peered":"2024-09-17T21:30:31.753839+0000","last_unstale":"2024-09-17T21:32:05.125288+0000","last_undegraded":"2024-09-17T21:32:05.125288+0000","last_fullsized":"2024-09-17T21:32:05.125288+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:57.420241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030177569999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5120,"num_read_kb":35075,"num_write":882,"num_write_kb":14888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2410412,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"134'412","reported_seq":3583,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.699862+0000","last_change":"2024-09-17T21:31:45.424510+0000","last_active":"2024-09-17T21:32:04.699862+0000","last_peered":"2024-09-17T21:32:04.699862+0000","last_clean":"2024-09-17T21:32:04.699862+0000","last_became_active":"2024-09-17T21:29:41.477965+0000","last_became_peered":"2024-09-17T21:29:41.477965+0000","last_unstale":"2024-09-17T21:32:04.699862+0000","last_undegraded":"2024-09-17T21:32:04.699862+0000","last_fullsized":"2024-09-17T21:32:04.699862+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:24:25.503913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062142999999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3781,"num_read_kb":28832,"num_write":427,"num_write_kb":12788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"136'595","reported_seq":4077,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.180946+0000","last_change":"2024-09-17T21:31:45.419838+0000","last_active":"2024-09-17T21:32:05.180946+0000","last_peered":"2024-09-17T21:32:05.180946+0000","last_clean":"2024-09-17T21:32:05.180946+0000","last_became_active":"2024-09-17T21:29:41.473420+0000","last_became_peered":"2024-09-17T21:29:41.473420+0000","last_unstale":"2024-09-17T21:32:05.180946+0000","last_undegraded":"2024-09-17T21:32:05.180946+0000","last_fullsized":"2024-09-17T21:32:05.180946+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:33:22.091302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058409800000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":4583,"num_read_kb":15568,"num_write":610,"num_write_kb":6172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"135'541","reported_seq":7208,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.125233+0000","last_change":"2024-09-17T21:31:45.420089+0000","last_active":"2024-09-17T21:32:05.125233+0000","last_peered":"2024-09-17T21:32:05.125233+0000","last_clean":"2024-09-17T21:32:05.125233+0000","last_became_active":"2024-09-17T21:29:41.472354+0000","last_became_peered":"2024-09-17T21:29:41.472354+0000","last_unstale":"2024-09-17T21:32:05.125233+0000","last_undegraded":"2024-09-17T21:32:05.125233+0000","last_fullsized":"2024-09-17T21:32:05.125233+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:30.670868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025676010000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7686,"num_read_kb":30319,"num_write":462,"num_write_kb":3994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"135'661","reported_seq":4876,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:04.703998+0000","last_change":"2024-09-17T21:31:45.435964+0000","last_active":"2024-09-17T21:32:04.703998+0000","last_peered":"2024-09-17T21:32:04.703998+0000","last_clean":"2024-09-17T21:32:04.703998+0000","last_became_active":"2024-09-17T21:31:15.457822+0000","last_became_peered":"2024-09-17T21:31:15.457822+0000","last_unstale":"2024-09-17T21:32:04.703998+0000","last_undegraded":"2024-09-17T21:32:04.703998+0000","last_fullsized":"2024-09-17T21:32:04.703998+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:45:34.971106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086872700000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7278,"num_read_kb":17574,"num_write":576,"num_write_kb":5172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":2355433,"num_keys_recovered":58,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"135'622","reported_seq":4516,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310922+0000","last_change":"2024-09-17T21:31:45.497982+0000","last_active":"2024-09-17T21:32:05.310922+0000","last_peered":"2024-09-17T21:32:05.310922+0000","last_clean":"2024-09-17T21:32:05.310922+0000","last_became_active":"2024-09-17T21:31:14.457648+0000","last_became_peered":"2024-09-17T21:31:14.457648+0000","last_unstale":"2024-09-17T21:32:05.310922+0000","last_undegraded":"2024-09-17T21:32:05.310922+0000","last_fullsized":"2024-09-17T21:32:05.310922+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:10:28.622328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016040850000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5354,"num_read_kb":12719,"num_write":707,"num_write_kb":4578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2843955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"136'580","reported_seq":4386,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311131+0000","last_change":"2024-09-17T21:31:45.497414+0000","last_active":"2024-09-17T21:32:05.311131+0000","last_peered":"2024-09-17T21:32:05.311131+0000","last_clean":"2024-09-17T21:32:05.311131+0000","last_became_active":"2024-09-17T21:29:41.473736+0000","last_became_peered":"2024-09-17T21:29:41.473736+0000","last_unstale":"2024-09-17T21:32:05.311131+0000","last_undegraded":"2024-09-17T21:32:05.311131+0000","last_fullsized":"2024-09-17T21:32:05.311131+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:12:54.605441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013374140000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5042,"num_read_kb":17595,"num_write":692,"num_write_kb":5927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"135'882","reported_seq":4533,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.311122+0000","last_change":"2024-09-17T21:31:45.498420+0000","last_active":"2024-09-17T21:32:05.311122+0000","last_peered":"2024-09-17T21:32:05.311122+0000","last_clean":"2024-09-17T21:32:05.311122+0000","last_became_active":"2024-09-17T21:29:41.473394+0000","last_became_peered":"2024-09-17T21:29:41.473394+0000","last_unstale":"2024-09-17T21:32:05.311122+0000","last_undegraded":"2024-09-17T21:32:05.311122+0000","last_fullsized":"2024-09-17T21:32:05.311122+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:45:31.022697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002038134,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6008,"num_read_kb":8422,"num_write":1180,"num_write_kb":21760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"134'623","reported_seq":2582,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.180906+0000","last_change":"2024-09-17T21:31:45.419674+0000","last_active":"2024-09-17T21:32:05.180906+0000","last_peered":"2024-09-17T21:32:05.180906+0000","last_clean":"2024-09-17T21:32:05.180906+0000","last_became_active":"2024-09-17T21:29:41.471342+0000","last_became_peered":"2024-09-17T21:29:41.471342+0000","last_unstale":"2024-09-17T21:32:05.180906+0000","last_undegraded":"2024-09-17T21:32:05.180906+0000","last_fullsized":"2024-09-17T21:32:05.180906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:23:05.457286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044458799999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3239,"num_read_kb":14667,"num_write":843,"num_write_kb":8817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"136'418","reported_seq":4857,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310899+0000","last_change":"2024-09-17T21:31:45.497400+0000","last_active":"2024-09-17T21:32:05.310899+0000","last_peered":"2024-09-17T21:32:05.310899+0000","last_clean":"2024-09-17T21:32:05.310899+0000","last_became_active":"2024-09-17T21:29:41.473252+0000","last_became_peered":"2024-09-17T21:29:41.473252+0000","last_unstale":"2024-09-17T21:32:05.310899+0000","last_undegraded":"2024-09-17T21:32:05.310899+0000","last_fullsized":"2024-09-17T21:32:05.310899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:36:31.575810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013588999999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4864,"num_read_kb":34676,"num_write":382,"num_write_kb":15080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"136'375","reported_seq":2283,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.248376+0000","last_change":"2024-09-17T21:31:45.430386+0000","last_active":"2024-09-17T21:32:05.248376+0000","last_peered":"2024-09-17T21:32:05.248376+0000","last_clean":"2024-09-17T21:32:05.248376+0000","last_became_active":"2024-09-17T21:29:41.477019+0000","last_became_peered":"2024-09-17T21:29:41.477019+0000","last_unstale":"2024-09-17T21:32:05.248376+0000","last_undegraded":"2024-09-17T21:32:05.248376+0000","last_fullsized":"2024-09-17T21:32:05.248376+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:07:16.797282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094423349999999996,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":1879,"num_read_kb":5231,"num_write":351,"num_write_kb":2424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"135'440","reported_seq":4067,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:05.310943+0000","last_change":"2024-09-17T21:31:45.497003+0000","last_active":"2024-09-17T21:32:05.310943+0000","last_peered":"2024-09-17T21:32:05.310943+0000","last_clean":"2024-09-17T21:32:05.310943+0000","last_became_active":"2024-09-17T21:29:41.473182+0000","last_became_peered":"2024-09-17T21:29:41.473182+0000","last_unstale":"2024-09-17T21:32:05.310943+0000","last_undegraded":"2024-09-17T21:32:05.310943+0000","last_fullsized":"2024-09-17T21:32:05.310943+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:29:40.454958+0000","last_clean_scrub_stamp":"2024-09-17T21:29:40.454958+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:55:27.817705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092929600000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3686,"num_read_kb":22454,"num_write":375,"num_write_kb":9447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":656,"num_object_clones":0,"num_object_copies":1312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":656,"num_whiteouts":652,"num_read":159356,"num_read_kb":893959,"num_write":21643,"num_write_kb":452621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1205,"num_bytes_recovered":117790825,"num_keys_recovered":327,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10476,"internal_metadata":0},"log_size":19112,"ondisk_log_size":19112,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509477,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55256,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316584,"statfs":{"total":96636764160,"available":96580182016,"internally_reserved":0,"allocated":1900544,"data_stored":1469370,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509477,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28828,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343012,"statfs":{"total":96636764160,"available":96607244288,"internally_reserved":0,"allocated":1843200,"data_stored":879002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509478,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41316,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330524,"statfs":{"total":96636764160,"available":96594456576,"internally_reserved":0,"allocated":1847296,"data_stored":879021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509477,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59368,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312472,"statfs":{"total":96636764160,"available":96575971328,"internally_reserved":0,"allocated":1847296,"data_stored":879021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509478,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55196,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316644,"statfs":{"total":96636764160,"available":96580243456,"internally_reserved":0,"allocated":1843200,"data_stored":879002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41316,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330524,"statfs":{"total":96636764160,"available":96594456576,"internally_reserved":0,"allocated":1843200,"data_stored":879002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":63,"seq":270582939671,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42916,"kb_used_data":1808,"kb_used_omap":20,"kb_used_meta":41067,"kb_avail":94328924,"statfs":{"total":96636764160,"available":96592818176,"internally_reserved":0,"allocated":1851392,"data_stored":879040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20925,"internal_metadata":42053187},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55252,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316588,"statfs":{"total":96636764160,"available":96580186112,"internally_reserved":0,"allocated":1900544,"data_stored":1469370,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10476,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:32:15.923 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.923 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.923 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.923 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.923 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.923 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.923 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.924 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.924 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.924 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.924 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.924 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.924 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-17T21:29:31.953070+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=31, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.924 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-17T21:29:31.953070+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=31, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.924 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.924 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-17T21:29:31.953070+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=31, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.924 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.925 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-17T21:29:31.953070+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=31, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.925 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.925 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.925 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-17T21:29:31.953070+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=31, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.925 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.925 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-17T21:29:31.953070+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=31, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.925 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.925 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-17T21:29:31.953070+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=31, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.925 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.926 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-17T21:29:31.953070+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=31, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.926 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-17T21:29:26.925225+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=26, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.926 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.926 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.926 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.926 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.926 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.926 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.926 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.926 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.927 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.927 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.927 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.927 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-17T21:29:40.454958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=32, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:32:15.927 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-17T21:32:35.928 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:32:36.225 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T21:32:36.225 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-17T21:32:36.237 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-17T21:32:34.893092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":660,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":660,"num_whiteouts":652,"num_read":159422,"num_read_kb":894014,"num_write":21746,"num_write_kb":454251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1208,"num_bytes_recovered":117790844,"num_keys_recovered":327,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19831,"ondisk_log_size":19831,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379392,"kb_used_data":14528,"kb_used_omap":31,"kb_used_meta":364704,"kb_avail":754595328,"statfs":{"total":773094113280,"available":772705615872,"internally_reserved":0,"allocated":14876672,"data_stored":8212828,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":32080,"internal_metadata":373457584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002867"},"pg_stats":[{"pgid":"3.1f","version":"154'829","reported_seq":5163,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:16.220820+0000","last_change":"2024-09-17T21:32:16.220820+0000","last_active":"2024-09-17T21:32:16.220820+0000","last_peered":"2024-09-17T21:32:16.220820+0000","last_clean":"2024-09-17T21:32:16.220820+0000","last_became_active":"2024-09-17T21:31:15.457506+0000","last_became_peered":"2024-09-17T21:31:15.457506+0000","last_unstale":"2024-09-17T21:32:16.220820+0000","last_undegraded":"2024-09-17T21:32:16.220820+0000","last_fullsized":"2024-09-17T21:32:16.220820+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'829","last_scrub_stamp":"2024-09-17T21:32:16.220752+0000","last_deep_scrub":"154'829","last_deep_scrub_stamp":"2024-09-17T21:32:16.220752+0000","last_clean_scrub_stamp":"2024-09-17T21:32:16.220752+0000","objects_scrubbed":20,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:44.421091+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.0098455079999999993,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6585,"num_read_kb":63347,"num_write":1151,"num_write_kb":32903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":27321072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"154'583","reported_seq":4264,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:19.228199+0000","last_change":"2024-09-17T21:32:19.228199+0000","last_active":"2024-09-17T21:32:19.228199+0000","last_peered":"2024-09-17T21:32:19.228199+0000","last_clean":"2024-09-17T21:32:19.228199+0000","last_became_active":"2024-09-17T21:29:41.472161+0000","last_became_peered":"2024-09-17T21:29:41.472161+0000","last_unstale":"2024-09-17T21:32:19.228199+0000","last_undegraded":"2024-09-17T21:32:19.228199+0000","last_fullsized":"2024-09-17T21:32:19.228199+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'583","last_scrub_stamp":"2024-09-17T21:32:19.228081+0000","last_deep_scrub":"154'583","last_deep_scrub_stamp":"2024-09-17T21:32:19.228081+0000","last_clean_scrub_stamp":"2024-09-17T21:32:19.228081+0000","objects_scrubbed":21,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:12:50.750597+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0.001079279,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3917,"num_read_kb":31515,"num_write":544,"num_write_kb":11792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"154'786","reported_seq":5429,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:20.204487+0000","last_change":"2024-09-17T21:32:20.204487+0000","last_active":"2024-09-17T21:32:20.204487+0000","last_peered":"2024-09-17T21:32:20.204487+0000","last_clean":"2024-09-17T21:32:20.204487+0000","last_became_active":"2024-09-17T21:31:14.449672+0000","last_became_peered":"2024-09-17T21:31:14.449672+0000","last_unstale":"2024-09-17T21:32:20.204487+0000","last_undegraded":"2024-09-17T21:32:20.204487+0000","last_fullsized":"2024-09-17T21:32:20.204487+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'786","last_scrub_stamp":"2024-09-17T21:32:20.204384+0000","last_deep_scrub":"154'786","last_deep_scrub_stamp":"2024-09-17T21:32:20.204384+0000","last_clean_scrub_stamp":"2024-09-17T21:32:20.204384+0000","objects_scrubbed":18,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:35:00.207058+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0.0010859089999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6612,"num_read_kb":51732,"num_write":893,"num_write_kb":18876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":15268754,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"154'461","reported_seq":2722,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:21.237127+0000","last_change":"2024-09-17T21:32:21.237127+0000","last_active":"2024-09-17T21:32:21.237127+0000","last_peered":"2024-09-17T21:32:21.237127+0000","last_clean":"2024-09-17T21:32:21.237127+0000","last_became_active":"2024-09-17T21:31:14.451037+0000","last_became_peered":"2024-09-17T21:31:14.451037+0000","last_unstale":"2024-09-17T21:32:21.237127+0000","last_undegraded":"2024-09-17T21:32:21.237127+0000","last_fullsized":"2024-09-17T21:32:21.237127+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'461","last_scrub_stamp":"2024-09-17T21:32:21.237039+0000","last_deep_scrub":"154'461","last_deep_scrub_stamp":"2024-09-17T21:32:21.237039+0000","last_clean_scrub_stamp":"2024-09-17T21:32:21.237039+0000","objects_scrubbed":28,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:45:53.639437+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0.00068460700000000005,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":2712,"num_read_kb":12910,"num_write":456,"num_write_kb":9120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":604511,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"154'603","reported_seq":8517,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:17.231199+0000","last_change":"2024-09-17T21:32:17.231199+0000","last_active":"2024-09-17T21:32:17.231199+0000","last_peered":"2024-09-17T21:32:17.231199+0000","last_clean":"2024-09-17T21:32:17.231199+0000","last_became_active":"2024-09-17T21:29:41.476041+0000","last_became_peered":"2024-09-17T21:29:41.476041+0000","last_unstale":"2024-09-17T21:32:17.231199+0000","last_undegraded":"2024-09-17T21:32:17.231199+0000","last_fullsized":"2024-09-17T21:32:17.231199+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'603","last_scrub_stamp":"2024-09-17T21:32:17.231108+0000","last_deep_scrub":"154'603","last_deep_scrub_stamp":"2024-09-17T21:32:17.231108+0000","last_clean_scrub_stamp":"2024-09-17T21:32:17.231108+0000","objects_scrubbed":14,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:44:56.259217+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0092951189999999993,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8770,"num_read_kb":53504,"num_write":580,"num_write_kb":24876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"154'655","reported_seq":6608,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:16.531520+0000","last_change":"2024-09-17T21:32:16.531520+0000","last_active":"2024-09-17T21:32:16.531520+0000","last_peered":"2024-09-17T21:32:16.531520+0000","last_clean":"2024-09-17T21:32:16.531520+0000","last_became_active":"2024-09-17T21:30:28.747031+0000","last_became_peered":"2024-09-17T21:30:28.747031+0000","last_unstale":"2024-09-17T21:32:16.531520+0000","last_undegraded":"2024-09-17T21:32:16.531520+0000","last_fullsized":"2024-09-17T21:32:16.531520+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'655","last_scrub_stamp":"2024-09-17T21:32:16.531416+0000","last_deep_scrub":"154'655","last_deep_scrub_stamp":"2024-09-17T21:32:16.531416+0000","last_clean_scrub_stamp":"2024-09-17T21:32:16.531416+0000","objects_scrubbed":25,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:21:02.761571+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0.00134892,"stat_sum":{"num_bytes":19,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":7065,"num_read_kb":21661,"num_write":665,"num_write_kb":6726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":235969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"154'448","reported_seq":4798,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:17.008280+0000","last_change":"2024-09-17T21:32:17.008280+0000","last_active":"2024-09-17T21:32:17.008280+0000","last_peered":"2024-09-17T21:32:17.008280+0000","last_clean":"2024-09-17T21:32:17.008280+0000","last_became_active":"2024-09-17T21:30:28.745634+0000","last_became_peered":"2024-09-17T21:30:28.745634+0000","last_unstale":"2024-09-17T21:32:17.008280+0000","last_undegraded":"2024-09-17T21:32:17.008280+0000","last_fullsized":"2024-09-17T21:32:17.008280+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'448","last_scrub_stamp":"2024-09-17T21:32:17.008199+0000","last_deep_scrub":"154'448","last_deep_scrub_stamp":"2024-09-17T21:32:17.008199+0000","last_clean_scrub_stamp":"2024-09-17T21:32:17.008199+0000","objects_scrubbed":24,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:16.318387+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.001320393,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4551,"num_read_kb":56994,"num_write":426,"num_write_kb":45225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":36017677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"154'530","reported_seq":4564,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:15.189543+0000","last_change":"2024-09-17T21:32:15.189543+0000","last_active":"2024-09-17T21:32:15.189543+0000","last_peered":"2024-09-17T21:32:15.189543+0000","last_clean":"2024-09-17T21:32:15.189543+0000","last_became_active":"2024-09-17T21:29:41.472468+0000","last_became_peered":"2024-09-17T21:29:41.472468+0000","last_unstale":"2024-09-17T21:32:15.189543+0000","last_undegraded":"2024-09-17T21:32:15.189543+0000","last_fullsized":"2024-09-17T21:32:15.189543+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'530","last_scrub_stamp":"2024-09-17T21:32:15.189463+0000","last_deep_scrub":"154'530","last_deep_scrub_stamp":"2024-09-17T21:32:15.189463+0000","last_clean_scrub_stamp":"2024-09-17T21:32:15.189463+0000","objects_scrubbed":18,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:40:25.725425+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0.002623551,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4791,"num_read_kb":27686,"num_write":500,"num_write_kb":13101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"154'835","reported_seq":3281,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:15.207082+0000","last_change":"2024-09-17T21:32:15.207082+0000","last_active":"2024-09-17T21:32:15.207082+0000","last_peered":"2024-09-17T21:32:15.207082+0000","last_clean":"2024-09-17T21:32:15.207082+0000","last_became_active":"2024-09-17T21:31:15.456240+0000","last_became_peered":"2024-09-17T21:31:15.456240+0000","last_unstale":"2024-09-17T21:32:15.207082+0000","last_undegraded":"2024-09-17T21:32:15.207082+0000","last_fullsized":"2024-09-17T21:32:15.207082+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'835","last_scrub_stamp":"2024-09-17T21:32:15.206972+0000","last_deep_scrub":"154'835","last_deep_scrub_stamp":"2024-09-17T21:32:15.206972+0000","last_clean_scrub_stamp":"2024-09-17T21:32:15.206972+0000","objects_scrubbed":21,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:39:11.390459+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.010220406,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4952,"num_read_kb":22844,"num_write":955,"num_write_kb":30092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":9394579,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"154'1130","reported_seq":3123,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:18.235904+0000","last_change":"2024-09-17T21:32:18.235904+0000","last_active":"2024-09-17T21:32:18.235904+0000","last_peered":"2024-09-17T21:32:18.235904+0000","last_clean":"2024-09-17T21:32:18.235904+0000","last_became_active":"2024-09-17T21:31:15.456612+0000","last_became_peered":"2024-09-17T21:31:15.456612+0000","last_unstale":"2024-09-17T21:32:18.235904+0000","last_undegraded":"2024-09-17T21:32:18.235904+0000","last_fullsized":"2024-09-17T21:32:18.235904+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1130","last_scrub_stamp":"2024-09-17T21:32:18.235811+0000","last_deep_scrub":"154'1130","last_deep_scrub_stamp":"2024-09-17T21:32:18.235811+0000","last_clean_scrub_stamp":"2024-09-17T21:32:18.235811+0000","objects_scrubbed":24,"log_size":1130,"log_dups_size":0,"ondisk_log_size":1130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:50:37.315233+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0.00051663100000000003,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":5737,"num_read_kb":20891,"num_write":1670,"num_write_kb":12347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3063082,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"154'539","reported_seq":3224,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:24.485993+0000","last_change":"2024-09-17T21:32:24.485993+0000","last_active":"2024-09-17T21:32:24.485993+0000","last_peered":"2024-09-17T21:32:24.485993+0000","last_clean":"2024-09-17T21:32:24.485993+0000","last_became_active":"2024-09-17T21:29:41.478346+0000","last_became_peered":"2024-09-17T21:29:41.478346+0000","last_unstale":"2024-09-17T21:32:24.485993+0000","last_undegraded":"2024-09-17T21:32:24.485993+0000","last_fullsized":"2024-09-17T21:32:24.485993+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'539","last_scrub_stamp":"2024-09-17T21:32:24.485897+0000","last_deep_scrub":"154'539","last_deep_scrub_stamp":"2024-09-17T21:32:24.485897+0000","last_clean_scrub_stamp":"2024-09-17T21:32:24.485897+0000","objects_scrubbed":27,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:37.938933+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0.0017578019999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3628,"num_read_kb":9299,"num_write":426,"num_write_kb":8355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"154'315","reported_seq":2911,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:15.528298+0000","last_change":"2024-09-17T21:32:15.528298+0000","last_active":"2024-09-17T21:32:15.528298+0000","last_peered":"2024-09-17T21:32:15.528298+0000","last_clean":"2024-09-17T21:32:15.528298+0000","last_became_active":"2024-09-17T21:29:41.477611+0000","last_became_peered":"2024-09-17T21:29:41.477611+0000","last_unstale":"2024-09-17T21:32:15.528298+0000","last_undegraded":"2024-09-17T21:32:15.528298+0000","last_fullsized":"2024-09-17T21:32:15.528298+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'315","last_scrub_stamp":"2024-09-17T21:32:15.528216+0000","last_deep_scrub":"154'315","last_deep_scrub_stamp":"2024-09-17T21:32:15.528216+0000","last_clean_scrub_stamp":"2024-09-17T21:32:15.528216+0000","objects_scrubbed":18,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:32:51.539291+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0.00064521399999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2154,"num_read_kb":14079,"num_write":352,"num_write_kb":12746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":288,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:17.054328+0000","last_change":"2024-09-17T21:32:17.054328+0000","last_active":"2024-09-17T21:32:17.054328+0000","last_peered":"2024-09-17T21:32:17.054328+0000","last_clean":"2024-09-17T21:32:17.054328+0000","last_became_active":"2024-09-17T21:29:32.971677+0000","last_became_peered":"2024-09-17T21:29:32.971677+0000","last_unstale":"2024-09-17T21:32:17.054328+0000","last_undegraded":"2024-09-17T21:32:17.054328+0000","last_fullsized":"2024-09-17T21:32:17.054328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:32:17.054221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:32:17.054221+0000","last_clean_scrub_stamp":"2024-09-17T21:32:17.054221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:18:22.761487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041121300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"154'863","reported_seq":4487,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:13.235036+0000","last_change":"2024-09-17T21:32:13.235036+0000","last_active":"2024-09-17T21:32:13.235036+0000","last_peered":"2024-09-17T21:32:13.235036+0000","last_clean":"2024-09-17T21:32:13.235036+0000","last_became_active":"2024-09-17T21:30:31.755039+0000","last_became_peered":"2024-09-17T21:30:31.755039+0000","last_unstale":"2024-09-17T21:32:13.235036+0000","last_undegraded":"2024-09-17T21:32:13.235036+0000","last_fullsized":"2024-09-17T21:32:13.235036+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'863","last_scrub_stamp":"2024-09-17T21:32:13.234973+0000","last_deep_scrub":"154'863","last_deep_scrub_stamp":"2024-09-17T21:32:13.234973+0000","last_clean_scrub_stamp":"2024-09-17T21:32:13.234973+0000","objects_scrubbed":15,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:09:05.153455+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.01026987,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6950,"num_read_kb":18293,"num_write":963,"num_write_kb":8597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":448871,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":221,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:13.964692+0000","last_change":"2024-09-17T21:32:13.964692+0000","last_active":"2024-09-17T21:32:13.964692+0000","last_peered":"2024-09-17T21:32:13.964692+0000","last_clean":"2024-09-17T21:32:13.964692+0000","last_became_active":"2024-09-17T21:30:28.744742+0000","last_became_peered":"2024-09-17T21:30:28.744742+0000","last_unstale":"2024-09-17T21:32:13.964692+0000","last_undegraded":"2024-09-17T21:32:13.964692+0000","last_fullsized":"2024-09-17T21:32:13.964692+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:32:13.964616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:32:13.964616+0000","last_clean_scrub_stamp":"2024-09-17T21:32:13.964616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:57.632900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"154'614","reported_seq":2825,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:14.166841+0000","last_change":"2024-09-17T21:32:14.166841+0000","last_active":"2024-09-17T21:32:14.166841+0000","last_peered":"2024-09-17T21:32:14.166841+0000","last_clean":"2024-09-17T21:32:14.166841+0000","last_became_active":"2024-09-17T21:29:41.472095+0000","last_became_peered":"2024-09-17T21:29:41.472095+0000","last_unstale":"2024-09-17T21:32:14.166841+0000","last_undegraded":"2024-09-17T21:32:14.166841+0000","last_fullsized":"2024-09-17T21:32:14.166841+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'614","last_scrub_stamp":"2024-09-17T21:32:14.166690+0000","last_deep_scrub":"154'614","last_deep_scrub_stamp":"2024-09-17T21:32:14.166690+0000","last_clean_scrub_stamp":"2024-09-17T21:32:14.166690+0000","objects_scrubbed":22,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:33:46.331075+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0028983989999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":3726,"num_read_kb":13082,"num_write":700,"num_write_kb":8995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":290,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:16.526857+0000","last_change":"2024-09-17T21:32:16.526857+0000","last_active":"2024-09-17T21:32:16.526857+0000","last_peered":"2024-09-17T21:32:16.526857+0000","last_clean":"2024-09-17T21:32:16.526857+0000","last_became_active":"2024-09-17T21:29:33.071665+0000","last_became_peered":"2024-09-17T21:29:33.071665+0000","last_unstale":"2024-09-17T21:32:16.526857+0000","last_undegraded":"2024-09-17T21:32:16.526857+0000","last_fullsized":"2024-09-17T21:32:16.526857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:32:16.526763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:32:16.526763+0000","last_clean_scrub_stamp":"2024-09-17T21:32:16.526763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:37:44.132821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047969899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"154'830","reported_seq":4910,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:15.980883+0000","last_change":"2024-09-17T21:32:15.980836+0000","last_active":"2024-09-17T21:32:15.980883+0000","last_peered":"2024-09-17T21:32:15.980883+0000","last_clean":"2024-09-17T21:32:15.980883+0000","last_became_active":"2024-09-17T21:30:31.752750+0000","last_became_peered":"2024-09-17T21:30:31.752750+0000","last_unstale":"2024-09-17T21:32:15.980883+0000","last_undegraded":"2024-09-17T21:32:15.980883+0000","last_fullsized":"2024-09-17T21:32:15.980883+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'830","last_scrub_stamp":"2024-09-17T21:32:15.980760+0000","last_deep_scrub":"154'830","last_deep_scrub_stamp":"2024-09-17T21:32:15.980760+0000","last_clean_scrub_stamp":"2024-09-17T21:32:15.980760+0000","objects_scrubbed":25,"log_size":830,"log_dups_size":0,"ondisk_log_size":830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:34:32.224105+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.00094011400000000003,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6270,"num_read_kb":32698,"num_write":1050,"num_write_kb":17424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3231045,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":222,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:14.959848+0000","last_change":"2024-09-17T21:32:14.959800+0000","last_active":"2024-09-17T21:32:14.959848+0000","last_peered":"2024-09-17T21:32:14.959848+0000","last_clean":"2024-09-17T21:32:14.959848+0000","last_became_active":"2024-09-17T21:30:28.747728+0000","last_became_peered":"2024-09-17T21:30:28.747728+0000","last_unstale":"2024-09-17T21:32:14.959848+0000","last_undegraded":"2024-09-17T21:32:14.959848+0000","last_fullsized":"2024-09-17T21:32:14.959848+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:32:14.959742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:32:14.959742+0000","last_clean_scrub_stamp":"2024-09-17T21:32:14.959742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:12:55.032603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"154'525","reported_seq":4886,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:15.233401+0000","last_change":"2024-09-17T21:32:15.233401+0000","last_active":"2024-09-17T21:32:15.233401+0000","last_peered":"2024-09-17T21:32:15.233401+0000","last_clean":"2024-09-17T21:32:15.233401+0000","last_became_active":"2024-09-17T21:31:14.455957+0000","last_became_peered":"2024-09-17T21:31:14.455957+0000","last_unstale":"2024-09-17T21:32:15.233401+0000","last_undegraded":"2024-09-17T21:32:15.233401+0000","last_fullsized":"2024-09-17T21:32:15.233401+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'525","last_scrub_stamp":"2024-09-17T21:32:15.233299+0000","last_deep_scrub":"154'525","last_deep_scrub_stamp":"2024-09-17T21:32:15.233299+0000","last_clean_scrub_stamp":"2024-09-17T21:32:15.233299+0000","objects_scrubbed":23,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:49:02.313136+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0.00087537200000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4988,"num_read_kb":31633,"num_write":558,"num_write_kb":9789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":1545182,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"154'872","reported_seq":5051,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:14.503181+0000","last_change":"2024-09-17T21:32:14.503181+0000","last_active":"2024-09-17T21:32:14.503181+0000","last_peered":"2024-09-17T21:32:14.503181+0000","last_clean":"2024-09-17T21:32:14.503181+0000","last_became_active":"2024-09-17T21:29:41.475273+0000","last_became_peered":"2024-09-17T21:29:41.475273+0000","last_unstale":"2024-09-17T21:32:14.503181+0000","last_undegraded":"2024-09-17T21:32:14.503181+0000","last_fullsized":"2024-09-17T21:32:14.503181+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'872","last_scrub_stamp":"2024-09-17T21:32:14.503101+0000","last_deep_scrub":"154'872","last_deep_scrub_stamp":"2024-09-17T21:32:14.503101+0000","last_clean_scrub_stamp":"2024-09-17T21:32:14.503101+0000","objects_scrubbed":17,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:09:01.650917+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.001059431,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":6333,"num_read_kb":18110,"num_write":998,"num_write_kb":6659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"154'3","reported_seq":125,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:16.192203+0000","last_change":"2024-09-17T21:32:16.192203+0000","last_active":"2024-09-17T21:32:16.192203+0000","last_peered":"2024-09-17T21:32:16.192203+0000","last_clean":"2024-09-17T21:32:16.192203+0000","last_became_active":"2024-09-17T21:31:11.421737+0000","last_became_peered":"2024-09-17T21:31:11.421737+0000","last_unstale":"2024-09-17T21:32:16.192203+0000","last_undegraded":"2024-09-17T21:32:16.192203+0000","last_fullsized":"2024-09-17T21:32:16.192203+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'3","last_scrub_stamp":"2024-09-17T21:32:16.192122+0000","last_deep_scrub":"154'3","last_deep_scrub_stamp":"2024-09-17T21:32:16.192122+0000","last_clean_scrub_stamp":"2024-09-17T21:32:16.192122+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:15:53.503475+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"154'429","reported_seq":3005,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:18.000218+0000","last_change":"2024-09-17T21:32:18.000166+0000","last_active":"2024-09-17T21:32:18.000218+0000","last_peered":"2024-09-17T21:32:18.000218+0000","last_clean":"2024-09-17T21:32:18.000218+0000","last_became_active":"2024-09-17T21:30:28.746690+0000","last_became_peered":"2024-09-17T21:30:28.746690+0000","last_unstale":"2024-09-17T21:32:18.000218+0000","last_undegraded":"2024-09-17T21:32:18.000218+0000","last_fullsized":"2024-09-17T21:32:18.000218+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'429","last_scrub_stamp":"2024-09-17T21:32:18.000078+0000","last_deep_scrub":"154'429","last_deep_scrub_stamp":"2024-09-17T21:32:18.000078+0000","last_clean_scrub_stamp":"2024-09-17T21:32:18.000078+0000","objects_scrubbed":16,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:55:22.542703+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00090442500000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3369,"num_read_kb":13331,"num_write":389,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1513,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":249,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:14.144042+0000","last_change":"2024-09-17T21:32:14.144042+0000","last_active":"2024-09-17T21:32:14.144042+0000","last_peered":"2024-09-17T21:32:14.144042+0000","last_clean":"2024-09-17T21:32:14.144042+0000","last_became_active":"2024-09-17T21:31:53.978829+0000","last_became_peered":"2024-09-17T21:31:53.978829+0000","last_unstale":"2024-09-17T21:32:14.144042+0000","last_undegraded":"2024-09-17T21:32:14.144042+0000","last_fullsized":"2024-09-17T21:32:14.144042+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:32:14.143972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:32:14.143972+0000","last_clean_scrub_stamp":"2024-09-17T21:32:14.143972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:31:28.059626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"154'456","reported_seq":3991,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:14.239055+0000","last_change":"2024-09-17T21:32:14.239055+0000","last_active":"2024-09-17T21:32:14.239055+0000","last_peered":"2024-09-17T21:32:14.239055+0000","last_clean":"2024-09-17T21:32:14.239055+0000","last_became_active":"2024-09-17T21:29:41.473964+0000","last_became_peered":"2024-09-17T21:29:41.473964+0000","last_unstale":"2024-09-17T21:32:14.239055+0000","last_undegraded":"2024-09-17T21:32:14.239055+0000","last_fullsized":"2024-09-17T21:32:14.239055+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'456","last_scrub_stamp":"2024-09-17T21:32:14.238982+0000","last_deep_scrub":"154'456","last_deep_scrub_stamp":"2024-09-17T21:32:14.238982+0000","last_clean_scrub_stamp":"2024-09-17T21:32:14.238982+0000","objects_scrubbed":25,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:41:33.975188+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.0097061290000000008,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3802,"num_read_kb":64522,"num_write":578,"num_write_kb":39811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":299,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:20.533438+0000","last_change":"2024-09-17T21:32:20.533438+0000","last_active":"2024-09-17T21:32:20.533438+0000","last_peered":"2024-09-17T21:32:20.533438+0000","last_clean":"2024-09-17T21:32:20.533438+0000","last_became_active":"2024-09-17T21:30:28.746261+0000","last_became_peered":"2024-09-17T21:30:28.746261+0000","last_unstale":"2024-09-17T21:32:20.533438+0000","last_undegraded":"2024-09-17T21:32:20.533438+0000","last_fullsized":"2024-09-17T21:32:20.533438+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:32:20.533305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:32:20.533305+0000","last_clean_scrub_stamp":"2024-09-17T21:32:20.533305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:31:35.059579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044115199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"154'355","reported_seq":4225,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:17.212811+0000","last_change":"2024-09-17T21:32:17.212811+0000","last_active":"2024-09-17T21:32:17.212811+0000","last_peered":"2024-09-17T21:32:17.212811+0000","last_clean":"2024-09-17T21:32:17.212811+0000","last_became_active":"2024-09-17T21:31:14.459936+0000","last_became_peered":"2024-09-17T21:31:14.459936+0000","last_unstale":"2024-09-17T21:32:17.212811+0000","last_undegraded":"2024-09-17T21:32:17.212811+0000","last_fullsized":"2024-09-17T21:32:17.212811+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'355","last_scrub_stamp":"2024-09-17T21:32:17.212746+0000","last_deep_scrub":"154'355","last_deep_scrub_stamp":"2024-09-17T21:32:17.212746+0000","last_clean_scrub_stamp":"2024-09-17T21:32:17.212746+0000","objects_scrubbed":22,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:37:15.969122+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0012333649999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":3924,"num_read_kb":72696,"num_write":302,"num_write_kb":18067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":13048770,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"154'2","reported_seq":125,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:17.207670+0000","last_change":"2024-09-17T21:32:17.207670+0000","last_active":"2024-09-17T21:32:17.207670+0000","last_peered":"2024-09-17T21:32:17.207670+0000","last_clean":"2024-09-17T21:32:17.207670+0000","last_became_active":"2024-09-17T21:31:11.422076+0000","last_became_peered":"2024-09-17T21:31:11.422076+0000","last_unstale":"2024-09-17T21:32:17.207670+0000","last_undegraded":"2024-09-17T21:32:17.207670+0000","last_fullsized":"2024-09-17T21:32:17.207670+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'2","last_scrub_stamp":"2024-09-17T21:32:17.207589+0000","last_deep_scrub":"154'2","last_deep_scrub_stamp":"2024-09-17T21:32:17.207589+0000","last_clean_scrub_stamp":"2024-09-17T21:32:17.207589+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:09.048411+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"154'74","reported_seq":383,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:17.556177+0000","last_change":"2024-09-17T21:32:17.556177+0000","last_active":"2024-09-17T21:32:17.556177+0000","last_peered":"2024-09-17T21:32:17.556177+0000","last_clean":"2024-09-17T21:32:17.556177+0000","last_became_active":"2024-09-17T21:29:28.553733+0000","last_became_peered":"2024-09-17T21:29:28.553733+0000","last_unstale":"2024-09-17T21:32:17.556177+0000","last_undegraded":"2024-09-17T21:32:17.556177+0000","last_fullsized":"2024-09-17T21:32:17.556177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'74","last_scrub_stamp":"2024-09-17T21:32:17.556089+0000","last_deep_scrub":"154'74","last_deep_scrub_stamp":"2024-09-17T21:32:17.556089+0000","last_clean_scrub_stamp":"2024-09-17T21:32:17.556089+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:39:35.315808+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"154'724","reported_seq":4593,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:18.200264+0000","last_change":"2024-09-17T21:32:18.200264+0000","last_active":"2024-09-17T21:32:18.200264+0000","last_peered":"2024-09-17T21:32:18.200264+0000","last_clean":"2024-09-17T21:32:18.200264+0000","last_became_active":"2024-09-17T21:30:31.753839+0000","last_became_peered":"2024-09-17T21:30:31.753839+0000","last_unstale":"2024-09-17T21:32:18.200264+0000","last_undegraded":"2024-09-17T21:32:18.200264+0000","last_fullsized":"2024-09-17T21:32:18.200264+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'724","last_scrub_stamp":"2024-09-17T21:32:18.200181+0000","last_deep_scrub":"154'724","last_deep_scrub_stamp":"2024-09-17T21:32:18.200181+0000","last_clean_scrub_stamp":"2024-09-17T21:32:18.200181+0000","objects_scrubbed":12,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:35:00.576479+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0030177569999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5120,"num_read_kb":35075,"num_write":882,"num_write_kb":14888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2410412,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"154'424","reported_seq":3603,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:17.476583+0000","last_change":"2024-09-17T21:32:17.476583+0000","last_active":"2024-09-17T21:32:17.476583+0000","last_peered":"2024-09-17T21:32:17.476583+0000","last_clean":"2024-09-17T21:32:17.476583+0000","last_became_active":"2024-09-17T21:29:41.477965+0000","last_became_peered":"2024-09-17T21:29:41.477965+0000","last_unstale":"2024-09-17T21:32:17.476583+0000","last_undegraded":"2024-09-17T21:32:17.476583+0000","last_fullsized":"2024-09-17T21:32:17.476583+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'424","last_scrub_stamp":"2024-09-17T21:32:17.476465+0000","last_deep_scrub":"154'424","last_deep_scrub_stamp":"2024-09-17T21:32:17.476465+0000","last_clean_scrub_stamp":"2024-09-17T21:32:17.476465+0000","objects_scrubbed":12,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:35:03.180739+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00062142999999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3781,"num_read_kb":28832,"num_write":427,"num_write_kb":12788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"154'618","reported_seq":4109,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:16.121710+0000","last_change":"2024-09-17T21:32:16.121710+0000","last_active":"2024-09-17T21:32:16.121710+0000","last_peered":"2024-09-17T21:32:16.121710+0000","last_clean":"2024-09-17T21:32:16.121710+0000","last_became_active":"2024-09-17T21:29:41.473420+0000","last_became_peered":"2024-09-17T21:29:41.473420+0000","last_unstale":"2024-09-17T21:32:16.121710+0000","last_undegraded":"2024-09-17T21:32:16.121710+0000","last_fullsized":"2024-09-17T21:32:16.121710+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'618","last_scrub_stamp":"2024-09-17T21:32:16.121612+0000","last_deep_scrub":"154'618","last_deep_scrub_stamp":"2024-09-17T21:32:16.121612+0000","last_clean_scrub_stamp":"2024-09-17T21:32:16.121612+0000","objects_scrubbed":23,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:20:12.123923+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0.00058409800000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":4583,"num_read_kb":15568,"num_write":610,"num_write_kb":6172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"154'558","reported_seq":7234,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:16.213173+0000","last_change":"2024-09-17T21:32:16.213173+0000","last_active":"2024-09-17T21:32:16.213173+0000","last_peered":"2024-09-17T21:32:16.213173+0000","last_clean":"2024-09-17T21:32:16.213173+0000","last_became_active":"2024-09-17T21:29:41.472354+0000","last_became_peered":"2024-09-17T21:29:41.472354+0000","last_unstale":"2024-09-17T21:32:16.213173+0000","last_undegraded":"2024-09-17T21:32:16.213173+0000","last_fullsized":"2024-09-17T21:32:16.213173+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'558","last_scrub_stamp":"2024-09-17T21:32:16.213068+0000","last_deep_scrub":"154'558","last_deep_scrub_stamp":"2024-09-17T21:32:16.213068+0000","last_clean_scrub_stamp":"2024-09-17T21:32:16.213068+0000","objects_scrubbed":17,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:26:42.077860+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0025676010000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7686,"num_read_kb":30319,"num_write":462,"num_write_kb":3994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"154'679","reported_seq":4903,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:19.182351+0000","last_change":"2024-09-17T21:32:19.182351+0000","last_active":"2024-09-17T21:32:19.182351+0000","last_peered":"2024-09-17T21:32:19.182351+0000","last_clean":"2024-09-17T21:32:19.182351+0000","last_became_active":"2024-09-17T21:31:15.457822+0000","last_became_peered":"2024-09-17T21:31:15.457822+0000","last_unstale":"2024-09-17T21:32:19.182351+0000","last_undegraded":"2024-09-17T21:32:19.182351+0000","last_fullsized":"2024-09-17T21:32:19.182351+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'679","last_scrub_stamp":"2024-09-17T21:32:19.182239+0000","last_deep_scrub":"154'679","last_deep_scrub_stamp":"2024-09-17T21:32:19.182239+0000","last_clean_scrub_stamp":"2024-09-17T21:32:19.182239+0000","objects_scrubbed":18,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:40:22.126416+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.00086872700000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7278,"num_read_kb":17574,"num_write":576,"num_write_kb":5172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":2355433,"num_keys_recovered":58,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"154'642","reported_seq":4545,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:18.564178+0000","last_change":"2024-09-17T21:32:18.564178+0000","last_active":"2024-09-17T21:32:18.564178+0000","last_peered":"2024-09-17T21:32:18.564178+0000","last_clean":"2024-09-17T21:32:18.564178+0000","last_became_active":"2024-09-17T21:31:14.457648+0000","last_became_peered":"2024-09-17T21:31:14.457648+0000","last_unstale":"2024-09-17T21:32:18.564178+0000","last_undegraded":"2024-09-17T21:32:18.564178+0000","last_fullsized":"2024-09-17T21:32:18.564178+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'642","last_scrub_stamp":"2024-09-17T21:32:18.564080+0000","last_deep_scrub":"154'642","last_deep_scrub_stamp":"2024-09-17T21:32:18.564080+0000","last_clean_scrub_stamp":"2024-09-17T21:32:18.564080+0000","objects_scrubbed":20,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:24:16.441365+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0016040850000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5354,"num_read_kb":12719,"num_write":707,"num_write_kb":4578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2843955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"154'599","reported_seq":4414,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:19.542899+0000","last_change":"2024-09-17T21:32:19.542899+0000","last_active":"2024-09-17T21:32:19.542899+0000","last_peered":"2024-09-17T21:32:19.542899+0000","last_clean":"2024-09-17T21:32:19.542899+0000","last_became_active":"2024-09-17T21:29:41.473736+0000","last_became_peered":"2024-09-17T21:29:41.473736+0000","last_unstale":"2024-09-17T21:32:19.542899+0000","last_undegraded":"2024-09-17T21:32:19.542899+0000","last_fullsized":"2024-09-17T21:32:19.542899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'599","last_scrub_stamp":"2024-09-17T21:32:19.542799+0000","last_deep_scrub":"154'599","last_deep_scrub_stamp":"2024-09-17T21:32:19.542799+0000","last_clean_scrub_stamp":"2024-09-17T21:32:19.542799+0000","objects_scrubbed":19,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:23:53.455432+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0.0013374140000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5042,"num_read_kb":17595,"num_write":692,"num_write_kb":5927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"154'907","reported_seq":4567,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:22.500476+0000","last_change":"2024-09-17T21:32:22.500476+0000","last_active":"2024-09-17T21:32:22.500476+0000","last_peered":"2024-09-17T21:32:22.500476+0000","last_clean":"2024-09-17T21:32:22.500476+0000","last_became_active":"2024-09-17T21:29:41.473394+0000","last_became_peered":"2024-09-17T21:29:41.473394+0000","last_unstale":"2024-09-17T21:32:22.500476+0000","last_undegraded":"2024-09-17T21:32:22.500476+0000","last_fullsized":"2024-09-17T21:32:22.500476+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'907","last_scrub_stamp":"2024-09-17T21:32:22.500373+0000","last_deep_scrub":"154'907","last_deep_scrub_stamp":"2024-09-17T21:32:22.500373+0000","last_clean_scrub_stamp":"2024-09-17T21:32:22.500373+0000","objects_scrubbed":25,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:16:26.253378+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0.002038134,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6008,"num_read_kb":8422,"num_write":1180,"num_write_kb":21760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"154'646","reported_seq":2614,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:18.044168+0000","last_change":"2024-09-17T21:32:18.044168+0000","last_active":"2024-09-17T21:32:18.044168+0000","last_peered":"2024-09-17T21:32:18.044168+0000","last_clean":"2024-09-17T21:32:18.044168+0000","last_became_active":"2024-09-17T21:29:41.471342+0000","last_became_peered":"2024-09-17T21:29:41.471342+0000","last_unstale":"2024-09-17T21:32:18.044168+0000","last_undegraded":"2024-09-17T21:32:18.044168+0000","last_fullsized":"2024-09-17T21:32:18.044168+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'646","last_scrub_stamp":"2024-09-17T21:32:18.044078+0000","last_deep_scrub":"154'646","last_deep_scrub_stamp":"2024-09-17T21:32:18.044078+0000","last_clean_scrub_stamp":"2024-09-17T21:32:18.044078+0000","objects_scrubbed":23,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:57:57.432847+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.00044458799999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3239,"num_read_kb":14667,"num_write":843,"num_write_kb":8817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"154'440","reported_seq":4888,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:21.515223+0000","last_change":"2024-09-17T21:32:21.515223+0000","last_active":"2024-09-17T21:32:21.515223+0000","last_peered":"2024-09-17T21:32:21.515223+0000","last_clean":"2024-09-17T21:32:21.515223+0000","last_became_active":"2024-09-17T21:29:41.473252+0000","last_became_peered":"2024-09-17T21:29:41.473252+0000","last_unstale":"2024-09-17T21:32:21.515223+0000","last_undegraded":"2024-09-17T21:32:21.515223+0000","last_fullsized":"2024-09-17T21:32:21.515223+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'440","last_scrub_stamp":"2024-09-17T21:32:21.515105+0000","last_deep_scrub":"154'440","last_deep_scrub_stamp":"2024-09-17T21:32:21.515105+0000","last_clean_scrub_stamp":"2024-09-17T21:32:21.515105+0000","objects_scrubbed":22,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:45:01.976561+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0013588999999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4864,"num_read_kb":34676,"num_write":382,"num_write_kb":15080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"154'394","reported_seq":2311,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:18.194616+0000","last_change":"2024-09-17T21:32:18.194616+0000","last_active":"2024-09-17T21:32:18.194616+0000","last_peered":"2024-09-17T21:32:18.194616+0000","last_clean":"2024-09-17T21:32:18.194616+0000","last_became_active":"2024-09-17T21:29:41.477019+0000","last_became_peered":"2024-09-17T21:29:41.477019+0000","last_unstale":"2024-09-17T21:32:18.194616+0000","last_undegraded":"2024-09-17T21:32:18.194616+0000","last_fullsized":"2024-09-17T21:32:18.194616+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'394","last_scrub_stamp":"2024-09-17T21:32:18.194512+0000","last_deep_scrub":"154'394","last_deep_scrub_stamp":"2024-09-17T21:32:18.194512+0000","last_clean_scrub_stamp":"2024-09-17T21:32:18.194512+0000","objects_scrubbed":19,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:41:01.735141+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0094423349999999996,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":1879,"num_read_kb":5231,"num_write":351,"num_write_kb":2424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"154'463","reported_seq":4099,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T21:32:23.481039+0000","last_change":"2024-09-17T21:32:23.481039+0000","last_active":"2024-09-17T21:32:23.481039+0000","last_peered":"2024-09-17T21:32:23.481039+0000","last_clean":"2024-09-17T21:32:23.481039+0000","last_became_active":"2024-09-17T21:29:41.473182+0000","last_became_peered":"2024-09-17T21:29:41.473182+0000","last_unstale":"2024-09-17T21:32:23.481039+0000","last_undegraded":"2024-09-17T21:32:23.481039+0000","last_fullsized":"2024-09-17T21:32:23.481039+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'463","last_scrub_stamp":"2024-09-17T21:32:23.480940+0000","last_deep_scrub":"154'463","last_deep_scrub_stamp":"2024-09-17T21:32:23.480940+0000","last_clean_scrub_stamp":"2024-09-17T21:32:23.480940+0000","objects_scrubbed":23,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:54:06.192518+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0.00092929600000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3686,"num_read_kb":22454,"num_write":375,"num_write_kb":9447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":656,"num_object_clones":0,"num_object_copies":1312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":656,"num_whiteouts":652,"num_read":159356,"num_read_kb":893959,"num_write":21643,"num_write_kb":452621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1205,"num_bytes_recovered":117790825,"num_keys_recovered":327,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10476,"internal_metadata":0},"log_size":19752,"ondisk_log_size":19752,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9938,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55248,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316592,"statfs":{"total":96636764160,"available":96580190208,"internally_reserved":0,"allocated":1900544,"data_stored":1469370,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28824,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343016,"statfs":{"total":96636764160,"available":96607248384,"internally_reserved":0,"allocated":1843200,"data_stored":879002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509482,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41308,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330532,"statfs":{"total":96636764160,"available":96594464768,"internally_reserved":0,"allocated":1847296,"data_stored":879021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59356,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312484,"statfs":{"total":96636764160,"available":96575983616,"internally_reserved":0,"allocated":1847296,"data_stored":879021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55192,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316648,"statfs":{"total":96636764160,"available":96580247552,"internally_reserved":0,"allocated":1843200,"data_stored":879002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41304,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330536,"statfs":{"total":96636764160,"available":96594468864,"internally_reserved":0,"allocated":1843200,"data_stored":879002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":63,"seq":270582939675,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42912,"kb_used_data":1808,"kb_used_omap":20,"kb_used_meta":41067,"kb_avail":94328928,"statfs":{"total":96636764160,"available":96592822272,"internally_reserved":0,"allocated":1851392,"data_stored":879040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20925,"internal_metadata":42053187},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55248,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316592,"statfs":{"total":96636764160,"available":96580190208,"internally_reserved":0,"allocated":1900544,"data_stored":1469370,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4969,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10476,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:32:36.240 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-17T21:32:36.540 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-17T21:32:36.540 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-17T21:32:36.540 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T21:32:36.540 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:32:36.582 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T21:32:36.582 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T21:32:36.582 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:32:36.622 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T21:32:36.623 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T21:32:36.623 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:32:36.691 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T21:32:36.691 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T21:32:36.691 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:32:36.762 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T21:32:36.763 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-17T21:32:36.763 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:32:36.803 INFO:tasks.ceph.osd.4:Stopped 2024-09-17T21:32:36.803 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T21:32:36.803 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:32:36.883 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T21:32:36.883 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-17T21:32:36.883 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:32:36.960 INFO:tasks.ceph.osd.6:Stopped 2024-09-17T21:32:36.960 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-17T21:32:36.960 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:32:37.036 INFO:tasks.ceph.osd.7:Stopped 2024-09-17T21:32:37.036 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-17T21:32:37.036 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-17T21:32:37.036 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:32:37.063 INFO:tasks.ceph.mgr.y:Stopped 2024-09-17T21:32:37.063 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-17T21:32:37.063 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:32:37.104 INFO:tasks.ceph.mgr.x:Stopped 2024-09-17T21:32:37.104 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-17T21:32:37.104 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T21:32:37.104 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:32:37.115 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T21:32:37.116 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T21:32:37.116 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:32:37.167 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T21:32:37.168 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T21:32:37.168 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:32:37.180 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T21:32:37.180 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-17T21:32:37.181 DEBUG:teuthology.orchestra.run.smithi129:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-17T21:32:37.240 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T21:32:37.240 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-17T21:32:37.473 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T21:32:37.474 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-17T21:32:37.604 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T21:32:37.604 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-17T21:32:37.655 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T21:32:37.655 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-17T21:32:37.706 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T21:32:37.706 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-17T21:32:37.839 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T21:32:37.839 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-17T21:32:37.916 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T21:32:37.917 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-17T21:32:37.975 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T21:32:37.976 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-17T21:32:38.035 INFO:tasks.ceph:Archiving mon data... 2024-09-17T21:32:38.036 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909352/data/mon.a.tgz 2024-09-17T21:32:38.037 DEBUG:teuthology.orchestra.run.smithi129:> mktemp 2024-09-17T21:32:38.052 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.31ouNQ5pne 2024-09-17T21:32:38.053 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.31ouNQ5pne 2024-09-17T21:32:38.199 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.31ouNQ5pne 2024-09-17T21:32:38.282 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.31ouNQ5pne is 389KB 2024-09-17T21:32:38.350 DEBUG:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.31ouNQ5pne 2024-09-17T21:32:38.365 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909352/data/mon.c.tgz 2024-09-17T21:32:38.365 DEBUG:teuthology.orchestra.run.smithi129:> mktemp 2024-09-17T21:32:38.420 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.qJbotB0V8c 2024-09-17T21:32:38.421 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.qJbotB0V8c 2024-09-17T21:32:38.589 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.qJbotB0V8c 2024-09-17T21:32:38.672 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.qJbotB0V8c is 527KB 2024-09-17T21:32:38.769 DEBUG:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.qJbotB0V8c 2024-09-17T21:32:38.784 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909352/data/mon.b.tgz 2024-09-17T21:32:38.784 DEBUG:teuthology.orchestra.run.smithi171:> mktemp 2024-09-17T21:32:38.802 INFO:teuthology.orchestra.run.smithi171.stdout:/tmp/tmp.TT23axkRtk 2024-09-17T21:32:38.803 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.TT23axkRtk 2024-09-17T21:32:38.990 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0666 /tmp/tmp.TT23axkRtk 2024-09-17T21:32:39.073 DEBUG:teuthology.orchestra.remote:smithi171:/tmp/tmp.TT23axkRtk is 527KB 2024-09-17T21:32:39.142 DEBUG:teuthology.orchestra.run.smithi171:> rm -fr /tmp/tmp.TT23axkRtk 2024-09-17T21:32:39.157 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-17T21:32:39.157 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T21:32:39.160 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T21:32:39.238 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T21:32:39.277 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T21:32:39.277 INFO:tasks.ceph:Archiving crash dumps... 2024-09-17T21:32:39.279 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909352/remote/smithi129/crash 2024-09-17T21:32:39.280 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T21:32:39.304 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909352/remote/smithi171/crash 2024-09-17T21:32:39.305 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T21:32:39.342 INFO:tasks.ceph:Compressing logs... 2024-09-17T21:32:39.342 DEBUG:teuthology.orchestra.run.smithi129:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T21:32:39.349 DEBUG:teuthology.orchestra.run.smithi171:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T21:32:39.370 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-17T21:32:39.370 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50720.log 2024-09-17T21:32:39.370 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56401.log 2024-09-17T21:32:39.424 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-17T21:32:39.424 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33026.log 2024-09-17T21:32:39.424 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-17T21:32:39.424 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-client.admin.33026.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T21:32:39.425 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33026.log.gz 2024-09-17T21:32:39.425 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33277.log 2024-09-17T21:32:39.425 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33157.log 2024-09-17T21:32:39.425 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.33277.log: /var/log/ceph/ceph-client.admin.33197.log 2024-09-17T21:32:39.425 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33277.log.gz 2024-09-17T21:32:39.425 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.33157.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.33157.log.gz -- 2024-09-17T21:32:39.425 INFO:teuthology.orchestra.run.smithi171.stderr: /var/log/ceph/ceph-client.admin.33117.log 2024-09-17T21:32:39.425 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.33197.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-17T21:32:39.425 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33197.log.gz 2024-09-17T21:32:39.425 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.33117.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.33117.log.gz 2024-09-17T21:32:39.426 INFO:teuthology.orchestra.run.smithi171.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-17T21:32:39.426 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T21:32:39.426 INFO:teuthology.orchestra.run.smithi171.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T21:32:39.426 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-17T21:32:39.426 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33237.log 2024-09-17T21:32:39.426 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32986.log 2024-09-17T21:32:39.426 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.33237.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33237.log.gz 2024-09-17T21:32:39.426 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.32986.log: 27.2% -- replaced with /var/log/ceph/ceph-client.admin.32986.log.gz 2024-09-17T21:32:39.426 INFO:teuthology.orchestra.run.smithi171.stderr: 84.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T21:32:39.426 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-17T21:32:39.426 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33068.log 2024-09-17T21:32:39.426 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.33068.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33068.log.gz 2024-09-17T21:32:39.426 INFO:teuthology.orchestra.run.smithi171.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-17T21:32:39.481 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37808.log 2024-09-17T21:32:39.482 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50720.log.gz 2024-09-17T21:32:39.482 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57955.log 2024-09-17T21:32:39.482 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56401.log.gz 2024-09-17T21:32:39.482 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37808.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.37808.log.gz /var/log/ceph/ceph-client.admin.57211.log 2024-09-17T21:32:39.482 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-17T21:32:39.482 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51856.log 2024-09-17T21:32:39.482 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57955.log.gz 2024-09-17T21:32:39.482 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57558.log 2024-09-17T21:32:39.482 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57211.log.gz 2024-09-17T21:32:39.482 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45004.log 2024-09-17T21:32:39.482 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51856.log.gz 2024-09-17T21:32:39.483 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40617.log 2024-09-17T21:32:39.483 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57558.log.gz 2024-09-17T21:32:39.483 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49434.log 2024-09-17T21:32:39.483 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45004.log.gz 2024-09-17T21:32:39.483 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37813.log 2024-09-17T21:32:39.483 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40617.log.gz 2024-09-17T21:32:39.483 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48668.log 2024-09-17T21:32:39.483 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49434.log.gz 2024-09-17T21:32:39.483 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53614.log 2024-09-17T21:32:39.483 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37813.log.gz 2024-09-17T21:32:39.483 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58291.log 2024-09-17T21:32:39.483 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48668.log.gz 2024-09-17T21:32:39.483 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53614.log.gz 2024-09-17T21:32:39.484 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41673.log 2024-09-17T21:32:39.484 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38607.log 2024-09-17T21:32:39.484 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58291.log.gz 2024-09-17T21:32:39.484 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58207.log 2024-09-17T21:32:39.484 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41673.log.gz 2024-09-17T21:32:39.484 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38723.log 2024-09-17T21:32:39.484 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38607.log.gz 2024-09-17T21:32:39.484 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38136.log 2024-09-17T21:32:39.484 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58207.log.gz 2024-09-17T21:32:39.484 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.0.38723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48329.log 2024-09-17T21:32:39.484 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47507.log 2024-09-17T21:32:39.484 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38136.log.gz 2024-09-17T21:32:39.484 INFO:teuthology.orchestra.run.smithi129.stderr: 94.9% -- replaced with /var/log/ceph/ceph-client.0.38723.log.gz 2024-09-17T21:32:39.485 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57787.log 2024-09-17T21:32:39.485 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48329.log.gz 2024-09-17T21:32:39.485 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53672.log 2024-09-17T21:32:39.485 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47507.log.gz 2024-09-17T21:32:39.485 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38439.log 2024-09-17T21:32:39.485 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57787.log.gz 2024-09-17T21:32:39.485 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48956.log 2024-09-17T21:32:39.485 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53672.log.gz 2024-09-17T21:32:39.485 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45240.log 2024-09-17T21:32:39.485 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38439.log.gz 2024-09-17T21:32:39.485 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52497.log 2024-09-17T21:32:39.486 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48956.log.gz 2024-09-17T21:32:39.486 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45240.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-17T21:32:39.486 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45240.log.gz 2024-09-17T21:32:39.486 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56877.log 2024-09-17T21:32:39.486 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52497.log.gz 2024-09-17T21:32:39.486 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57409.log 2024-09-17T21:32:39.486 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56877.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.42844.log 2024-09-17T21:32:39.486 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56877.log.gz 2024-09-17T21:32:39.486 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57409.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.50959.log 2024-09-17T21:32:39.486 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57409.log.gz 2024-09-17T21:32:39.486 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/objectstore_tool.42844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53091.log 2024-09-17T21:32:39.486 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50959.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.49580.log 2024-09-17T21:32:39.487 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50959.log.gz 2024-09-17T21:32:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53091.log.gz 2024-09-17T21:32:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58333.log 2024-09-17T21:32:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56107.log 2024-09-17T21:32:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49580.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49580.log.gz 2024-09-17T21:32:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58333.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58333.log.gz 2024-09-17T21:32:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49822.log 2024-09-17T21:32:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57144.log 2024-09-17T21:32:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56107.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56107.log.gz 2024-09-17T21:32:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49822.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.49822.log.gz 2024-09-17T21:32:39.487 INFO:teuthology.orchestra.run.smithi129.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47648.log 2024-09-17T21:32:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40396.log 2024-09-17T21:32:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57144.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57144.log.gz 2024-09-17T21:32:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47648.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47648.log.gz 2024-09-17T21:32:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47781.log 2024-09-17T21:32:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56149.log 2024-09-17T21:32:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40396.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40396.log.gz 2024-09-17T21:32:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54533.log 2024-09-17T21:32:39.488 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47781.log.gz 2024-09-17T21:32:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56149.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56149.log.gz 2024-09-17T21:32:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38523.log 2024-09-17T21:32:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58501.log 2024-09-17T21:32:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54533.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54533.log.gz 2024-09-17T21:32:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38523.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38523.log.gz 2024-09-17T21:32:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54575.log 2024-09-17T21:32:39.489 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37464.log 2024-09-17T21:32:39.489 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58501.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58501.log.gz 2024-09-17T21:32:39.489 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54575.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54575.log.gz 2024-09-17T21:32:39.489 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38481.log 2024-09-17T21:32:39.489 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58711.log 2024-09-17T21:32:39.489 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37464.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37464.log.gz 2024-09-17T21:32:39.489 INFO:teuthology.orchestra.run.smithi129.stderr: 94.9% -- replaced with /var/log/ceph/objectstore_tool.42844.log.gz 2024-09-17T21:32:39.489 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56695.log 2024-09-17T21:32:39.489 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38481.log.gz 2024-09-17T21:32:39.489 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54636.log 2024-09-17T21:32:39.489 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58711.log.gz 2024-09-17T21:32:39.489 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56695.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56695.log.gz 2024-09-17T21:32:39.489 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58459.log 2024-09-17T21:32:39.490 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54636.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.54636.log.gz 2024-09-17T21:32:39.490 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.45460.log 2024-09-17T21:32:39.490 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39672.log 2024-09-17T21:32:39.490 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58459.log.gz 2024-09-17T21:32:39.490 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45460.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45460.log.gz 2024-09-17T21:32:39.490 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49633.log 2024-09-17T21:32:39.490 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49540.log 2024-09-17T21:32:39.490 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39672.log.gz 2024-09-17T21:32:39.490 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49633.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49633.log.gz 2024-09-17T21:32:39.490 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54332.log 2024-09-17T21:32:39.490 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40096.log 2024-09-17T21:32:39.490 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49540.log.gz 2024-09-17T21:32:39.490 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42131.log 2024-09-17T21:32:39.491 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54332.log.gz 2024-09-17T21:32:39.491 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40096.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40096.log.gz 2024-09-17T21:32:39.491 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40138.log 2024-09-17T21:32:39.491 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47566.log 2024-09-17T21:32:39.491 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42131.log.gz 2024-09-17T21:32:39.491 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48555.log 2024-09-17T21:32:39.491 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40138.log.gz 2024-09-17T21:32:39.491 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47566.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.56874.log -- replaced with /var/log/ceph/ceph-client.admin.47566.log.gz 2024-09-17T21:32:39.491 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-17T21:32:39.491 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50899.log 2024-09-17T21:32:39.491 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48555.log.gz 2024-09-17T21:32:39.491 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56874.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56874.log.gz 2024-09-17T21:32:39.492 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46829.log 2024-09-17T21:32:39.492 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40576.log 2024-09-17T21:32:39.492 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50899.log.gz 2024-09-17T21:32:39.492 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46829.log.gz 2024-09-17T21:32:39.492 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55691.log 2024-09-17T21:32:39.492 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49215.log 2024-09-17T21:32:39.492 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40576.log.gz 2024-09-17T21:32:39.492 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55691.log.gz 2024-09-17T21:32:39.492 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51519.log 2024-09-17T21:32:39.492 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53774.log 2024-09-17T21:32:39.492 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49215.log.gz 2024-09-17T21:32:39.492 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54449.log 2024-09-17T21:32:39.493 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51519.log.gz 2024-09-17T21:32:39.493 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52019.log 2024-09-17T21:32:39.493 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53774.log.gz 2024-09-17T21:32:39.493 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54449.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54449.log.gz 2024-09-17T21:32:39.493 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.43800.log 2024-09-17T21:32:39.493 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49963.log 2024-09-17T21:32:39.493 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52019.log.gz 2024-09-17T21:32:39.493 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/objectstore_tool.43800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55938.log 2024-09-17T21:32:39.493 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49963.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49963.log.gz 2024-09-17T21:32:39.493 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52846.log 2024-09-17T21:32:39.493 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41395.log 2024-09-17T21:32:39.493 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55938.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55938.log.gz 2024-09-17T21:32:39.494 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56873.log 2024-09-17T21:32:39.494 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52846.log.gz 2024-09-17T21:32:39.494 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56876.log 2024-09-17T21:32:39.494 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41395.log.gz 2024-09-17T21:32:39.494 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56873.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56873.log.gz 2024-09-17T21:32:39.494 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39205.log 2024-09-17T21:32:39.494 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55309.log 2024-09-17T21:32:39.494 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56876.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56876.log.gz 2024-09-17T21:32:39.494 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39205.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39205.log.gz 2024-09-17T21:32:39.494 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41693.log 2024-09-17T21:32:39.494 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38055.log 2024-09-17T21:32:39.494 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55309.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55309.log.gz 2024-09-17T21:32:39.494 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41693.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41693.log.gz 2024-09-17T21:32:39.495 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44822.log 2024-09-17T21:32:39.495 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57429.log 2024-09-17T21:32:39.495 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38055.log.gz 2024-09-17T21:32:39.495 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44822.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44822.log.gz 2024-09-17T21:32:39.495 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48031.log 2024-09-17T21:32:39.495 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39148.log 2024-09-17T21:32:39.495 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57429.log.gz 2024-09-17T21:32:39.495 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48031.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48031.log.gz 2024-09-17T21:32:39.495 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40295.log 2024-09-17T21:32:39.495 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57871.log 2024-09-17T21:32:39.495 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39148.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39148.log.gz 2024-09-17T21:32:39.495 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40295.log.gz 2024-09-17T21:32:39.495 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T21:32:39.496 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52796.log 2024-09-17T21:32:39.496 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57871.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57871.log.gz 2024-09-17T21:32:39.496 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58417.log 2024-09-17T21:32:39.496 INFO:teuthology.orchestra.run.smithi129.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.43800.log.gz 2024-09-17T21:32:39.496 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44709.log 2024-09-17T21:32:39.496 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52796.log.gz 2024-09-17T21:32:39.496 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58375.log 2024-09-17T21:32:39.496 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58417.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58417.log.gz 2024-09-17T21:32:39.496 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38565.log 2024-09-17T21:32:39.496 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44709.log.gz 2024-09-17T21:32:39.496 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58375.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58375.log.gz 2024-09-17T21:32:39.496 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37809.log 2024-09-17T21:32:39.496 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46754.log 2024-09-17T21:32:39.497 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38565.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38565.log.gz 2024-09-17T21:32:39.497 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37809.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37809.log.gz 2024-09-17T21:32:39.497 INFO:teuthology.orchestra.run.smithi129.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T21:32:39.497 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37515.log 2024-09-17T21:32:39.497 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56611.log 2024-09-17T21:32:39.497 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46754.log.gz 2024-09-17T21:32:39.497 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37515.log.gz 2024-09-17T21:32:39.497 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52995.log 2024-09-17T21:32:39.497 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56485.log 2024-09-17T21:32:39.497 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56611.log.gz 2024-09-17T21:32:39.497 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52995.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52995.log.gz 2024-09-17T21:32:39.497 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39857.log 2024-09-17T21:32:39.497 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41989.log 2024-09-17T21:32:39.498 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56485.log.gz 2024-09-17T21:32:39.498 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51754.log 2024-09-17T21:32:39.498 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39857.log.gz 2024-09-17T21:32:39.498 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52435.log 2024-09-17T21:32:39.498 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41989.log.gz 2024-09-17T21:32:39.498 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51754.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.51754.log.gz /var/log/ceph/ceph-client.admin.48427.log 2024-09-17T21:32:39.498 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-17T21:32:39.498 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52435.log.gz 2024-09-17T21:32:39.498 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58669.log 2024-09-17T21:32:39.498 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54053.log 2024-09-17T21:32:39.498 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48427.log.gz 2024-09-17T21:32:39.498 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58669.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58669.log.gz 2024-09-17T21:32:39.498 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56569.log 2024-09-17T21:32:39.499 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57153.log 2024-09-17T21:32:39.499 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54053.log.gz 2024-09-17T21:32:39.499 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56569.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.33305.log 2024-09-17T21:32:39.499 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56569.log.gz 2024-09-17T21:32:39.499 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57153.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57153.log.gz -5 2024-09-17T21:32:39.499 INFO:teuthology.orchestra.run.smithi129.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56000.log 2024-09-17T21:32:39.499 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40465.log 2024-09-17T21:32:39.499 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33305.log.gz 2024-09-17T21:32:39.499 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56000.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56000.log.gz 2024-09-17T21:32:39.499 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57997.log 2024-09-17T21:32:39.499 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37641.log 2024-09-17T21:32:39.500 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40465.log.gz 2024-09-17T21:32:39.500 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57997.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57997.log.gz 2024-09-17T21:32:39.500 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56527.log 2024-09-17T21:32:39.500 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51638.log 2024-09-17T21:32:39.500 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37641.log.gz 2024-09-17T21:32:39.500 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52415.log 2024-09-17T21:32:39.500 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56527.log.gz 2024-09-17T21:32:39.500 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51638.log.gz 2024-09-17T21:32:39.500 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39714.log 2024-09-17T21:32:39.500 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37439.log 2024-09-17T21:32:39.500 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52415.log.gz 2024-09-17T21:32:39.500 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39714.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39714.log.gz 2024-09-17T21:32:39.500 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54110.log 2024-09-17T21:32:39.501 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58123.log 2024-09-17T21:32:39.501 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37439.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37439.log.gz 2024-09-17T21:32:39.501 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38669.log 2024-09-17T21:32:39.501 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54110.log.gz 2024-09-17T21:32:39.501 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42518.log 2024-09-17T21:32:39.501 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58123.log.gz 2024-09-17T21:32:39.501 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38669.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38669.log.gz 2024-09-17T21:32:39.501 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50380.log 2024-09-17T21:32:39.501 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42518.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-17T21:32:39.501 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42518.log.gz 2024-09-17T21:32:39.501 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50380.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.50380.log.gz 2024-09-17T21:32:39.501 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57142.log 2024-09-17T21:32:39.502 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33697.log 2024-09-17T21:32:39.502 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57745.log 2024-09-17T21:32:39.502 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57142.log.gz 2024-09-17T21:32:39.502 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33697.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.33697.log.gz 2024-09-17T21:32:39.502 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33607.log 2024-09-17T21:32:39.502 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42871.log 2024-09-17T21:32:39.502 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57745.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57745.log.gz 2024-09-17T21:32:39.502 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33607.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33607.log.gz 2024-09-17T21:32:39.502 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54957.log 2024-09-17T21:32:39.502 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40972.log 2024-09-17T21:32:39.502 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42871.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42871.log.gz 2024-09-17T21:32:39.503 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54957.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54957.log.gz 2024-09-17T21:32:39.503 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38138.log 2024-09-17T21:32:39.503 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.42492.log 2024-09-17T21:32:39.503 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40972.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40972.log.gz 2024-09-17T21:32:39.503 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53453.log 2024-09-17T21:32:39.503 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38138.log.gz 2024-09-17T21:32:39.503 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/objectstore_tool.42492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47467.log 2024-09-17T21:32:39.503 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53453.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53453.log.gz 2024-09-17T21:32:39.503 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51178.log 2024-09-17T21:32:39.503 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37599.log 2024-09-17T21:32:39.503 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47467.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47467.log.gz 2024-09-17T21:32:39.503 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51178.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.51178.log.gz -- 2024-09-17T21:32:39.504 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.51158.log 2024-09-17T21:32:39.504 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38806.log 2024-09-17T21:32:39.504 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37599.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37599.log.gz 2024-09-17T21:32:39.504 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56870.log 2024-09-17T21:32:39.504 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51158.log.gz 2024-09-17T21:32:39.504 INFO:teuthology.orchestra.run.smithi129.stderr: 94.7% -- replaced with /var/log/ceph/objectstore_tool.42492.log.gz 2024-09-17T21:32:39.504 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55891.log 2024-09-17T21:32:39.504 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38806.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38806.log.gz 2024-09-17T21:32:39.504 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56443.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56870.log.gz 2024-09-17T21:32:39.504 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-17T21:32:39.504 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55891.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55891.log.gz 2024-09-17T21:32:39.505 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51240.log 2024-09-17T21:32:39.505 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45720.log 2024-09-17T21:32:39.505 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56443.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56443.log.gz 2024-09-17T21:32:39.505 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51240.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51240.log.gz 2024-09-17T21:32:39.505 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38808.log 2024-09-17T21:32:39.505 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55352.log 2024-09-17T21:32:39.505 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45720.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45720.log.gz 2024-09-17T21:32:39.505 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56868.log 2024-09-17T21:32:39.505 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38808.log.gz 2024-09-17T21:32:39.505 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55352.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.55352.log.gz 2024-09-17T21:32:39.505 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51596.log 2024-09-17T21:32:39.505 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56317.log 2024-09-17T21:32:39.506 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56868.log.gz 2024-09-17T21:32:39.506 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55173.log 2024-09-17T21:32:39.506 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51596.log.gz 2024-09-17T21:32:39.506 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56317.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56317.log.gz 2024-09-17T21:32:39.506 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39608.log 2024-09-17T21:32:39.506 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.55173.log: /var/log/ceph/ceph-mon.a.log 2024-09-17T21:32:39.506 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55173.log.gz 2024-09-17T21:32:39.506 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52238.log 2024-09-17T21:32:39.506 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39608.log.gz 2024-09-17T21:32:39.506 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57555.log 2024-09-17T21:32:39.506 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38139.log 2024-09-17T21:32:39.506 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52238.log.gz 2024-09-17T21:32:39.507 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56275.log 2024-09-17T21:32:39.507 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57555.log.gz 2024-09-17T21:32:39.507 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38139.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38139.log.gz 2024-09-17T21:32:39.507 INFO:teuthology.orchestra.run.smithi129.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57213.log 2024-09-17T21:32:39.507 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56275.log.gz 2024-09-17T21:32:39.507 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37822.log 2024-09-17T21:32:39.507 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56001.log 2024-09-17T21:32:39.507 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57213.log.gz 2024-09-17T21:32:39.507 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57212.log 2024-09-17T21:32:39.507 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37822.log.gz 2024-09-17T21:32:39.507 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56001.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56001.log.gz 2024-09-17T21:32:39.507 INFO:teuthology.orchestra.run.smithi129.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38805.log 2024-09-17T21:32:39.508 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57212.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57212.log.gz 2024-09-17T21:32:39.508 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53276.log 2024-09-17T21:32:39.508 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38805.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38805.log.gz 2024-09-17T21:32:39.508 INFO:teuthology.orchestra.run.smithi129.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48891.log 2024-09-17T21:32:39.508 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53276.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53276.log.gz 2024-09-17T21:32:39.508 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38059.log 2024-09-17T21:32:39.508 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48891.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48891.log.gz 2024-09-17T21:32:39.508 INFO:teuthology.orchestra.run.smithi129.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58081.log 2024-09-17T21:32:39.508 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38059.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38059.log.gz 2024-09-17T21:32:39.508 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47847.log 2024-09-17T21:32:39.508 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58081.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58081.log.gz 2024-09-17T21:32:39.508 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58585.log 2024-09-17T21:32:39.508 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47847.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47847.log.gz 2024-09-17T21:32:39.509 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37818.log 2024-09-17T21:32:39.509 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53194.log 2024-09-17T21:32:39.509 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58585.log.gz 2024-09-17T21:32:39.509 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37818.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37818.log.gz 2024-09-17T21:32:39.509 INFO:teuthology.orchestra.run.smithi129.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38058.log 2024-09-17T21:32:39.509 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53194.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53194.log.gz 2024-09-17T21:32:39.509 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57150.log 2024-09-17T21:32:39.509 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38058.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38058.log.gz 2024-09-17T21:32:39.509 INFO:teuthology.orchestra.run.smithi129.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43804.log 2024-09-17T21:32:39.509 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57150.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57150.log.gz 2024-09-17T21:32:39.509 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37821.log 2024-09-17T21:32:39.509 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43804.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43804.log.gz 2024-09-17T21:32:39.510 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45059.log 2024-09-17T21:32:39.510 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37821.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37821.log.gz 2024-09-17T21:32:39.510 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-17T21:32:39.510 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45059.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45059.log.gz 2024-09-17T21:32:39.510 INFO:teuthology.orchestra.run.smithi129.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57913.log 2024-09-17T21:32:39.510 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50181.log 2024-09-17T21:32:39.510 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57913.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57913.log.gz 2024-09-17T21:32:39.510 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58165.log 2024-09-17T21:32:39.510 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50181.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50181.log.gz 2024-09-17T21:32:39.510 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56875.log 2024-09-17T21:32:39.510 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58165.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37823.log 2024-09-17T21:32:39.510 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58165.log.gz 2024-09-17T21:32:39.510 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57565.log 2024-09-17T21:32:39.511 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56875.log: /var/log/ceph/ceph-client.admin.37823.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-17T21:32:39.511 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37823.log.gz -- replaced with /var/log/ceph/ceph-client.admin.56875.log.gz 2024-09-17T21:32:39.511 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-17T21:32:39.511 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57565.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57565.log.gz 2024-09-17T21:32:39.511 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48830.log 2024-09-17T21:32:39.511 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49235.log 2024-09-17T21:32:39.511 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.48830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48447.log 2024-09-17T21:32:39.511 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48830.log.gz 2024-09-17T21:32:39.511 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49235.log.gz 2024-09-17T21:32:39.511 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57563.log 2024-09-17T21:32:39.512 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48447.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T21:32:39.512 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48447.log.gz 2024-09-17T21:32:39.512 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57563.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57563.log.gz 2024-09-17T21:32:39.512 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56653.log 2024-09-17T21:32:39.512 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46951.log 2024-09-17T21:32:39.512 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56653.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56653.log.gz 2024-09-17T21:32:39.512 INFO:teuthology.orchestra.run.smithi129.stderr: 91.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T21:32:39.512 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37383.log 2024-09-17T21:32:39.512 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39609.log 2024-09-17T21:32:39.512 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46951.log.gz 2024-09-17T21:32:39.513 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37383.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37383.log.gz 2024-09-17T21:32:39.513 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55529.log 2024-09-17T21:32:39.513 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39609.log.gz 2024-09-17T21:32:39.513 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56191.log 2024-09-17T21:32:39.513 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56359.log 2024-09-17T21:32:39.513 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55529.log.gz 2024-09-17T21:32:39.513 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56191.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56191.log.gz 2024-09-17T21:32:39.513 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40915.log 2024-09-17T21:32:39.513 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56359.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56359.log.gz 2024-09-17T21:32:39.513 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41354.log 2024-09-17T21:32:39.513 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37820.log 2024-09-17T21:32:39.514 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40915.log.gz 2024-09-17T21:32:39.514 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41354.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41354.log.gz 2024-09-17T21:32:39.514 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48957.log 2024-09-17T21:32:39.514 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37820.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37820.log.gz 2024-09-17T21:32:39.514 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39797.log 2024-09-17T21:32:39.514 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38397.log 2024-09-17T21:32:39.514 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48957.log.gz 2024-09-17T21:32:39.514 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39797.log.gz 2024-09-17T21:32:39.514 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57569.log 2024-09-17T21:32:39.514 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38397.log.gz 2024-09-17T21:32:39.514 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58039.log 2024-09-17T21:32:39.515 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57829.log 2024-09-17T21:32:39.515 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57569.log.gz 2024-09-17T21:32:39.515 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58039.log.gz 2024-09-17T21:32:39.515 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53840.log 2024-09-17T21:32:39.515 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57829.log.gz 2024-09-17T21:32:39.515 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58249.log 2024-09-17T21:32:39.515 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47708.log 2024-09-17T21:32:39.515 INFO:teuthology.orchestra.run.smithi129.stderr: 28.5% -- replaced with /var/log/ceph/ceph-client.admin.53840.log.gz 2024-09-17T21:32:39.515 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58249.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58249.log.gz 2024-09-17T21:32:39.515 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41156.log 2024-09-17T21:32:39.515 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48128.log 2024-09-17T21:32:39.515 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47708.log.gz 2024-09-17T21:32:39.516 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41156.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41156.log.gz 2024-09-17T21:32:39.516 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49433.log 2024-09-17T21:32:39.516 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48128.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48128.log.gz 2024-09-17T21:32:39.516 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55768.log 2024-09-17T21:32:39.516 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58835.log 2024-09-17T21:32:39.516 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49433.log.gz 2024-09-17T21:32:39.516 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55768.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55768.log.gzgzip 2024-09-17T21:32:39.516 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57152.log 2024-09-17T21:32:39.516 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58835.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58835.log.gz 2024-09-17T21:32:39.516 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43725.log 2024-09-17T21:32:39.516 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37278.log 2024-09-17T21:32:39.517 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57152.log.gz 2024-09-17T21:32:39.517 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43725.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43725.log.gz 2024-09-17T21:32:39.517 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38355.log 2024-09-17T21:32:39.517 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37278.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37278.log.gz 2024-09-17T21:32:39.517 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40992.log 2024-09-17T21:32:39.517 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39547.log 2024-09-17T21:32:39.517 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38355.log.gz 2024-09-17T21:32:39.517 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40992.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40992.log.gz 2024-09-17T21:32:39.517 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54491.log 2024-09-17T21:32:39.517 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58627.log 2024-09-17T21:32:39.517 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39547.log.gz 2024-09-17T21:32:39.518 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54491.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54491.log.gz 2024-09-17T21:32:39.518 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41852.log 2024-09-17T21:32:39.518 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58627.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58627.log.gz 2024-09-17T21:32:39.518 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38137.log 2024-09-17T21:32:39.518 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57566.log 2024-09-17T21:32:39.518 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41852.log.gz 2024-09-17T21:32:39.518 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38137.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38137.log.gz 2024-09-17T21:32:39.518 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55850.log 2024-09-17T21:32:39.518 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57566.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57566.log.gz 2024-09-17T21:32:39.518 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56065.log 2024-09-17T21:32:39.519 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47128.log 2024-09-17T21:32:39.519 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55850.log.gz 2024-09-17T21:32:39.519 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56065.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56065.log.gz 2024-09-17T21:32:39.519 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56869.log 2024-09-17T21:32:39.519 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58543.log 2024-09-17T21:32:39.519 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47128.log.gz 2024-09-17T21:32:39.519 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56869.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56869.log.gz 2024-09-17T21:32:39.519 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37557.log 2024-09-17T21:32:39.519 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47169.log 2024-09-17T21:32:39.519 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58543.log.gz 2024-09-17T21:32:39.519 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37557.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37557.log.gz 2024-09-17T21:32:39.520 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50457.log 2024-09-17T21:32:39.520 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47169.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47169.log.gz 2024-09-17T21:32:39.520 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50081.log 2024-09-17T21:32:39.520 INFO:teuthology.orchestra.run.smithi129.stderr:gzip/var/log/ceph/ceph-client.admin.50457.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49498.log 2024-09-17T21:32:39.520 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50457.log.gz 2024-09-17T21:32:39.520 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50081.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50081.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58753.log 2024-09-17T21:32:39.520 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-17T21:32:39.520 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49498.log.gz 2024-09-17T21:32:39.520 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54914.log 2024-09-17T21:32:39.520 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45393.log 2024-09-17T21:32:39.520 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58753.log.gz 2024-09-17T21:32:39.521 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54914.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54914.log.gz 2024-09-17T21:32:39.521 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39450.log 2024-09-17T21:32:39.521 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45393.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.42202.log 2024-09-17T21:32:39.521 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45393.log.gz 2024-09-17T21:32:39.521 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39450.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39450.log.gz 2024-09-17T21:32:39.521 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-17T21:32:39.521 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/objectstore_tool.42202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45699.log 2024-09-17T21:32:39.521 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.3.log: 94.5% -- replaced with /var/log/ceph/objectstore_tool.42202.log.gz 2024-09-17T21:32:39.521 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27469.log 2024-09-17T21:32:39.521 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45699.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45699.log.gz 2024-09-17T21:32:39.521 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52076.log 2024-09-17T21:32:39.521 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph.tmp-client.admin.27469.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27469.log.gz 2024-09-17T21:32:39.522 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38173.log 2024-09-17T21:32:39.522 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52076.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52076.log.gz 2024-09-17T21:32:39.522 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58795.log 2024-09-17T21:32:39.522 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38173.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38173.log.gz 2024-09-17T21:32:39.522 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50777.log 2024-09-17T21:32:39.522 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58795.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58795.log.gz 2024-09-17T21:32:39.522 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49033.log 2024-09-17T21:32:39.522 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50777.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50777.log.gz 2024-09-17T21:32:39.522 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56233.log 2024-09-17T21:32:39.522 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49033.log.gz 2024-09-17T21:32:39.522 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56233.log.gz 2024-09-17T21:32:39.523 INFO:teuthology.orchestra.run.smithi129.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-17T21:32:39.538 INFO:teuthology.orchestra.run.smithi129.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-17T21:32:39.570 INFO:teuthology.orchestra.run.smithi171.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-17T21:32:39.663 INFO:teuthology.orchestra.run.smithi129.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-17T21:32:39.875 INFO:teuthology.orchestra.run.smithi129.stderr: 94.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-17T21:32:40.248 INFO:teuthology.orchestra.run.smithi171.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-17T21:32:40.381 INFO:teuthology.orchestra.run.smithi171.stderr: 94.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-17T21:32:40.778 INFO:teuthology.orchestra.run.smithi129.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-17T21:32:42.072 INFO:teuthology.orchestra.run.smithi171.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-17T21:32:42.320 INFO:teuthology.orchestra.run.smithi171.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-17T21:32:42.321 INFO:teuthology.orchestra.run.smithi171.stderr: 2024-09-17T21:32:42.321 INFO:teuthology.orchestra.run.smithi171.stderr:real 0m2.917s 2024-09-17T21:32:42.322 INFO:teuthology.orchestra.run.smithi171.stderr:user 0m7.233s 2024-09-17T21:32:42.322 INFO:teuthology.orchestra.run.smithi171.stderr:sys 0m0.299s 2024-09-17T21:32:42.419 INFO:teuthology.orchestra.run.smithi129.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-17T21:32:42.926 INFO:teuthology.orchestra.run.smithi129.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-17T21:32:42.928 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-17T21:32:42.928 INFO:teuthology.orchestra.run.smithi129.stderr:real 0m3.567s 2024-09-17T21:32:42.928 INFO:teuthology.orchestra.run.smithi129.stderr:user 0m8.556s 2024-09-17T21:32:42.928 INFO:teuthology.orchestra.run.smithi129.stderr:sys 0m0.463s 2024-09-17T21:32:42.928 INFO:tasks.ceph:Archiving logs... 2024-09-17T21:32:42.929 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909352/remote/smithi129/log 2024-09-17T21:32:42.930 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T21:32:44.205 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909352/remote/smithi171/log 2024-09-17T21:32:44.206 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T21:32:45.126 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T21:32:45.135 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T21:32:45.135 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T21:32:45.138 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T21:32:45.215 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T21:32:45.223 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T21:32:45.223 DEBUG:teuthology.orchestra.run.smithi129:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T21:32:45.226 DEBUG:teuthology.orchestra.run.smithi171:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T21:32:45.241 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-09-17T21:32:45.242 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-09-17T21:32:45.246 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T21:32:45.246 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-09-17T21:32:45.246 INFO:teuthology.orchestra.run.smithi129.stdout:^* hv01.front.sepia.ceph.com 3 6 377 62 -598us[ -598us] +/- 43ms 2024-09-17T21:32:45.246 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 -127us[ -127us] +/- 58ms 2024-09-17T21:32:45.246 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +1387us[+1387us] +/- 82ms 2024-09-17T21:32:45.246 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:32:45.247 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T21:32:45.247 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-09-17T21:32:45.248 INFO:teuthology.orchestra.run.smithi171.stdout:^* hv01.front.sepia.ceph.com 3 6 377 60 -596us[ -586us] +/- 43ms 2024-09-17T21:32:45.248 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 60 -122us[ -122us] +/- 58ms 2024-09-17T21:32:45.248 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 +1393us[+1403us] +/- 82ms 2024-09-17T21:32:45.248 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:32:45.248 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T21:32:45.256 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T21:32:45.257 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T21:32:45.265 DEBUG:teuthology.orchestra.run.smithi129:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T21:32:45.368 DEBUG:teuthology.orchestra.run.smithi171:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T21:32:45.443 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T21:32:45.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:32:45.474 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T21:32:45.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:32:45.503 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T21:32:45.512 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T21:32:45.561 INFO:teuthology.task.internal:Duration was 617.773351 seconds 2024-09-17T21:32:45.561 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T21:32:45.569 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T21:32:45.569 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T21:32:45.571 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T21:32:45.610 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T21:32:45.611 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T21:32:45.968 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T21:32:45.968 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T21:32:45.968 DEBUG:teuthology.orchestra.run.smithi129:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T21:32:46.029 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T21:32:46.029 DEBUG:teuthology.orchestra.run.smithi171:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T21:32:46.051 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T21:32:46.051 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T21:32:46.074 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T21:32:46.117 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T21:32:46.117 DEBUG:teuthology.orchestra.run.smithi129:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T21:32:46.142 DEBUG:teuthology.orchestra.run.smithi171:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T21:32:46.271 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T21:32:46.281 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T21:32:46.282 DEBUG:teuthology.orchestra.run.smithi129:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T21:32:46.308 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T21:32:46.334 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T21:32:46.360 DEBUG:teuthology.orchestra.run.smithi129:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T21:32:46.363 DEBUG:teuthology.orchestra.run.smithi171:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T21:32:46.383 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = core 2024-09-17T21:32:46.397 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = core 2024-09-17T21:32:46.418 DEBUG:teuthology.orchestra.run.smithi129:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T21:32:46.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:32:46.473 DEBUG:teuthology.orchestra.run.smithi171:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T21:32:46.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:32:46.489 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T21:32:46.499 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T21:32:46.500 DEBUG:teuthology.misc:Transferring archived files from smithi129:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909352/remote/smithi129 2024-09-17T21:32:46.500 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T21:32:46.554 DEBUG:teuthology.misc:Transferring archived files from smithi171:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_20:16:19-rbd-main-distro-default-smithi/7909352/remote/smithi171 2024-09-17T21:32:46.555 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T21:32:46.588 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T21:32:46.588 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T21:32:46.590 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T21:32:46.638 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T21:32:46.648 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T21:32:46.648 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T21:32:46.657 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T21:32:46.658 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T21:32:46.660 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T21:32:46.673 INFO:teuthology.orchestra.run.smithi129.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 21:32 /home/ubuntu/cephtest 2024-09-17T21:32:46.694 INFO:teuthology.orchestra.run.smithi171.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 21:32 /home/ubuntu/cephtest 2024-09-17T21:32:46.695 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T21:32:46.704 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T21:32:46.729 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} duration: 617.7733507156372 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-17T21:32:46.729 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T21:32:46.820 INFO:teuthology.run:pass